We give a classical algorithm for linear regression analogous to the quantum matrix inversion algorithm [Harrow, Hassidim, and Lloyd, Physical Review Letters'09] for low-rank matrices [Wossnig, Zhao, and Prakash, Physical Review Letters'18], when the input matrix A is stored in a data structure applicable for QRAM-based state preparation. Namely, suppose we are given an A 2 is an element of C-mxn with minimum non-zero singular value sigma which supports certain efficient l(2)-norm importance sampling queries, along with a b is an element of C-m. Then, for some x is an element of C-n satisfying parallel to x - A(+)b parallel to <= epsilon parallel to A(+)b parallel to, we can output a measurement of vertical bar x > in the computational basis and output an entry of x with classical algorithms that run in (O) over tilde(parallel to A parallel to(6)(F)parallel to A parallel to(6)/sigma(12)epsilon(4)) and (O) over tilde parallel to A parallel to(6)(F)parallel to A parallel to(2)/sigma(8)epsilon(4)) time, respectively. This improves on previous "quantum-inspired" algorithms in this line of research by at least a factor of parallel to A parallel to(16)/sigma(16)epsilon(2) [Chia, Gilyen, Li, Lin, Tang, and Wang, STOC'20]. As a consequence, we show that quantum computers can achieve at most a factor-of-12 speedup for linear regression in this QRAM data structure setting and related settings. Our work applies techniques from sketching algorithms and optimization to the quantum-inspired literature. Unlike earlier works, this is a promising avenue that could lead to feasible implementations of classical regression in a quantuminspired settings, for comparison against future quantum computers.