Matrix sensing has many real-world applications in science and engineering, such as system control, distance embedding, and computer vision. The goal of matrix sensing is to recover a matrix A(star) is an element of R-nxn, based on a sequence of measurements (u(i), b(i)) is an element of R(n)xR such that u(i)(inverted perpendicular) A(star)u(i) = b(i). Previous work (Zhong et al., 2015) focused on the scenario where matrix A. has a small rank, e.g. rank-k. Their analysis heavily relies on the RIP assumption, making it unclear how to generalize to high-rank matrices. In this paper, we relax that rank-k assumption and solve a much more general matrix sensing problem. Given an accuracy parameter delta is an element of (0, 1), we can compute A is an element of R-nxn in (O) over tilde (m(3/2)n(2)delta(-1)), such that vertical bar u(i)(inverted perpendicular) Au-i - b(i)vertical bar <= delta for all i is an element of [m]. We design an efficient algorithm with provable convergence guarantees using stochastic gradient descent for this problem.