Under Python 3, `scipy.sparse.coo_matrix` fails when passed memoryviews.
We coerce the memoryviews to numpy arrays before passing them to the
`coo_matrix`. Even though this is kind of a hack it shouldn't be very
inefficient as no data is actually copied. See
https://github.com/scipy/scipy/issues/5123 for details.