I have a Sqlite database that contains following type of schema:
termcount(doc_num, term , count)
This table contains terms with their respective counts in the document. like
(doc1 , term1 ,12)
(doc1, term 22, 2)
.
.
(docn,term1 , 10)
This matrix can be considered as sparse matrix as each documents contains very few terms that will have a non-zero value.
How would I create a dense matrix from this sparse matrix using numpy as I have to calculate the similarity among documents using cosine similarity.
This dense matrix will look like a table that have docid as the first column and all the terms will be listed as the first row.and remaining cells will contain counts.
from scipy.sparse import csr_matrix
A = csr_matrix([[1,0,2],[0,3,0]])
>>>A
<2x3 sparse matrix of type '<type 'numpy.int64'>'
with 3 stored elements in Compressed Sparse Row format>
>>> A.todense()
matrix([[1, 0, 2],
[0, 3, 0]])
>>> A.toarray()
array([[1, 0, 2],
[0, 3, 0]])
this is an example of how to convert a sparse matrix to a dense matrix taken from scipy