I have a large matrix (2e6 x 3) which I have to write to a text file.
dlmwrite takes about 230s to achieve this task.
From your experience what is the fastest way to write a large matrix to a text file?
The following applies to MATLAB, but I suggest you try it in Octave. First of all, if you can - transpose the matrix. Here are examples using fprintf
and csvwrite
(essentially dlmwrite
)
A = rand(3, 1e6);
tic;
fid = fopen('data.txt', 'w+');
for i=1:size(A, 1)
fprintf(fid, '%f ', A(i,:));
fprintf(fid, '\n');
end
fclose(fid);
toc
tic;
csvwrite('data.txt', A);
toc;
Elapsed time is 1.311512 seconds.
Elapsed time is 2.487737 seconds.
If not transposed, it will take ages indeed. By default, fprintf
flushes the buffer after every call. You can try to use W
instead of w
to open the file, but it does not improve the situation here too much.