What is the equivalent of 'fread' from Matlab in Python?

Question

I have practically no knowledge of Matlab, and need to translate some parsing routines into Python. They are for large files, that are themselves divided into 'blocks', and I'm having difficulty right from the off with the checksum at the top of the file.

What exactly is going on here in Matlab?

status = fseek(fid, 0, 'cof');
fposition = ftell(fid);
disp(' ');
disp(['** Block ',num2str(iBlock),' File Position = ',int2str(fposition)]);

% ----------------- Block Start ------------------ %
[A, count] = fread(fid, 3, 'uint32');
if(count == 3)
    magic_l = A(1);
    magic_h = A(2);
    block_length = A(3);
else
    if(fposition == file_length)
        disp(['** End of file OK']);
    else
        disp(['** Cannot read block start magic !  Note File Length = ',num2str(file_length)]);
    end
    ok = 0;
    break;
end

fid is the file currently being looked at iBlock is a counter for which 'block' you're in within the file

magic_l and magic_h are to do with checksums later, here is the code for that (follows straight from the code above):

disp(sprintf('  Magic_L = %08X, Magic_H = %08X, Length = %i', magic_l, magic_h, block_length));
correct_magic_l = hex2dec('4D445254');
correct_magic_h = hex2dec('43494741');

if(magic_l ~= correct_magic_l | magic_h ~= correct_magic_h)
    disp(['** Bad block start magic !']);
    ok = 0;
    return;
end

remaining_length = block_length - 3*4 - 3*4;   % We read Block Header, and we expect a footer
disp(sprintf('  Remaining Block bytes = %i', remaining_length));

• What is going on with the %08X and the hex2dec stuff?
• Also, why specify 3*4 instead of 12?

Really though, I want to know how to replicate [A, count] = fread(fid, 3, 'uint32'); in Python, as io.readline() is just pulling the first 3 characters of the file. Apologies if I'm missing the point somewhere here. It's just that using io.readline(3) on the file seems to return something it shouldn't, and I don't understand how the block_length can fit in a single byte when it could potentially be very long.

Thanks for reading this ramble. I hope you can understand kind of what I want to know! (Any insight at all is appreciated.)

Answer 1

Python Code for Reading a 1-Dimensional Array

When replacing Matlab with Python, I wanted to read binary data into a numpy.array, so I used numpy.fromfile to read the data into a 1-dimensional array:

import numpy as np

with open(inputfilename, 'rb') as fid:
    data_array = np.fromfile(fid, np.int16)

Some advantages of using numpy.fromfile versus other Python solutions include:

• Not having to manually determine the number of items to be read. You can specify them using the count= argument, but it defaults to -1 which indicates reading the entire file.

• Being able to specify either an open file object (as I did above with fid) or you can specify a filename. I prefer using an open file object, but if you wanted to use a filename, you could replace the two lines above with:

  data_array = numpy.fromfile(inputfilename, numpy.int16)
  

Matlab Code for a 2-Dimensional Array

Matlab's fread has the ability to read the data into a matrix of form [m, n] instead of just reading it into a column vector. For instance, to read data into a matrix with 2 rows use:

fid = fopen(inputfilename, 'r');
data_array = fread(fid, [2, inf], 'int16');
fclose(fid);

Equivalent Python Code for a 2-Dimensional Array

You can handle this scenario in Python using Numpy's shape and transpose.

import numpy as np

with open(inputfilename, 'rb') as fid:
    data_array = np.fromfile(fid, np.int16).reshape((-1, 2)).T

• The -1 tells numpy.reshape to infer the length of the array for that dimension based on the other dimension—the equivalent of Matlab's inf infinity representation.
• The .T transposes the array so that it is a 2-dimensional array with the first dimension—the axis—having a length of 2.