Processing raw data read from serial port with Python serial library?

Question

I am not a Python programmer but am rather electronic circuit designer, however this time I must process some raw data sent by a microcontroller via RS232 port towards Python script (which is called by PHP script).

I've spent quite a few hours trying to determine the best ways of reading raw bytes from serial (RS232) port using Python and I did get the results - but I would like if someone could clarify certain inconsistencies I noticed during researching and here they are:

1:
I can see a lot of people who asked similar question had been asked whether they are using serial or pySerial module and how did they install the serial library. I can only say I don't really know which module I am using as the module worked out-of-the-box. Somewhere I read serial and pySerial is the same thing but I cannot find if that is true. All I know is I am using Python 2.7.9 with Raspbian OS.

2:
I've read there are read() and readline() methods for reading from the serial port but in the pySerial API docs there is no mention of the readline() method. Futhermore, I discovered the 'number of bytes to read' argument can be passed to readline() method as well as to the read() method (and works the same way, limiting the number of bytes to be read) but I cannot find that to be documented.

3:
When searching for how to determine if all of the data from the RS232 buffer has been read I have here found the following code:

read_byte = ser.read()
while read_byte is not None:
    read_byte = ser.read()
    print '%x' % ord(read_byte)

but that results with the:

Traceback (most recent call last):
  File "./testread.py", line 53, in <module>
    read_all()
  File "./testread.py", line 32, in read_all
    print '%x' % ord(read_byte)
TypeError: ord() expected a character, but string of length 0 found

upon reading the last byte from the buffer and I was able to detect the empty buffer only with the following code:

while True:
    c = rs232.read()
    if len(c) == 0:
        break
    print int(c.encode("hex"), 16), " ",

so I am not sure if the code that didn't work for me is for some serial library that is other than mine. My code for openinig port is BTW:

rs232 = serial.Serial(
    port = '/dev/ttyUSB0',
    baudrate = 2400,
    parity = serial.PARITY_NONE,
    stopbits = serial.STOPBITS_ONE,
    bytesize = serial.EIGHTBITS,
    timeout = 1
)

4:
The data I am receiving from µC is in the format:

0x16 0x02 0x0b 0xc9 ... 0x0d 0x0a

That is some raw bytes + \r\n. Since 'raw bytes' can contain 0x00, can someone confirm that is not a problem regarding reading the bytes into the Python string variable? As I understand that should work well but am not 100% sure.

Answer 1

PySerial works for me although haven't used it on a Pi.

3: Read() returns a string - this will be zero length if no data is read, so your later version is correct. As a string is not a character, you should use e.g. ord(read_byte[0]) to print the number corresponding to the first character (if the length of the string >0) Your function:

while True:
    c = rs232.read()
    if len(c) == 0:
        break
    print int(c.encode("hex"), 16), " ",

Needs something adding to accumulate the data read, otherwise it is thrown away

rcvd = ""
while True:
    c = rs232.read()
    if len(c) == 0:
        break
    rcvd += c
    for ch in c:
        print ord(ch), " ",

4: Yes you can receive and put nul (0x00) bytes in a string. For example:

a="\x00"
print len(a)

will print length 1