qDebug isn't printing a full QByteArray containing binary data

Question

I have a QByteArray to store data received from a GPS, which is part binary and part ASCII. I want to know for debug proposals know what's being received, so I'm writing a qDebug like this:

//QByteArray buffer;
//...
qDebug() << "GNSS msg (" << buffer.size() << "): " << buffer;

And I get messages like this at console:

GNSS msg ( 1774 ): "ygnnsdgk...(many data)..PR085hlHJGOLH
(more data into a new line, which is OK because it is a new GNSS sentence and
probably has a \n at the end of each one) blablabla...

But suddenly I get a new print iteration. Data has not been erased yet, it has been appended. So new message size its for example 3204, bigger than the previous print obviously. But it prints exactly the same (but with the new size 3204 between brackets). No new data is printed, just the same as the previous message had:

GNSS msg ( 3204 ): "ygnnsdgk...(many data)..PR085hlHJGOLH
(more data into a new line, which is OK because it is a new GNSS sentence and
probably has a \n at the end of each one) blablabla...

I guess qDebug stops printing because it has a limit, or because it reaches a terminating character or something like that, but I'm only guessing.

Any help or explanation for this behaviour?

Answer 1

Solution / workaround:

Indeed, the qDebug() output of QByteArray gets truncated at a '\0' character. This doesn't have something to do with the QByteArray; you even can't ever output a '\0' character using qDebug(). For an explanation see below.

QByteArray buffer;
buffer.append("hello");
buffer.append('\0');
buffer.append("world");

qDebug() << "GNSS msg (" << buffer.size() << "): " << buffer;

Output:

GNSS msg ( 11 ):  "hello

Even any following arguments are ignored:

qDebug() << "hello" << '\0' << "world";

Output:

hello

You can work around this "problem" by replacing the special characters in your byte array before debugging them:

QByteArray dbg = buffer;   // create a copy to not alter the buffer itself
dbg.replace('\\', "\\\\"); // escape the backslash itself
dbg.replace('\0', "\\0");  // get rid of 0 characters
dbg.replace('"', "\\\"");  // more special characters as you like

qDebug() << "GNSS msg (" << buffer.size() << "): " << dbg; // not dbg.size()!

Output:

GNSS msg ( 11 ):  "hello\0world" 

---

So why is this happening? Why can't I output a '\0' using qDebug()?

Let's dive into the Qt internal code to find out what qDebug() does. The following code snippets are from the Qt 4.8.0 source code.

This method is called when you do qDebug() << buffer:

inline QDebug &operator<<(const QByteArray & t) {
    stream->ts  << '\"' << t << '\"'; return maybeSpace();
}

The stream->ts above is of type QTextStream, which converts the QByteArray into a QString:

QTextStream &QTextStream::operator<<(const QByteArray &array)
{
    Q_D(QTextStream);
    CHECK_VALID_STREAM(*this);
    // Here, Qt constructs a QString from the binary data. Until now,
    // the '\0' and following data is still captured.
    d->putString(QString::fromAscii(array.constData(), array.length()));
    return *this;
}

As you can see, d->putString(QString) is called (the type of d is the internal private class of the text stream), which calls write(QString) after doing some padding for constant-width fields. I skip the code of putString(QString) and directly jump into d->write(QString), which is defined like this:

inline void QTextStreamPrivate::write(const QString &data)
{
    if (string) {
        string->append(data);
    } else {
        writeBuffer += data;
        if (writeBuffer.size() > QTEXTSTREAM_BUFFERSIZE)
            flushWriteBuffer();
    }
}

As you can see, the QTextStreamPrivate has a buffer. This buffer is of type QString. So what happens when the buffer is finally printed on the terminal? For this, we have to find out what happens when your qDebug() statement finishes and the buffer is passed to the message handler, which, per default, prints the buffer on the terminal. This is happening in the destructor of the QDebug class, which is defined as follows:

inline ~QDebug() {
   if (!--stream->ref) {
      if(stream->message_output) {
         QT_TRY {
            qt_message_output(stream->type, stream->buffer.toLocal8Bit().data());
         } QT_CATCH(std::bad_alloc&) { /* We're out of memory - give up. */ }
      }
      delete stream;
   }
}

So here is the non-binary-safe part. Qt takes the textual buffer, converts it to "local 8bit" binary representation (until now, AFAIK we should still have the binary data we want to debug).

But then passes it to the message handler without the additional specification of the length of the binary data. As you should know, it is impossible to find out the length of a C-string which should also be able to hold '\0' characters. (That's why QString::fromAscii() in the code above needs the additional length parameter for binary-safety.)

So if you want to handle the '\0' characters, even writing your own message handler will not solve the problem, as you can't know the length. Sad, but true.