Speeding up writing images into hard disk in OpenCV

Question

I am working with a 50 fps camera (in Ubuntu environment and Qt framework) and every 20 ms I get a frame to process.

I wrote a code to read images from camera and then store them in hard drive.

while(3.14)
{
 cv::Mat Camera_Image = Capture_Image();
 double T1 = (double)cv::getTickCount();
 cv::imwrite (STORE_ADDRESS,Camera_Image);
 T1 = (((double)cv::getTickCount() -T1)*1000)/cv::getTickFrequency();
 print(T1);
}

when I see the output the time to write a single image into hard disk is around 30 ms for a 2048*1080 image size. each image is single channel (gray scale) but I'm writing them in .jpg format in hard disk. size of each image in hard disk is approximately 500Kbytes.

Since I'm capturing a frame in around 20 ms, I'm not able to write them all into hard disk in real time. I've written my code using Qthread and created a queue to see if there's any improvement but the results were the same and it was only a memory overhead.

Is it possible to change this situation, or use some other library to write these images into hard disk much faster? I would also prefer a Qt solution if available ...

Plus I need to write every single frame into hard disk so please do not propose to use Motion compression algorithms since they don't apply to my situation ....

Code: Mainwindow.cpp

 Qlist<cv::Mat> FINAL_IM_VEC;
MainWindow::MainWindow(QWidget *parent) :
  QMainWindow(parent),
  ui(new Ui::MainWindow)
{
  ui->setupUi(this);

  IMREAD *IMR = new IMREAD(this);   // an instance of IMREAD Class which reads camera frames
  IMWRITE *IMW = new IMWRITE(this);  // an instance of IMWRITE Class which Writes camera frames into hard disk
  QThread *IMAGE_READ_Thread = new QThread(this);
  QThread *Image_Store_Thread = new QThread(this);
  connect(IMAGE_READ_Thread,SIGNAL(started()),IMR,SLOT(IMREAD_Process()));
  connect(Image_Store_Thread,SIGNAL(started()),IMW,SLOT(IMWrite_Process()));
  IMR.moveToThread(IMAGE_READ_Thread);
  IMW.moveToThread(Image_Store_Thread);
  IMAGE_READ_Thread->start();
  Image_Store_Thread->start();
}

imread.hpp

class IMREAD : public QObject
{
    Q_OBJECT
public:
    explicit IMREAD(QObject *parent = 0);

signals:

public slots:
    void IMREAD_Process();
private:
    bool Stop;
};

imread.cpp

IMREAD::IMREAD(QObject *parent) :
    QObject(parent)
{
  this->Stop = false;
}

void IMREAD::IMREAD_Process()
{

  while(!Stop)
    {
          cv::Mat Image = CAM::Campture_F(25);//wait a maximum of 25 milisecond to grab a new frame
          if(Image.data())
            {
          FINAL_IM_VEC.push_back(Image);
            }
      }
    }

}

imwrite.hpp

#ifndef IMWRITE_H
#define IMWRITE_H
#pragma once
#include <QObject>
class IMWRITE : public QObject
{
    Q_OBJECT
public:
    explicit IMWRITE(QObject *parent = 0);
signals:

public slots:
    void IMWrite_Process();
private:
    bool Stop;
};

imwrite.cpp

IMWRITE::IMWRITE(QObject *parent) :
    QObject(parent)
{
  this->Stop =false;
}
void IMWRITE::IMWrite_Process()
{
    static int counter = 0;
    while(!Stop)
      {
        for(int i = 0 ; i < FINAL_IM_VEC.size() ; i++)
            {
                QString address = "/home/Provisioner/ThreadT/Results/" + QString::number(counter++) + ".jpg";
                cv::imwrite(address.toUtf8().constData(),FINAL_IM_VEC[i]);
                FINAL_IM_VEC.erase(FINAL_IM_VEC.begin() + i);
                i--;
            }
      }

}

Since this is just part of the whole project, I've removed some of its irrelevant parts ...but it shows how I wrote my multithreaded code in a big picture... so if there's any thing wrong with please inform me.

Thanks in advance.

Answer 1

Let's see: 2048*1080*3(number of channels)*50 fps ~= 316MB/s, if you were writing the images in raw. If you're using JPEG, depending on the compression parameters you may get a substantial reduction, but if it's 1/5th, you're still writing a lot of data to the harddrive, specially if you're using a 5400rpm on a laptop.

Things you could do:

1. As David Schwartz suggests, you should use queues and multiple threads.
2. If you're effectively writing an image sequence, save a video instead. The data is compressed much more and the writing to disk is faster.
3. Check the specks of your current device and get an estimate of the maximum size of the images that you can write to it. Choose compression parameters to fit that size constraint.