I am facing the following problem: I'm trying to implement a simulator for supply chains. These will produce a lot of EPCIS Events (events that occur at RFID readers). These events should then be written to a csv file in order to load them into any database and run analytical algorithms on them.
The simulator is implemented using python and works fine. What I'm now trying to do is buffered writing of the events to the file in order to decrease the time, that is needed to access the disk. Browsing the python documentation I stumbled upon the io.BufferedWriter, which sounds exactly like the thing I was looking for. Anyway, I can't quite get it to work.
Here's what I did so far. I implemented my CsvWriter
class, that inherits from RawIOBase and manages the file handle. When it is instantiated, it will create a BufferedWriter
, handing in itself, as the raw
parameter (might that already be a problem?)
class CsvWriter(AbstractWriter):
def __init__(self, filename):
self.filename = filename
self.file = self.openFile()
self.buffer = BufferedWriter(self, settings.WRITE_THRESHOLD)
When I know want to write something I call write_buffered
buffered, which looks like that:
def write_buffered(self, data_dict):
self.buffer.write(b';'.join(map(str, data_dict.values())) + '\n')
The actual write
methods which (as I figured) needs to be implemented on the CsvWriter
class itself looks like that:
def write(self, data):
if self.file.closed:
self.file = self.openFile()
return self.file.write(data)
The problem is, that when I try to run the simulator, I get the following error:
IOError: raw write() returned invalid length -1 (should have been between 0 and 78)
Do any of you have a clue for me how to fix this?