Python Random Access File

python file file-io large-files random-access
Mantas Vidutis picture Mantas Vidutis · Feb 15, 2011 · Viewed 17.9k times · Source

Is there a Python file type for accessing random lines without traversing the whole file? I need to search within a large file, reading the whole thing into memory wouldn't be possible.

Any types or methods would be appreciated.

Answer

senderle picture senderle · Feb 15, 2011

This seems like just the sort of thing mmap was designed for. A mmap object creates a string-like interface to a file:

>>> f = open("bonnie.txt", "wb")
>>> f.write("My Bonnie lies over the ocean.")
>>> f.close()
>>> f.open("bonnie.txt", "r+b")
>>> mm = mmap(f.fileno(), 0)
>>> print mm[3:9]
Bonnie

In case you were wondering, mmap objects can also be assigned to: 

>>> print mm[24:]
ocean.
>>> mm[24:] = "sea.  "
>>> print mm[:]
My Bonnie lies over the sea.

Related questions

Writing a list to a file with Python

Is this the cleanest way to write a list to a file, since writelines() doesn't insert newline characters? file.writelines(["%s\n" % item for item in list]) It seems like there would be a standard way...

Read More
How to open a file using the open with statement

I'm looking at how to do file input and output in Python. I've written the following code to read a list of names (one per line) from a file into another file while checking a name against the names in …

Read More
Python open() gives FileNotFoundError/IOError: Errno 2 No such file or directory

For some reason my code is having trouble opening a simple file: This is the code: file1 = open('recentlyUpdated.yaml') And the error is: IOError: [Errno 2] No such file or directory: 'recentlyUpdated.yaml' Naturally I checked that this is the …

Read More