Edit very large sql dump/text file (on linux)

linux mysqldump large-files
geo picture geo · Mar 31, 2009 · Viewed 23.5k times · Source

I have to import a large mysql dump (up to 10G). However the sql dump already predefined with a database structure with index definition. I want to speed up the db insert by removing the index and table definition.

That means I have to remove/edit the first few lines of a 10G text file. What is the most efficient way to do this on linux?

Programs that require loading the entire file into RAM will be an overkill to me.

Answer

rmmh picture rmmh · Mar 31, 2009

Rather than removing the first few lines, try editing them to be whitespace.

The hexedit program can do this-- it reads files in chunks, so opening a 10GB file is no different from opening a 100KB file to it.

$ hexedit largefile.sql.dump
tab (switch to ASCII side)
space (repeat as needed until your header is gone)
F2 (save)/Ctrl-X (save and exit)/Ctrl-C (exit without saving)

Related questions

Too many open files error on Ubuntu 8.04

mysqldump: Couldn't execute 'show fields from `tablename`': Out of resources when opening file './databasename/tablename#P#p125.MYD' (Errcode: 24) (23) on checking the error 24 on the shell it says >>perror 24 OS error code 24: Too many open files …

Read More
How do I find all files containing specific text on Linux?

I'm trying to find a way to scan my entire Linux system for all files containing a specific string of text. Just to clarify, I'm looking for text within the file, not in the file name. When I was looking …

Read More
How to change permissions for a folder and its subfolders/files in one step?

I would like to change permissions of a folder and all its sub folders and files in one step (command) in Linux. I have already tried the below command but it works only for the mentioned folder: chmod 775 /opt/lampp/…

Read More