Batch Combine CSV Remove Header

Ken J picture Ken J · Oct 5, 2012 · Viewed 23.1k times · Source

I have multiple CSV files with the same header and I'm trying to combine them together in Batch and keep only a single header. Any ideas?

Answer

dbenham picture dbenham · Oct 5, 2012

You could use MORE +1 to output all but the 1st line.

>new.csv (
   type file1.csv
   more +1 file2.csv
   more +1 file3.csv
   REM etc.
)

Obviously you can adjust the number of lines to skip in each file as needed.

To combine all csv files in the current folder: Edit: modified to not use newly created output csv as input

@echo off
setlocal
set first=1
>new.csv.tmp (
  for %%F in (*.csv) do (
    if defined first (
      type "%%F"
      set "first="
    ) else more +1 "%%F"
  )
)
ren new.csv.tmp new.csv

Obviously this is only effective if all the csv files share the same format.

EDIT 2015-07-30: There are some limitations:

  • Tab characters will be converted into a string of spaces
  • Each CSV source file must have fewer than 64k lines