How to parse a CSV file in Bash?

linux bash csv
User1 picture User1 · Nov 26, 2010 · Viewed 182.5k times · Source

I'm working on a long Bash script. I want to read cells from a CSV file into Bash variables. I can parse lines and the first column, but not any other column. Here's my code so far:


  cat myfile.csv|while read line
  do
    read -d, col1 col2 < <(echo $line)
    echo "I got:$col1|$col2"
  done

It's only printing the first column. As an additional test, I tried the following:

read -d, x y < <(echo a,b,)

And $y is empty. So I tried:

read x y < <(echo a b)

And $y is b. Why?

Answer

Paused until further notice. picture Paused until further notice. · Nov 26, 2010

You need to use IFS instead of -d:

while IFS=, read -r col1 col2
do
    echo "I got:$col1|$col2"
done < myfile.csv

Note that for general purpose CSV parsing you should use a specialized tool which can handle quoted fields with internal commas, among other issues that Bash can't handle by itself. Examples of such tools are cvstool and csvkit.

Related questions

Looping through the content of a file in Bash

How do I iterate through each line of a text file with Bash? With this script: echo "Start!" for p in (peptides.txt) do echo "${p}" done I get this output on the screen: Start! ./runPep.sh: line 3: syntax error …

Read More
How to create a file in Linux from terminal window?

What's the easiest way to create a file in Linux terminal?

Read More
How to permanently set $PATH on Linux/Unix?

I'm trying to add a directory to my path so it will always be in my Linux path. I've tried: export PATH=$PATH:/path/to/dir This works, however each time I exit the terminal and start a new terminal …

Read More