How split a file in words in unix command line?

unix command-line awk tokenize
jaundavid picture jaundavid · Mar 19, 2013 · Viewed 34.4k times · Source

I'm doing a faster tests for a naive boolean information retrival system, and I would like use awk, grep, egrep, sed or thing similiar and pipes for split a text file into words and save them into other file with a word per line. Example my file cotains:

Hola mundo, hablo español y no sé si escribí bien la
pregunta, ojalá me puedan entender y ayudar
Adiós.

The output file should contain:

Hola
mundo
hablo
español
...

Thank!

Answer

Guru picture Guru · Mar 19, 2013

Using tr:

tr -s '[[:punct:][:space:]]' '\n' < file

Related questions

How can I pretty-print JSON in a shell script?

Is there a (Unix) shell script to format JSON in human-readable form? Basically, I want it to transform the following: { "foo": "lorem", "bar": "ipsum" } ... into something like this: { "foo": "lorem", "bar": "ipsum" }

Read More
grep, but only certain file extensions

I am working on writing some scripts to grep certain directories, but these directories contain all sorts of file types. I want to grep just .h and .cpp for now, but maybe a few others in the future. So far …

Read More
Use grep --exclude/--include syntax to not grep through certain files

I'm looking for the string foo= in text files in a directory tree. It's on a common Linux machine, I have bash shell: grep -ircl "foo=" * In the directories are also many binary files which match "foo=". As these results …

Read More