Ruby regex extracting words

ruby regex string word
Shabu picture Shabu · Nov 17, 2011 · Viewed 10k times · Source

I'm currently struggling to come up with a regex that can split up a string into words where words are defined as a sequence of characters surrounded by whitespace, or enclosed between double quotes. I'm using String#scan

For instance, the string:

'   hello "my name" is    "Tom"'

should match the words:

hello
my name
is
Tom

I managed to match the words enclosed in double quotes by using:

/"([^\"]*)"/

but I can't figure out how to incorporate the surrounded by whitespace characters to get 'hello', 'is', and 'Tom' while at the same time not screw up 'my name'.

Any help with this would be appreciated!

Answer

Narendra Yadala picture Narendra Yadala · Nov 17, 2011 
result = '   hello "my name" is    "Tom"'.split(/\s+(?=(?:[^"]*"[^"]*")*[^"]*$)/)

will work for you. It will print 

=> ["", "hello", "\"my name\"", "is", "\"Tom\""]

Just ignore the empty strings.

Explanation

"
\\s            # Match a single character that is a “whitespace character” (spaces, tabs, and line breaks)
   +             # Between one and unlimited times, as many times as possible, giving back as needed (greedy)
(?=           # Assert that the regex below can be matched, starting at this position (positive lookahead)
   (?:           # Match the regular expression below
      [^\"]          # Match any character that is NOT a “\"”
         *             # Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
      \"             # Match the character “\"” literally
      [^\"]          # Match any character that is NOT a “\"”
         *             # Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
      \"             # Match the character “\"” literally
   )*            # Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
   [^\"]          # Match any character that is NOT a “\"”
      *             # Between zero and unlimited times, as many times as possible, giving back as needed (greedy)
   \$             # Assert position at the end of a line (at the end of the string or before a line break character)
)
"

You can use reject like this to avoid empty strings

result = '   hello "my name" is    "Tom"'
            .split(/\s+(?=(?:[^"]*"[^"]*")*[^"]*$)/).reject {|s| s.empty?}

prints

=> ["hello", "\"my name\"", "is", "\"Tom\""]

Related questions

Extract a substring from a string in Ruby using a regular expression

How can I extract a substring from within a string in Ruby? Example: String1 = "<name> <substring>" I want to extract substring from String1 (i.e. everything within the last occurrence of < and >).

Read More
Return first match of Ruby regex

I'm looking for a way to perform a regex match on a string in Ruby and have it short-circuit on the first match. The string I'm processing is long and from what it looks like the standard way (match method) …

Read More
Convert a string to regular expression ruby

I need to convert string like "/[\w\s]+/" to regular expression. "/[\w\s]+/" => /[\w\s]+/ I tried using different Regexp methods like: Regexp.new("/[\w\s]+/") => /\/[w ]+\//, similarly Regexp.compile and Regexp.escape. But none of them returns …

Read More