Split text on paragraphs where paragraph delimiters are non-standard

Renklauf picture Renklauf · Apr 10, 2013 · Viewed 9.5k times · Source

If I have text with standard paragraph formatting (a blank line followed by an indent) such as text 1 it's easy enough to extract the paragraphs using text.split("\n\n").

Text 1:

      Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus sit amet sapien velit, ac sodales   
 ante. Integer mattis eros non turpis interdum et auctor enim consectetur, etc.

      Praesent molestie suscipit bibendum. Donec justo purus, venenatis eget convallis sed, feugiat    
 vitae velit,etc.

But what if I have text with non-standard paragraph formatting such as text 2? No blank lines and variable leading whitespace.

Text 2:

      Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus sit amet sapien velit, ac sodales   
 ante. Integer mattis eros non turpis interdum et auctor enim consectetur, etc.
    Praesent molestie suscipit bibendum. Donec justo purus, venenatis eget convallis sed, feugiat    
 vitae velit,etc.

Since leading whitespace is common to both standard and non-standard formats I've thought about indexing on the regex match for leading whitespace and getting the paragraph breaks that way, but there has to be a more elegant way to do this.

Answer

Ofri Raviv picture Ofri Raviv · Apr 10, 2013

The regex solution you propose seems elegant enough:

re.split('\s{4,}',text)

This uses 4 consecutive whitespace chars as paragraph delimiter. You can use '\n\s{3,}' or something similar, if it fits better.