(grep) Regex to match non-ASCII characters?

regex unicode grep ascii non-ascii-characters
Rory picture Rory · Jan 23, 2010 · Viewed 150.5k times · Source

On Linux, I have a directory with lots of files. Some of them have non-ASCII characters, but they are all valid UTF-8. One program has a bug that prevents it working with non-ASCII filenames, and I have to find out how many are affected. I was going to do this with find and then do a grep to print the non-ASCII characters, and then do a wc -l to find the number. It doesn't have to be grep; I can use any standard Unix regular expression, like Perl, sed, AWK, etc.

However, is there a regular expression for 'any character that's not an ASCII character'?

Answer

Alix Axel picture Alix Axel · Jan 23, 2010

This will match a single non-ASCII character:

[^\x00-\x7F]

This is a valid PCRE (Perl-Compatible Regular Expression).

You can also use the POSIX shorthands:

  • [[:ascii:]] - matches a single ASCII char
  • [^[:ascii:]] - matches a single non-ASCII char

[^[:print:]] will probably suffice for you.**

Related questions

How do I grep for all non-ASCII characters?

I have several very large XML files and I'm trying to find the lines that contain non-ASCII characters. I've tried the following: grep -e "[\x{00FF}-\x{FFFF}]" file.xml But this returns every line in the file, regardless …

Read More
JavaScript + Unicode regexes

How can I use Unicode-aware regular expressions in JavaScript? For example, there should be something akin to \w that can match any code-point in Letters or Marks category (not just the ASCII ones), and hopefully have filters like [[P*]] for …

Read More
Concrete Javascript Regex for Accented Characters (Diacritics)

I've looked on Stack Overflow (replacing characters.. eh, how JavaScript doesn't follow the Unicode standard concerning RegExp, etc.) and haven't really found a concrete answer to the question: How can JavaScript match for accented characters (those with diacritical marks)? I'm …

Read More