Matching numbers with regular expressions — only digits and commas
I can't figure out how to construct a regex for the example values:
123,456,789
-12,34
1234
-8
Could you help me?
Answer
What’s a Number?
I have a simple question for your “simple” question: What precisely do you mean by “a number”?
- Is
−0a number? - How do you feel about
√−1? - Is
⅝or⅔a number? - Is
186,282.42±0.02miles/second one number — or is it two or three of them? - Is
6.02e23a number? - Is
3.141_592_653_589a number? How aboutπ, orℯ? And−2π⁻³ ͥ? - How many numbers in
0.083̄? - How many numbers in
128.0.0.1? - What number does
⚄hold? How about⚂⚃? - Does
10,5 mmhave one number in it — or does it have two? - Is
∛8³a number — or is it three of them? - What number does
ↀↀⅮⅭⅭⅬⅫ AUCrepresent, 2762 or 2009? - Are
४५६७and৭৮৯৮numbers? - What about
0377,0xDEADBEEF, and0b111101101? - Is
Infa number? IsNaN? - Is
④②a number? What about⓰? - How do you feel about
㊅? - What do
ℵ₀andℵ₁have to do with numbers? Orℝ,ℚ, andℂ?
Suggested Patterns
Also, are you familiar with these patterns? Can you explain the pros and cons of each?
/\D//^\d+$//^\p{Nd}+$//^\pN+$//^\p{Numeric_Value:10}$//^\P{Numeric_Value:NaN}+$//^-?\d+$//^[+-]?\d+$//^-?\d+\.?\d*$//^-?(?:\d+(?:\.\d*)?|\.\d+)$//^([+-]?)(?=\d|\.\d)\d*(\.\d*)?([Ee]([+-]?\d+))?$//^((\d)(?(?=(\d))|$)(?(?{ord$3==1+ord$2})(?1)|$))$//^(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))$//^(?:(?:[0-9a-fA-F]{1,2}):(?:[0-9a-fA-F]{1,2}):(?:[0-9a-fA-F]{1,2}):(?:[0-9a-fA-F]{1,2}):(?:[0-9a-fA-F]{1,2}):(?:[0-9a-fA-F]{1,2}))$//^(?:(?:[+-]?)(?:[0123456789]+))$//(([+-]?)([0123456789]{1,3}(?:,?[0123456789]{3})*))//^(?:(?:[+-]?)(?:[0123456789]{1,3}(?:,?[0123456789]{3})*))$//^(?:(?i)(?:[+-]?)(?:(?=[0123456789]|[.])(?:[0123456789]*)(?:(?:[.])(?:[0123456789]{0,}))?)(?:(?:[E])(?:(?:[+-]?)(?:[0123456789]+))|))$//^(?:(?i)(?:[+-]?)(?:(?=[01]|[.])(?:[01]{1,3}(?:(?:[,])[01]{3})*)(?:(?:[.])(?:[01]{0,}))?)(?:(?:[E])(?:(?:[+-]?)(?:[01]+))|))$//^(?:(?i)(?:[+-]?)(?:(?=[0123456789ABCDEF]|[.])(?:[0123456789ABCDEF]{1,3}(?:(?:[,])[0123456789ABCDEF]{3})*)(?:(?:[.])(?:[0123456789ABCDEF]{0,}))?)(?:(?:[G])(?:(?:[+-]?)(?:[0123456789ABCDEF]+))|))$//((?i)([+-]?)((?=[0123456789]|[.])([0123456789]{1,3}(?:(?:[_,]?)[0123456789]{3})*)(?:([.])([0123456789]{0,}))?)(?:([E])(([+-]?)([0123456789]+))|))/
I suspect that some of those patterns above may serve your needs. But I cannot tell you which one or ones — or, if none, supply you another — because you haven’t said what you mean by “number”.
As you see, there are a huge number of number possibilities: quite probably ℵ₁ worth of them, in fact. ☺
Key to Suggested Patterns
Each numbered explanation listed below describes the pattern of the corresponding numbered pattern listed above.
- Match if there are any non-digits anywhere in the string, including whitespace like line breaks.
- Match only if the string contains nothing but digits, with the possible exception of a trailing line break. Note that a digit is defined as having the property General Category Decimal Number, which is available as
\p{Nd},\p{Decimal_Number}, or\p{General_Category=Decimal_Number}. This is turn is actually just a reflection of those code points whose Numeric Type category is Decimal, which is available as\p{Numeric_Type=Decimal}. - This is the same as 2 in most regex languages. Java is an exception here, because it does not map the simple charclass escapes like
\wand\W,\dand\D,\sand\S, and\bor\Binto the appropriate Unicode property. That means you must not use any of those eight one-character escapes for any Unicode data in Java, because they work only on ASCII even though Java always uses Unicode characters internally. - This is slightly different from 3 in that it isn’t limited to decimal numbers, but can be any number at all; that is,
any character with the
\pN,\p{Number}, or\p{General_Category=Number}property. These include\p{Nl}or\p{Letter_Number}for things like Roman numerals and\p{No}or\p{Other_Number}for subscripted and subscripted numbers, fractions, and circled numbers — amongst others, like counting rods. - This matches only those strings composed entirely of numbers whose decimal value is 10, so things like
Ⅹthe Roman numeral ten, and⑩,⑽,⒑,⓾,❿,➉, and➓. - Only those strings that contain characters that lack the Numeric Value NaN; in other words, all chars must have some numeric value.
- Matches only Decimal Numbers, optionally with a leading HYPHEN MINUS.
- Same as 7 but now also works if the sign is plus instead of minus.
- Looks for decimal numbers, with optional HYPHEN MINUS and optional FULL STOP plus zero or more decimal numbers following.
- Same as 9 but doesn't require digits before the dot if it has some afterwards.
- Standard floating-point notation per C and many other languages, allowing for scientific notation.
- Finds numbers composed only of two or more decimals of any script in descending order, like 987 or 54321. This recursive regex includes a callout to Perl code that checks whether the look ahead digit has a code point value that is the successor of the current digit; that is, its ordinal value is one greater. One could do this in PCRE using a C function as the callout.
- This looks for a valid IPv4 address with four decimal numbers in the valid range, like 128.0.0.1 or 255.255.255.240, but not 999.999.999.999.
- This looks for a valid MAC addr, so six colon-separate pairs of two ASCII hex digits.
- This looks for whole numbers in the ASCII range with an optional leading sign. This is the normal pattern for matching ASCII integers.
- This is like 15, except that it requires a comma to separate groups of three.
- This is like 15, except that the comma for separating groups is now optional.
- This is the normal pattern for matching C-style floating-point numbers in ASCII.
- This is like 18, but requiring a comma to separate groups of 3 and in base-2 instead of in base-10.
- This is like 19, but in hex. Note that the optional exponent is now indicated by a G instead of an E, since E is a valid hex digit.
- This checks that the string contains a C-style floating-point number, but with an optional grouping separator every three digits of either a comma or an underscore (LOW LINE) between them. It also stores that string into the
\1capture group, making available as$1after the match succeeds.
Sources and Maintainability
Patterns number 1,2,7–11 come from a previous incarnation of the Perl Frequently Asked Questions list in the question, “How do I validate input?”. That section has been replaced by a suggestion to use the Regexp::Common module, written by Abigail and Damian Conway. The original patterns can still be found in Recipe 2.1 of the Perl Cookbook, “Checking Whether a String Is a Valid Number”, solutions to which can be found for a dizzying number of diverse languages, including ada, common lisp, groovy, guile, haskell, java, merd, ocaml, php, pike, python, rexx, ruby, and tcl at the the PLEAC project.
Pattern 12 could be more legibly rewritten
m{
^
(
( \d )
(?(?= ( \d ) ) | $ )
(?(?{ ord $3 == 1 + ord $2 }) (?1) | $ )
)
$
}x
It uses regex recursion, which is found in many pattern engines, including Perl and all the PCRE-derived languages. But it also uses an embedded code callout as the test of its second conditional pattern; to my knowledge, code callouts are available only in Perl and PCRE.
Patterns 13–21 were derived from the aforementioned Regexp::Common module. Note that for brevity, these are all written without the whitespace and comments that you would definitely want in production code. Here is how that might look in /x mode:
$real_rx = qr{ ( # start $1 to hold entire pattern
( [+-]? ) # optional leading sign, captured into $2
( # start $3
(?= # look ahead for what next char *will* be
[0123456789] # EITHER: an ASCII digit
| [.] # OR ELSE: a dot
) # end look ahead
( # start $4
[0123456789]{1,3} # 1-3 ASCII digits to start the number
(?: # then optionally followed by
(?: [_,]? ) # an optional grouping separator of comma or underscore
[0123456789]{3} # followed by exactly three ASCII digits
) * # repeated any number of times
) # end $4
(?: # begin optional cluster
( [.] ) # required literal dot in $5
( [0123456789]{0,} ) # then optional ASCII digits in $6
) ? # end optional cluster
) # end $3
(?: # begin cluster group
( [E] ) # base-10 exponent into $7
( # exponent number into $8
( [+-] ? ) # optional sign for exponent into $9
( [0123456789] + ) # one or more ASCII digits into $10
) # end $8
| # or else nothing at all
) # end cluster group
) }xi; # end $1 and whole pattern, enabling /x and /i modes
From a software engineering perspective, there are still several issues with the style used in the /x mode version immediately above. First, there is a great deal of code repetition, where you see the same [0123456789]; what happens if one of those sequences accidentally leaves a digit out? Second, you are relying on positional parameters, which you must count. That means you might write something like:
(
$real_number, # $1
$real_number_sign, # $2
$pre_exponent_part, # $3
$pre_decimal_point, # $4
$decimal_point, # $5
$post_decimal_point, # $6
$exponent_indicator, # $7
$exponent_number, # $8
$exponent_sign, # $9
$exponent_digits, # $10
) = ($string =~ /$real_rx/);
which is frankly abominable! It is easy to get the numbering wrong, hard to remember what symbolic names go where, and tedious to write, especially if you don’t need all those pieces. Rewriting that to used named groups instead of just numbered ones. Again, I’ll use Perl syntax for the variables, but the contents of the Pattern should work anywhere that named groups are supported.
use 5.010; # Perl got named patterns in 5.10
$real_rx = qr{
(?<real_number>
# optional leading sign
(?<real_number_sign> [+-]? )
(?<pre_exponent_part>
(?= # look ahead for what next char *will* be
[0123456789] # EITHER: an ASCII digit
| [.] # OR ELSE: a dot
) # end look ahead
(?<pre_decimal_point>
[0123456789]{1,3} # 1-3 ASCII digits to start the number
(?: # then optionally followed by
(?: [_,]? ) # an optional grouping separator of comma or underscore
[0123456789]{3} # followed by exactly three ASCII digits
) * # repeated any number of times
) # end <pre_decimal_part>
(?: # begin optional anon cluster
(?<decimal_point> [.] ) # required literal dot
(?<post_decimal_point>
[0123456789]{0,} )
) ? # end optional anon cluster
) # end <pre_exponent_part>
# begin anon cluster group:
(?:
(?<exponent_indicator> [E] ) # base-10 exponent
(?<exponent_number> # exponent number
(?<exponent_sign> [+-] ? )
(?<exponent_digits> [0123456789] + )
) # end <exponent_number>
| # or else nothing at all
) # end anon cluster group
) # end <real_number>
}xi;
Now the abstractions are named, which helps. You can pull the groups out by name, and you only need the ones you care about. For example:
if ($string =~ /$real_rx/) {
($pre_exponent, $exponent_number) =
@+{ qw< pre_exponent exponent_number > };
}
There’s one more thing to do this pattern to make it still more maintainable. The problem is that there’s still too much repetition, which means it’s too easily changed in one place but not in another. If you were doing a McCabe analysis, you would say its complexity metric is too high. Most of us would just say it’s too indented. This makes it hard to follow. To fix all these things, what we need is a “grammatical pattern”, one with a definition block to create named abstractions, which we then treat somewhat like a subroutine call later on in the match.
use 5.010; # Perl first got regex subs in v5.10
$real__rx = qr{
^ # anchor to front
(?&real_number) # call &real_number regex sub
$ # either at end or before final newline
##################################################
# the rest is definition only; think of ##
# each named buffer as declaring a subroutine ##
# by that name ##
##################################################
(?(DEFINE)
(?<real_number>
(?&mantissa)
(?&abscissa) ?
)
(?<abscissa>
(?&exponent_indicator)
(?&exponent)
)
(?<exponent>
(&?sign) ?
(?&a_digit) +
)
(?<mantissa>
# expecting either of these....
(?= (?&a_digit)
| (?&point)
)
(?&a_digit) {1,3}
(?: (?&digit_separator) ?
(?&a_digit) {3}
) *
(?: (?&point)
(?&a_digit) *
) ?
)
(?<point> [.] )
(?<sign> [+-] )
(?<digit_separator> [_,] )
(?<exponent_indicator> [Ee] )
(?<a_digit> [0-9] )
) # end DEFINE block
}x;
See how insanely better the grammatical pattern is than the original line-noisy pattern? It’s also far easier to get the syntax right: I typed that in without even one regex syntax error that needed correcting. (OK fine, I typed all the others in without any syntax errors either, but I've been doing this for a while. :)
Grammatical patterns look much more like a BNF than the ugly old regular expressions that people have come to hate. They are far easier to read, write, and maintain. So let’s have no more ugly patterns, OK?