How to implement the OWASP ESAPI validator with groups of validation attempts in ColdFusion?
I have been playing around with the OWASP ESAPI utilities that are included with ColdFusion 9. ColdFusion's Builtin Enterprise Security API. The encoder utilities are pretty straight forward and I believe I have them working fine. My problem is with the validator utilities.
I can get them to work singly. That is, if I call the validator.getValidInput() method with "invalid" data it will throw an error that I can catch. However, when I try to call the validator method in batch I get a null pointer exception. By batch I mean attempting to execute groups of validation attempts. This is supposed to work by passing the validator.getValidInput() method a ValidationErrorList parameter which should tell it NOT to throw an error but instead just add the error to the error list. I cannot get it to work in this mode. My best attempt is giving me a null pointer exception.
Here is the specific error:
java.lang.NullPointerException
With this stack trace:
java.lang.NullPointerException at
org.owasp.esapi.reference.DefaultValidator.getValidInput(DefaultValidator.java:238) at
sun.reflect.GeneratedMethodAccessor377.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at
java.lang.reflect.Method.invoke(Unknown Source) at
coldfusion.runtime.StructBean.invoke(StructBean.java:536) at
coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:2393) at
cftest2ecfm989071068.runPage(D:\Web\internet\fboc\test.cfm:19) at
coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:231) at
coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:416) at
coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2722) at
cfApplication2ecfc1705903666$funcONREQUEST.runFunction(D:\Web\internet\fboc\Application.cfc:70) at
coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472) at
coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405) at
coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368) at
coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55) at
coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321) at
coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220) at
coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:491) at
coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:337) at
coldfusion.runtime.AppEventInvoker.invoke(AppEventInvoker.java:88) at
coldfusion.runtime.AppEventInvoker.onRequest(AppEventInvoker.java:280) at
coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:356) at
coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48) at
coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40) at
coldfusion.filter.PathFilter.invoke(PathFilter.java:94) at
coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70) at
coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:79) at
coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28) at
coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38) at
coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46) at
coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38) at
coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22) at
coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62) at
coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:126) at
coldfusion.CfmServlet.service(CfmServlet.java:201) at
coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89) at
jrun.servlet.FilterChain.doFilter(FilterChain.java:86) at
coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42) at
coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46) at
jrun.servlet.FilterChain.doFilter(FilterChain.java:94) at
jrun.servlet.FilterChain.service(FilterChain.java:101) at
jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106) at
jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42) at
jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286) at
jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543) at
jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203) at
jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428) at
jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
Here is a simple test script. You will notice that I have one line commented out. That line works without the ErrorList but throws an error (as it should). I am trying to get the method to work without throwing the error:
<cftry>
<cfsilent>
<cfparam name="form.TestField" default="" type="string" />
<cfset Esapi = CreateObject("java", "org.owasp.esapi.ESAPI") />
<cfset EsapiEncoder = Esapi.encoder() />
<cfset EsapiValidator = Esapi.validator() />
<cfset Clean = StructNew() />
<cfset Clean.Css = EsapiEncoder.encodeForCss(form.TestField) />
<cfset Clean.Html = EsapiEncoder.encodeForHtml(form.TestField) />
<cfset Clean.HtmlAttribute = EsapiEncoder.encodeForHtmlAttribute(form.TestField) />
<cfset Clean.JavaScript = EsapiEncoder.encodeForJavaScript(form.TestField) />
<cfset Clean.Url = EsapiEncoder.encodeForUrl(form.TestField) />
<cfset Clean.Xml = EsapiEncoder.encodeForXml(form.TestField) />
<cfset ErrorList = CreateObject("java", "org.owasp.esapi.ValidationErrorList") />
<cfset Valid = StructNew() />
<cfset Valid.Input = EsapiValidator.getValidInput("Test Field", form.TestField, "SafeString", 128, false, true, ErrorList) />
<!---<cfset Valid.Input = EsapiValidator.getValidInput("Test Field", form.TestField, "SafeString", 128, false, true) />--->
</cfsilent>
<!DOCTYPE HTML>
<head>
<meta charset='UTF-8' />
<title>ESAPI Test</title>
</head>
<body>
<div>
<h3>ESAPI Test</h3>
<cfoutput>
<form name="frmtest" id="frmtest" action="#cgi.script_name#" method="post">
<p>Enter text to test:</p>
<p><input type="text" name="TestField" id="TestField" size="64" maxlength="128" value="#Clean.HtmlAttribute#" /></p>
<p><input type="submit" name="submit" id="submit" value=" Submit " /></p>
</form>
</cfoutput>
<hr />
<cfdump var="#Clean#" label="Clean Structure" />
<hr />
<cfdump var="#Valid#" label="Valid Structure" />
</div>
</body>
</html>
<cfcatch type="any">
<hr />
<div>
<h3>ERROR</h3>
<cfdump var="#cfcatch#" label="Error" />
</div>
</cfcatch>
</cftry>
When I run this script with "valid" data it works fine (no errors thrown). If I enter an "invalid" character then I get the null pointer exception.
Example of "valid" data: this is a safe string 0123456789
Example of "invalid" data: this is a safe string 0123456789- (notice the hyphen at the end)
Here is a link to the documentation that shows what I am trying to implement.
For what it's worth, the validation "rules" are defined in the validation.properties file that comes with ColdFusion. That file is located in the {cfusion lib} directory. Here are the contents of that file from my server:
# The ESAPI validator does many security checks on input, such as canonicalization
# and whitelist validation. Note that all of these validation rules are applied *after*
# canonicalization. Double-encoded characters (even with different encodings involved,
# are never allowed.
#
# To use:
#
# First set up a pattern below. You can choose any name you want, prefixed by the word
# "Validation." For example:
# Validation.Email=^[A-Za-z0-9._%-]+@[A-Za-z0-9.-]+\\.[a-zA-Z]{2,4}$
#
# Then you can validate in your code against the pattern like this:
# ESAPI.validator().isValidInput("User Email", input, "Email", maxLength, allowNull);
# Where maxLength and allowNull are set for you needs, respectively.
#
# But note, when you use boolean variants of validation functions, you lose critical
# canonicalization. It is preferable to use the "get" methods (which throw exceptions) and
# and use the returned user input which is in canonical form. Consider the following:
#
# try {
# someObject.setEmail(ESAPI.validator().getValidInput("User Email", input, "Email", maxLength, allowNull));
#
Validator.SafeString=^[.\\p{Alnum}\\p{Space}]{0,1024}$
Validator.Email=^[A-Za-z0-9._%-]+@[A-Za-z0-9.-]+\\.[a-zA-Z]{2,4}$
Validator.IPAddress=^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$
Validator.URL=^(ht|f)tp(s?)\\:\\/\\/[0-9a-zA-Z]([-.\\w]*[0-9a-zA-Z])*(:(0-9)*)*(\\/?)([a-zA-Z0-9\\-\\.\\?\\,\\:\\'\\/\\\\\\+=&%\\$#_]*)?$
Validator.CreditCard=^(\\d{4}[- ]?){3}\\d{4}$
Validator.SSN=^(?!000)([0-6]\\d{2}|7([0-6]\\d|7[012]))([ -]?)(?!00)\\d\\d\\3(?!0000)\\d{4}$
Validator.CFContainerID=^[\\p{Alnum}_\\-\\.:]+$
Validator.GOOGLEMAPAPI=^[\\p{Alnum}_\\+=\\/\\-]+$
Validator.CFFORMSCRIPTSRC=^[^\\*\\?\"'<>|%]*$
I presume that the idea is to add rules to this file for your own applications.
Has anyone gotten the validator.getValidInput() method to work in batch (groups of validation attempts)?
Update 1
I noticed that the following is being written to my cfusion-out.log on the server every time I get the null pointer exception. It leads me to believe that it is working to a point but then gets a null pointer while attempting to assign the validation exception:
06/25 16:08:14 [jrpp-3225] WARN [SECURITY FAILURE Anonymous:null@unknown -> /IntrusionDetector] Invalid input: context=Test Field, type(SafeString)=^[.\p{Alnum}\p{Space}]{0,1024}$, input=this is a safe string 0123456789-
org.owasp.esapi.errors.ValidationException: Test Field: Invalid input. Please conform to regex ^[.\p{Alnum}\p{Space}]{0,1024}$ with a maximum length of 128
at org.owasp.esapi.reference.validation.StringValidationRule.checkWhitelist(StringValidationRule.java:144)
at org.owasp.esapi.reference.validation.StringValidationRule.checkWhitelist(StringValidationRule.java:160)
at org.owasp.esapi.reference.validation.StringValidationRule.getValid(StringValidationRule.java:284)
at org.owasp.esapi.reference.DefaultValidator.getValidInput(DefaultValidator.java:199)
at org.owasp.esapi.reference.DefaultValidator.getValidInput(DefaultValidator.java:236)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at coldfusion.runtime.StructBean.invoke(StructBean.java:508)
at coldfusion.runtime.CfJspPage._invoke(CfJspPage.java:2393)
at cftest2ecfm989071068.runPage(D:\Web\internet\fboc\test.cfm:19)
at coldfusion.runtime.CfJspPage.invoke(CfJspPage.java:231)
at coldfusion.tagext.lang.IncludeTag.doStartTag(IncludeTag.java:416)
at coldfusion.runtime.CfJspPage._emptyTcfTag(CfJspPage.java:2722)
at cfApplication2ecfc1705903666$funcONREQUEST.runFunction(D:\Web\internet\fboc\Application.cfc:70)
at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:472)
at coldfusion.runtime.UDFMethod$ReturnTypeFilter.invoke(UDFMethod.java:405)
at coldfusion.runtime.UDFMethod$ArgumentCollectionFilter.invoke(UDFMethod.java:368)
at coldfusion.filter.FunctionAccessFilter.invoke(FunctionAccessFilter.java:55)
at coldfusion.runtime.UDFMethod.runFilterChain(UDFMethod.java:321)
at coldfusion.runtime.UDFMethod.invoke(UDFMethod.java:220)
at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:491)
at coldfusion.runtime.TemplateProxy.invoke(TemplateProxy.java:337)
at coldfusion.runtime.AppEventInvoker.invoke(AppEventInvoker.java:88)
at coldfusion.runtime.AppEventInvoker.onRequest(AppEventInvoker.java:280)
at coldfusion.filter.ApplicationFilter.invoke(ApplicationFilter.java:356)
at coldfusion.filter.RequestMonitorFilter.invoke(RequestMonitorFilter.java:48)
at coldfusion.filter.MonitoringFilter.invoke(MonitoringFilter.java:40)
at coldfusion.filter.PathFilter.invoke(PathFilter.java:94)
at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:70)
at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:79)
at coldfusion.filter.ClientScopePersistenceFilter.invoke(ClientScopePersistenceFilter.java:28)
at coldfusion.filter.BrowserFilter.invoke(BrowserFilter.java:38)
at coldfusion.filter.NoCacheFilter.invoke(NoCacheFilter.java:46)
at coldfusion.filter.GlobalsFilter.invoke(GlobalsFilter.java:38)
at coldfusion.filter.DatasourceFilter.invoke(DatasourceFilter.java:22)
at coldfusion.filter.CachingFilter.invoke(CachingFilter.java:62)
at coldfusion.filter.RequestThrottleFilter.invoke(RequestThrottleFilter.java:126)
at coldfusion.CfmServlet.service(CfmServlet.java:201)
at coldfusion.bootstrap.BootstrapServlet.service(BootstrapServlet.java:89)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:86)
at coldfusion.monitor.event.MonitoringServletFilter.doFilter(MonitoringServletFilter.java:42)
at coldfusion.bootstrap.BootstrapFilter.doFilter(BootstrapFilter.java:46)
at jrun.servlet.FilterChain.doFilter(FilterChain.java:94)
at jrun.servlet.FilterChain.service(FilterChain.java:101)
at jrun.servlet.ServletInvoker.invoke(ServletInvoker.java:106)
at jrun.servlet.JRunInvokerChain.invokeNext(JRunInvokerChain.java:42)
at jrun.servlet.JRunRequestDispatcher.invoke(JRunRequestDispatcher.java:286)
at jrun.servlet.ServletEngineService.dispatch(ServletEngineService.java:543)
at jrun.servlet.jrpp.JRunProxyService.invokeRunnable(JRunProxyService.java:203)
at jrunx.scheduler.ThreadPool$ThreadThrottle.invokeRunnable(ThreadPool.java:428)
at jrunx.scheduler.WorkerThread.run(WorkerThread.java:66)
Update 2
I have been digging through Damon Miller's implementation of the OWASP ESAPI methods for ColdFusion. I noticed in his code that he does not call the getValidInput() method with the ValidationErrorList attribute. Rather he wrote the code to catch the generated error and then add the error to the list himself. Hmmm? I thought the method was supposed to do that for you????
As an aside, I am trying not to use a library such as his to avoid the additional bloat that I do not need.
Excerpt from his code:
if(structKeyExists( arguments, "errorList" )) {
try {
return getValidInput( arguments.context, arguments.input, arguments.type, arguments.maxLength, arguments.allowNull );
}
catch(esapi4cf.org.owasp.esapi.errors.ValidationException e) {
arguments.errorList.addError( arguments.context, e );
}
return arguments.input;
}
else {
...
Answer
This looks to be a bug in the Coldfusion implementation of ESAPI - we have full coverage testing of the getValidInput method in your unit test suite for ESAPI that demonstrate that the method works as advertised.
Based off of your second update above I would guess that in the CF implementation code there is an uninitialized variable being accessed (perhaps errorList is uninitialized in this context)
I am the project leader of the OWASP ESAPI project and very familiar with this bit of code in ESAPI itself but am not a CF developer and have not seen all of the implementation code for CF9.
** Edit **
In order to make the validation methods work in batch using ColdFusion a call to the init() method is required for the org.owasp.esapi.ValidationErrorList class before calling the validator methods. Add the following line to the test script and it will work:
<cfset ErrorList = ErrorList.init() />
In context:
<cfset ErrorList = CreateObject("java", "org.owasp.esapi.ValidationErrorList") />
<cfset ErrorList = ErrorList.init() />
<cfset Valid.TestField = EsapiValidator.getValidInput("Test Field", form.TestField, "SafeString", 128, true, true, ErrorList) />
Now when invalid input is entered the error will be added to the ErrorList variable instead of throwing an error.