I`m getting lots of false positives [??]after just setting up mod_security. I'm running it in detection only so no issues yet but these filters will start blocking requests once I need it to go live.
Afraid I don't 100% understand what the significance of these filters are, I get 100s of them on nearly every domain & all the requests look legitimate.
Request Missing a User Agent Header
Request Missing an Accept Header
What is the best thing to do here? Should I disable these filters? Can I set the severity lower so that requests won't be blocked?
Here is a complete entry
[22/Nov/2011:21:32:37 --0500] u6t6IX8AAAEAAHSiwYMAAAAG 72.47.232.216 38543 72.47.232.216 80
--5fcb9215-B--
GET /Assets/XHTML/mainMenu.html HTTP/1.0
Host: www.domain.com
Content-type: text/html
Cookie: pdgcomm-babble=413300:451807c5d49b8f61024afdd94e57bdc3; __utma=100306584.1343043347.1321115981.1321478968.1321851203.4; __utmz=100306584.1321115981.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=XXXXXXXX%20clip%20ons
--5fcb9215-F--
HTTP/1.1 200 OK
Last-Modified: Wed, 23 Nov 2011 02:01:02 GMT
ETag: "21e2a7a-816d"
Accept-Ranges: bytes
Content-Length: 33133
Vary: Accept-Encoding
Connection: close
Content-Type: text/html
--5fcb9215-H--
Message: Operator EQ matched 0 at REQUEST_HEADERS. [file "/etc/httpd/modsecurity_crs/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "47"] [id "960015"] [rev "2.2.1"] [msg "Request Missing an Accept Header"] [severity "CRITICAL"] [tag "PROTOCOL_VIOLATION/MISSING_HEADER_ACCEPT"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"]
Message: Operator EQ matched 0 at REQUEST_HEADERS. [file "/etc/httpd/modsecurity_crs/base_rules/modsecurity_crs_21_protocol_anomalies.conf"] [line "66"] [id "960009"] [rev "2.2.1"] [msg "Request Missing a User Agent Header"] [severity "NOTICE"] [tag "PROTOCOL_VIOLATION/MISSING_HEADER_UA"] [tag "WASCTC/WASC-21"] [tag "OWASP_TOP_10/A7"] [tag "PCI/6.5.10"]
Message: Warning. Operator LT matched 5 at TX:inbound_anomaly_score. [file "/etc/httpd/modsecurity_crs/base_rules/modsecurity_crs_60_correlation.conf"] [line "33"] [id "981203"] [msg "Inbound Anomaly Score (Total Inbound Score: 4, SQLi=5, XSS=): Request Missing a User Agent Header"]
Stopwatch: 1322015557122593 24656 (- - -)
Stopwatch2: 1322015557122593 24656; combined=23703, p1=214, p2=23251, p3=2, p4=67, p5=168, sr=88, sw=1, l=0, gc=0
Producer: ModSecurity for Apache/2.6.1 (http://www.modsecurity.org/); core ruleset/2.2.1.
Server: Apache/2.2.3 (CentOS)
Answer
If you look under Section H of the audit log entry you showed at the Producer line, you will see that you are using the OWASP ModSecurity Core Rule Set (CRS) v2.2.1. In this case, I suggest you review the documentation information on the project page -
https://www.owasp.org/index.php/Category:OWASP_ModSecurity_Core_Rule_Set_Project#tab=Documentation
Specifically, you should review these two blog posts that I did -
- http://blog.spiderlabs.com/2010/11/advanced-topic-of-the-week-traditional-vs-anomaly-scoring-detection-modes.html
- http://blog.spiderlabs.com/2011/08/modsecurity-advanced-topic-of-the-week-exception-handling.html
Blog post #1 is useful so that you understand which "mode of operation" you are using for the CRS. By looking at your audit log, it appears you are running in anomaly scoring mode. This is where the rules are doing detection but the blocking decision is being done separately by inspecting the overall anomaly score in the modsecurity_crs_49_inbound_blocking.conf file.
Blog post #2 is useful so that you can decided exactly how you want to handle these two rules. If you feel that these are not important to you - then I would suggest that you use the SecRuleRemoveById directive to disable these rules from your own modsecurity_crs_60_exceptions.conf file. The way that it stands now, these two alert are only generating an inbound anomaly score of 4 - which is below the default threshold of 5 set in the modsecurity_crs_10_config.conf file so it is not blocked.
Looking at your audit log example, while this request did generate alerts, the transaction was not blocked. If it was, the message data under Section H would have stated "Access denied...".
As for the purposed of these rules - they are meant to flag requests that are not generated from standard web browsers (IE, Chrome, Firefox, etc...) as all of these browsers will send both User-Agent and Accept requests headers per the HTTP RFC spec.
One last comment - I would suggest that you use the official OWASP ModSecurity CRS mail-list for these types of questions - https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
You can also search the archives that for answers.
Cheers, Ryan Barnett ModSecurity Project Lead OWASP ModSecurity CRS Project Lead