Tool for parsing SMTP logs that finds bounces
Our web application sends e-mails. We have lots of users, and we get lots of bounces. For example, user changes company and his company e-mail is no longer valid.
To find bounces, I parse SMTP log file with log parser. The logs come from Microsoft SMTP server.
Some bounces are great, like 550+#[email protected]. There is [email protected] in bounce.
But some do not have e-mail in error message, like 550+No+such+recipient.
I have created simple Ruby script that parses logs (uses log parser) to find which mail caused something like 550+No+such+recipient.
I am just surprised that I could not find a tool that does it. I have found tools like Zabbix and Splunk for log analysis, but they look like overkill for such simple task.
Anybody knows a tool that would parse SMTP logs, find bounces and e-mails that cause them?
Answer
As far as I can see, log file analysis is really only useful to detect mails which are rejected at the SMTP session level. What about bounces which occur after the remote MTA has accepted a message for delivery but subsequently fails to deliver it?
We use the following set up to detect and classify all bounces after delivery to the remote MTA.
All outgoing mails are given a unique return-path header which, when decoded, identifies the recipient email address and the particular mailing.
An Apache James server which receives mail returned to the returned-path address.
A custom mailet, developed in Java and executing within Apache James which decodes the to address, sends the email text to boogietools bounce studio for bounce type classification and then persists the results to our database.
It works very, very well. We are able to detect permanent hard bounces and transient soft bounces which are further classified into very granular bounce types such as spam rejections, out of office replies etc.