Extra logging wrapper script for SES Postfix transport

I’m using Amazon’s SES service for my servers’ emails. To implement, instead of re-writing all of our code to hook into the SES API, I simply configured Postfix to use the example script ses-send-mail.pl provided by Amazon. It works fine and dandy, with mails happily going out to their intended recipients via SES.

However, that’s not good enough for me. You see, if you send a mail through SES and it bounces, you’ll receive the bounce message at the original From: address, as expected, but because a lot of ISPs/ESPs strip the original To: header in their bounce templates to prevent backscatter, and SES mangles the message ID set on the email by Postfix (replacing it with their own), it’s very possible to get bounce messages that have no information on the intended recipient. How do you do bounce management when you have no information that links the bounce to the original email that you sent?

While Amazon strips the message ID assigned by Postfix, it adds its own message ID — AWSMessageID. This value is returned by the SES API when you submit an email to the service; the provided example scripts, however, don’t do anything with this ID.

To address this issue in my environment, I wrote the following script, which I set as my Postfix transport (rather than ses-send-email.pl).

#!/bin/bash
# send mail via SES and create a log with returned messageid for bounce processing

MAILFROM=$1
RCPTTOLOG=`echo $* | awk '{$1=""; print $0}' | awk '{sub(/^[ \t]+/, "")};1'`
RCPTTO=`echo $RCPTTOLOG | sed -e 's/\ /,/g'`
SCRIPT=/usr/local/amazon/ses-send-email.pl
SCRIPTOPTS="-r"
TIMESTAMP=`date +"%Y-%m-%d %H:%M:%S"`
ACCESSFILE=/usr/local/amazon/access
THEMAIL=`cat -`
SUBJECT=`echo "$THEMAIL" |awk '($0 ~ /Subject: /) {$1=""; print $0}' |awk '{sub(/^[ \t]+/, "")};1'`

OUTPUT=`echo "$THEMAIL" | $SCRIPT $SCRIPTOPTS --verbose -k $ACCESSFILE -f $MAILFROM $RCPTTO`
if echo "$OUTPUT" |grep -q Error
then
	exit 1 # SES error, postfix should defer this msg
fi

MESSAGEID=`echo $OUTPUT |awk '{print $4}' |awk -F\> '{print $2}' |awk -F\< '{print " AWSMessageID=" $1}'`

# log
echo "$TIMESTAMP from=$MAILFROM to=\"$RCPTTOLOG\" subject=\"$SUBJECT\" $MESSAGEID" >> /var/log/ses_maillog

Set ACCESS to the location of the file containing your AWS key and secret, and of course configure paths as needbe. The transport should be configured as such in master.cf:

# AWS-SES
aws-email  unix  -       n       n       -       -       pipe
  flags=R user=mail argv=/usr/local/amazon/ses-log-n-send.sh ${sender} ${recipient}

You’ll get a log file at /var/log/ses_maillog that looks something like this:

2011-08-23 16:26:24 from=bugs@butt.com to="butts@gmail.com" subject="this is my email subject"  AWSMessageID=00000131f8f261e2-75f27db7-b6d2-43ca-9c26-9a4a92ecbfd0-000000
2011-08-23 16:26:23 from=bugs@butt.com to="morebutts@gmail.com" subject="Re: this is my email subject"  AWSMessageID=00000131f8f761b9-acfceec3-73ab-4d5e-8959-f7bb9ee00665-000000
2011-08-23 16:26:25 from=bugs@butt.com to="toomanybutts@gmail.com" subject="another email subject"  AWSMessageID=00000131f8f76669-1540d563-41c0-4ba9-adc0-122ee41f4b28-000000

Now you can grep grep grep away for the AWSMessageID to match the one in the bounce email to find the original recipient and update your lists accordingly.

2 Comments

  • bro

    August 23, 2011

    I am stealing this and putting it on my blog and pretending I came up with it. Thanks for making me look good, bro.

    -Bro

  • Ralph

    October 22, 2011

    Indicate your fee to configure an EC2 server with a SES stack or tell me who can ? THANKS
    RalphJuliano@gmail.com

Leave a Reply