Last Updated: 2024-11-21

When creating an endpoint for receiving Amazon SNS notifications, I received JSON like the following:

"{\"Type\":\"Notification\",\"MessageId\":\"afa70128-12cf-5285-980c-54353313a8e0\",\"TopicArn\":\"arn:aws:sns:us-east-1:734356286499:less-penguiny-email-bounce\",\"Message\":\"{\\\"notificationType\\\":\\\"Bounce\\\",\\\"bounce\\\":{\\\"bounceType\\\":\\\"Permanent\\\",\\\"bounceSubType\\\":\\\"General\\\",\\\"bouncedRecipients\\\":[{\\\"emailAddress\\\":\\\"helloasdf@asd.cmop\\\",\\\"action\\\":\\\"failed\\\",\\\"status\\\":\\\"5.4.4\\\",\\\"diagnosticCode\\\":\\\"smtp;
550 5.4.4 Invalid
domain\\\"}],\\\"timestamp\\\":\\\"2019-04-02T08:38:52.330Z\\\",\\\"feedbackId\\\":\\\"01000169dd337a6c-59c15c30-a62c-4897-a1dd-4962dbaab1e8-000000\\\",\\\"reportingMTA\\\":\\\"dsn;
a8-27.smtp-out.amazonses.com\\\"},\\\"mail\\\":{\\\"timestamp\\\":\\\"2019-04-02T08:38:51.000Z\\\",\\\"source\\\":\\\"<no-reply@redcated.com>\\\",\\\"sourceArn\\\":\\\"arn:aws:ses:us-east-1:734356286499:identity/no-reply@redcated.com\\\",\\\"sourceIp\\\":\\\"3.81.81.236\\\",\\\"sendingAccountId\\\":\\\"734356286499\\\",\\\"messageId\\\":\\\"01000169dd3378bc-f8b21149-3159-4215-85a9-ba187f5e3cec-000000\\\",\\\"destination\\\":[\\\"helloasdf@asd.cmop\\\"]}}\",\"Timestamp\":\"2019-04-02T08:38:52.372Z\",\"SignatureVersion\":\"1\",\"Signature\":\"DNa7xn0q9fTqR0V+ubNEfWySYhIEIjF/GwrH4vfjw6Ze77zOXTJA8fzBZugxNF2+j+XAjcfkvFr0p3IK7/POMm9mF04KVujGkFVrYjYBrUELWZsB+7TCvAru7uOHmKwWh4usZVJKq28DCVy0awufHTdo6/8ts6Y4/K8/Q9ua6ATbV8Cnw9nYhJnNAWmY8Zji4OCg3NemnjMiB0PkOJjNljE3vEiTV/FtL6yIjoCVK4rHbevPwl2t+Exg86oyEQlqIvyRIRKE8OSElIDVN6qpZcFxpuzWayfc17WpIb4RmotYGuC3ryt7EX/2H1aFvLWxBumVw8aN3iEibjido9/Miw==\",\"SigningCertURL\":\"https://sns.us-east-1.amazonaws.com/SimpleNotificationService-6aad65c2f9911b05cd53efda11f913f9.pem\",\"UnsubscribeURL\":\"https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:734356286499:less-penguiny-email-bounce:3ff961a9-aa59-416a-93bc-7bfc67f91229\"}"

Running JSON.parse gave me:

{"Type"=>"Notification", "MessageId"=>"afa70128-12cf-5285-980c-54353313a8e0", "TopicArn"=>"arn:aws:sns:us-east-1:734356286499:less-penguiny-email-bounce", "Message"=>"{\"notificationType\":\"Bounce\",\"bounce\":{\"bounceType\":\"Permanent\",\"bounceSubType\":\"General\",\"bouncedRecipients\":[{\"emailAddress\":\"helloasdf@asd.cmop\",\"action\":\"failed\",\"status\":\"5.4.4\",\"diagnosticCode\":\"smtp; 550 5.4.4 Invalid domain\"}],\"timestamp\":\"2019-04-02T08:38:52.330Z\",\"feedbackId\":\"01000169dd337a6c-59c15c30-a62c-4897-a1dd-4962dbaab1e8-000000\",\"reportingMTA\":\"dsn; a8-27.smtp-out.amazonses.com\"},\"mail\":{\"timestamp\":\"2019-04-02T08:38:51.000Z\",\"source\":\"Less Penguiny <no-reply@redcated.com>\",\"sourceArn\":\"arn:aws:ses:us-east-1:734356286499:identity/no-reply@redcated.com\",\"sourceIp\":\"3.81.81.236\",\"sendingAccountId\":\"734356286499\",\"messageId\":\"01000169dd3378bc-f8b21149-3159-4215-85a9-ba187f5e3cec-000000\",\"destination\":[\"helloasdf@asd.cmop\"]}}", "Timestamp"=>"2019-04-02T08:38:52.372Z", "SignatureVersion"=>"1", "Signature"=>"DNa7xn0q9fTqR0V+ubNEfWySYhIEIjF/GwrH4vfjw6Ze77zOXTJA8fzBZugxNF2+j+XAjcfkvFr0p3IK7/POMm9mF04KVujGkFVrYjYBrUELWZsB+7TCvAru7uOHmKwWh4usZVJKq28DCVy0awufHTdo6/8ts6Y4/K8/Q9ua6ATbV8Cnw9nYhJnNAWmY8Zji4OCg3NemnjMiB0PkOJjNljE3vEiTV/FtL6yIjoCVK4rHbevPwl2t+Exg86oyEQlqIvyRIRKE8OSElIDVN6qpZcFxpuzWayfc17WpIb4RmotYGuC3ryt7EX/2H1aFvLWxBumVw8aN3iEibjido9/Miw==", "SigningCertURL"=>"https://sns.us-east-1.amazonaws.com/SimpleNotificationService-6aad65c2f9911b05cd53efda11f913f9.pem", "UnsubscribeURL"=>"https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:734356286499:less-penguiny-email-bounce:3ff961a9-aa59-416a-93bc-7bfc67f91229"}
=> {"Type"=>"Notification",
 "MessageId"=>"afa70128-12cf-5285-980c-54353313a8e0",
 "Message"=>
  "{\"notificationType\":\"Bounce\",\"bounce\":{\"bounceType\":\"Permanent\",\"bounceSubType\":\"General\",\"bouncedRecipients\":[{\"emailAddress\":\"helloasdf@asd.cmop\",\"action\":\"failed\",\"status\":\"5.4.4\",\"diagnosticCode\":\"smtp; 550 5.4.4 Invalid domain\"}],\"timestamp\":\"2019-04-02T08:38:52.330Z\",\"feedbackId\":\"01000169dd337a6c-59c15c30-a62c-4897-a1dd-4962dbaab1e8-000000\",\"reportingMTA\":\"dsn; a8-27.smtp-out.amazonses.com\"},\"mail\":{\"timestamp\":\"2019-04-02T08:38:51.000Z\",\"source\":\"",\"sourceArn\":\"arn:aws:ses:us-east-1:734356286499:identity/no-reply@redacted.com\",\"sourceIp\":\"3.81.81.236\",\"sendingAccountId\":\"734356286499\",\"messageId\":\"01000169dd3378bc-f8b21149-3159-4215-85a9-ba187f5e3cec-000000\",\"destination\":[\"helloasdf@asd.cmop\"]}}",
 "Timestamp"=>"2019-04-02T08:38:52.372Z",
 "SignatureVersion"=>"1",
 "Signature"=>
  "DNa7xn0q9fTqR0V+ubNEfWySYhIEIjF/GwrH4vfjw6Ze77zOXTJA8fzBZugxNF2+j+XAjcfkvFr0p3IK7/POMm9mF04KVujGkFVrYjYBrUELWZsB+7TCvAru7uOHmKwWh4usZVJKq28DCVy0awufHTdo6/8ts6Y4/K8/Q9ua6ATbV8Cnw9nYhJnNAWmY8Zji4OCg3NemnjMiB0PkOJjNljE3vEiTV/FtL6yIjoCVK4rHbevPwl2t+Exg86oyEQlqIvyRIRKE8OSElIDVN6qpZcFxpuzWayfc17WpIb4RmotYGuC3ryt7EX/2H1aFvLWxBumVw8aN3iEibjido9/Miw==",
 "SigningCertURL"=>"https://sns.us-east-1.amazonaws.com/SimpleNotificationService-6aad65c2f9911b05cd53efda11f913f9.pem",
 "UnsubscribeURL"=>"https://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:734356286499:decated-email-bounce:3ff961a9-aa59-416a-93bc-7bfc67f91229"}

I then fed this into my own functions and had surprising failures about the 'bouncedRecipients' property being called on nil.

What was up? Had I looked more closely at the parsed JSON above, I would have seen that the "Message" key contained further parsed JSON that I would have to parse again:

JSON.parse(previous_json["Message"])
=> {"notificationType"=>"Bounce",
 "bounce"=>
  {"bounceType"=>"Permanent",
   "bounceSubType"=>"General",
   "bouncedRecipients"=>[{"emailAddress"=>"wanttoseenewposts@butdonthabituallyusemailinglists.ever", "action"=>"failed", "status"=>"5.4.4", "diagnosticCode"=>"smtp; 550 5.4.4 Invalid domain"}],
   "timestamp"=>"2019-04-02T08:39:05.837Z",
   "feedbackId"=>"01000169dd33af2a-a381083c-283c-4b07-af90-534c33ee1af6-000000",
   "reportingMTA"=>"dsn; a8-24.smtp-out.amazonses.com"},
 "mail"=>
  {"timestamp"=>"2019-04-02T08:39:05.000Z",
   "sourceIp"=>"3.81.81.236",
   "sendingAccountId"=>"734356286499",
   "messageId"=>"01000169dd33ae8d-36e36e3e-af56-4660-a431-3a4013f38f92-000000",
   "destination"=>["wanttoseenewposts@butdonthabituallyusemailinglists.ever"]}}

Lesson

Be aware that some programs can give you serialized JSON containing sub-keys that are encoded once again in serialized JSON. Therefore you'll have to decode it twice to extract the information.