Category: Messaging & Event Streaming Tags: amazon-sns pubsub event-driven-architecture

What Is Amazon SNS?

By Oleksandr Andrushchenko — Published on — Modified on
0 0

Amazon Simple Notification Service (Amazon SNS) is a fully managed publish/subscribe messaging service from AWS. It allows applications to publish messages to a topic and automatically distribute those messages to subscribed endpoints, including Amazon SQS queues, AWS Lambda functions, HTTP endpoints, email addresses, and supported mobile notification destinations. Publishers do not need to know how many subscribers exist or how those subscribers process the message.

SNS is commonly used for event fan-out, application notifications, operational alerts, and communication between microservices. It removes direct dependencies between event producers and consumers, but it does not automatically guarantee successful business processing. Reliable production architectures must account for delivery retries, message filtering, duplicate delivery, subscriber failures, and the difference between publishing an event and completing the work triggered by that event.

Table of Contents

What Amazon SNS Provides

SNS implements a publish/subscribe messaging model. Instead of sending separate requests to every downstream service, an application publishes an event to a topic. SNS evaluates the topic's subscriptions and attempts to deliver the message to the matching endpoints.

This separates event production from event distribution. A publisher does not need to change its application code when a new subscriber is added, provided the new subscriber can consume the existing event contract.

Common SNS Use Cases

SNS is especially useful when one event must trigger independent activities in different services.

  • Order events: Notify fulfillment, billing, analytics, and customer notification services.
  • Account events: Distribute registration, verification, and profile change notifications.
  • Infrastructure alerts: Deliver CloudWatch alarm notifications to configured endpoints.
  • Background processing: Fan out events to separate SQS queues for independent worker groups.
  • Mobile notifications: Deliver push notifications through supported mobile platform integrations.
  • Email and SMS notifications: Send supported application-to-person messages.
  • Data synchronization: Inform downstream systems about changes that require updating local projections.

For example, an e-commerce application can publish order.created once. An email service may send a confirmation, an analytics service may update reporting metrics, and a warehouse service may create a fulfillment request.

Each subscriber owns its processing logic. A temporary outage of the analytics worker should not prevent the email service from processing its own copy of the event.

This general communication model is explained further in What Is Pub/Sub?.

SNS vs. SQS, EventBridge, and Kafka

SNS, SQS, and EventBridge can all participate in event-driven systems, but they solve different routing and delivery problems.

Service Primary Purpose Typical Pattern
Amazon SNS Distribute messages to multiple subscribers Publish/subscribe fan-out
Amazon SQS Buffer work for asynchronous consumers Queue with competing consumers
Amazon EventBridge Route events through configurable rules Event bus and content-based routing
Apache Kafka Persist partitioned event streams Durable event log and consumer offsets

With SQS, multiple workers consuming one queue generally compete for the same messages. With SNS, separate subscriptions can each receive a copy of an event.

SNS and SQS are frequently used together. SNS handles event distribution, while SQS provides durable buffering and independent acknowledgment for each downstream service.

EventBridge becomes attractive when routing requires many event rules, schema-based conditions, integrations, or centralized event-bus management. Kafka is useful when event retention, replay, partition offsets, and stream-processing requirements are central to the architecture.

How Amazon SNS Works

SNS organizes messaging around topics and subscriptions. A topic is the logical destination to which publishers send messages. A subscription connects the topic to a delivery endpoint.

Each subscription can have its own filtering and delivery configuration, allowing the publisher to remain independent from subscriber-specific routing decisions.

Topics, Publishers, and Subscriptions

A basic SNS application contains three components:

  • Publisher: An application or AWS service that calls the SNS Publish API.
  • Topic: The SNS resource that receives messages and evaluates subscriptions.
  • Subscriber: An endpoint or service that receives messages through a subscription.

Supported subscription protocols include Amazon SQS, AWS Lambda, HTTP/HTTPS, email, SMS, and supported mobile notification services. Availability and behavior depend on topic type and endpoint capabilities.

For example, an SNS topic may have two SQS subscriptions, one Lambda subscription, and an HTTPS endpoint. A published event can be delivered to all matching subscribers without the producer sending four separate messages.

Subscribing an email address or HTTPS endpoint may require confirmation. Infrastructure automation must account for these confirmation flows instead of assuming that every subscription becomes active immediately.

Publishing Messages and Message Attributes

A publisher calls Publish with a topic ARN and a message payload. The message can be accompanied by a subject, message attributes, or protocol-specific content where supported.

For event-driven applications, JSON is a practical payload format. Use an explicit schema version and a stable event identifier so consumers can validate and deduplicate messages.

{
  "event_id": "evt-10042",
  "event_type": "order.created",
  "schema_version": 1,
  "order_id": "ord-9821",
  "tenant_id": "tenant-42",
  "created_at": "2026-10-21T15:00:00Z"
}

Message attributes carry metadata separately from the body. For example, an event_type attribute can be used by SNS subscription filters without requiring the publisher to know which subscribers are interested in that event.

Keep events focused on business facts rather than copying entire database records. Large payloads increase messaging cost and create tighter coupling between producers and consumers.

Subscription Message Filtering

SNS subscription filter policies allow subscribers to receive only messages that match defined conditions. Filtering can evaluate message attributes or JSON message-body fields, depending on the configured filter policy scope.

Suppose the topic publishes several event types:

orders-events
  order.created
  order.cancelled
  payment.completed
  shipment.delivered

The notification service may subscribe only to order.created messages, while the analytics service receives all events.

For message-attribute filtering, the notification subscription can use:

{
  "event_type": [
    "order.created"
  ]
}

The publisher still sends every event to the same topic. SNS evaluates the filter and delivers matching messages to the notification subscriber.

Filter policies can support conditions involving string values, numeric comparisons, prefixes, exclusions, and other documented operators. This allows reasonably sophisticated routing without building a separate routing service.

Filter policy changes are eventually propagated and can take time to become fully effective. Do not assume a newly updated subscription filter is immediately active across the entire service.

Standard vs. FIFO Topics

Amazon SNS supports Standard and FIFO topics. They have different message ordering, deduplication, and subscriber support characteristics.

Choose the topic type based on the business requirements of subscribers, especially whether message ordering matters and whether the intended endpoints support FIFO delivery.

Standard Topics

Standard topics support high-throughput publish/subscribe messaging with at-least-once delivery behavior and best-effort ordering.

Subscribers must tolerate messages arriving more than once or in a different order from publication.

Standard topics are suitable for:

  • Application events where strict ordering is unnecessary.
  • Independent background processing notifications.
  • Operational alerts.
  • Fan-out to multiple Standard SQS queues.
  • Supported email, SMS, HTTP, and Lambda subscribers.

A Standard topic is often the appropriate default for event-driven applications because independent subscribers typically do not require one global event order.

FIFO Ordering and Deduplication

FIFO topics preserve message order within a message group and support publish-side deduplication within a five-minute interval.

Two important message parameters are:

  • MessageGroupId: Defines the ordering group for related messages.
  • MessageDeduplicationId: Identifies duplicate publishes within the applicable deduplication scope.

For example, inventory events for one product may need to remain ordered, while events for different products can be processed independently.

import json
import boto3


sns = boto3.client("sns")


def publish_inventory_event(
    topic_arn: str,
    product_id: str,
    event_id: str,
    new_quantity: int,
) -> str:
    response = sns.publish(
        TopicArn=topic_arn,
        Message=json.dumps({
            "event_id": event_id,
            "event_type": "inventory.updated",
            "product_id": product_id,
            "quantity": new_quantity,
        }),
        MessageGroupId=f"PRODUCT#{product_id}",
        MessageDeduplicationId=event_id,
    )

    return response["MessageId"]

For end-to-end ordered consumption, subscribe compatible SQS FIFO queues and configure consumers to respect the ordering requirements of each message group.

FIFO deduplication does not eliminate the need for idempotent business processing. A consumer may complete its database transaction and then fail before acknowledging the message at the queue layer.

FIFO Subscription Limitations

SNS FIFO topics can deliver to SQS FIFO queues and Standard queues, but their guarantees differ depending on the subscribed queue type.

With an SQS FIFO queue, the messaging system can preserve order within groups. With an SQS Standard queue, best-effort ordering and possible duplicate deliveries apply at the queue-consumption layer.

SNS FIFO topics do not directly deliver to all endpoint types supported by Standard topics. In particular, they do not directly support Lambda, email, SMS, or HTTP/HTTPS endpoints as FIFO subscriptions. Lambda processing can be connected through a subscribed SQS queue.

Message filtering adds another important distinction. Filtering can exclude events from a subscriber, and AWS documents additional qualifications for FIFO exactly-once delivery claims when filters are enabled.

Use FIFO when ordering is a genuine requirement, not simply because the term sounds safer.

Message Delivery and Reliability

Successful publication to SNS means the topic accepted the message for distribution. It does not mean that every subscriber has completed its business operation or that all destinations are currently healthy.

Failures can occur during delivery to a subscribed endpoint or later, while that subscriber processes the message. These failures require different recovery mechanisms.

Delivery Retries and Subscription DLQs

SNS applies protocol-specific delivery retry policies. When an endpoint is temporarily unavailable, SNS may retry delivery according to the applicable policy and error classification.

For supported subscriptions, an SNS subscription dead-letter queue can capture messages that SNS cannot successfully deliver after its configured retry behavior.

Consider two separate failures:

  • SNS delivery failure: SNS cannot send the event to the subscribed SQS queue because of permissions, endpoint availability, or another delivery error.
  • Consumer processing failure: SNS successfully delivers the event to SQS, but the worker repeatedly fails to process it.

An SNS subscription DLQ protects the first boundary. An SQS source-queue redrive policy protects the second boundary.

Production systems may need both. An SQS processing DLQ cannot recover an event that SNS never successfully delivered to the source queue.

Subscription DLQs use Amazon SQS queues and require appropriate queue permissions for SNS to send failed-delivery records. Monitor both delivery failures and consumer failures instead of treating them as one metric.

Reliable Fan-Out with SNS and SQS

A common production pattern is to give every independently operated subscriber its own SQS queue.

Suppose an order event has three consumers:

  1. The notification service sends customer emails.
  2. The fulfillment service prepares warehouse work.
  3. The analytics service updates reporting data.

Subscribe three queues to one SNS topic. Every matching queue receives its own message copy, and workers for each service acknowledge only messages from their own queue.

If the email service is temporarily unavailable, its queue buffers the messages while fulfillment and analytics continue processing.

This is different from attaching several worker groups to one shared SQS queue, where they would compete for individual messages rather than each receiving a copy.

For the queue-side lifecycle and acknowledgment behavior, read What Is a Message Queue?.

Duplicate Events and Idempotency

Retries can cause repeated deliveries. Even if SNS successfully distributes one copy to a queue, a consumer can later receive the same logical message again because processing or acknowledgment failed.

For example, the email service may send a confirmation email and crash before marking the event as completed. After the message becomes visible again, another worker attempts to send the same email.

For business-sensitive operations, maintain a stable event ID and an idempotency record. A consumer should detect whether a logical operation has already completed before repeating its side effects.

When the business update and deduplication record are in one database, a single transaction can coordinate them. External actions such as email delivery or payment calls require more careful coordination because their effects cannot always be rolled back with the database.

Broker-level guarantees and application-level effects are separate concepts. Reliable event consumers should explicitly handle the case where the result of a previous attempt is uncertain.

Production Design Example: Order Event Distribution

Consider an e-commerce application with an order API, PostgreSQL, a notification service, an inventory service, and an analytics service. The order API must complete its transaction without waiting for all downstream activities.

SNS is responsible for distributing committed order events. Each consumer service owns its queue, retry configuration, and processing logic.

Architecture and Event Flow

A production deployment uses the following components:

  • FastAPI order service: Validates requests and writes orders to PostgreSQL.
  • Transactional outbox: Records outgoing events in the same database transaction.
  • Outbox publisher: Delivers committed event records to SNS.
  • SNS Standard topic: Distributes order events to subscribed destinations.
  • Notification SQS queue: Buffers messages for customer communications.
  • Fulfillment SQS queue: Buffers warehouse and shipping work.
  • Analytics SQS queue: Buffers reporting updates.
  • Independent workers: Process and acknowledge messages on their own schedules.

The order service publishes facts such as order.created, order.cancelled, or order.shipped. Subscription filters determine which queues receive which event types.

For example, the notification service may receive order.created and order.shipped, while analytics receives every order lifecycle event.

When an order is inserted into PostgreSQL, the application also writes an outbox record in the same transaction. A publisher processes that record and sends the event to SNS.

This avoids the dual-write failure where an order is committed successfully but its event is lost because the application crashes before publishing. The mechanism is explained in What Is a Transactional Outbox?.

Failure Scenarios and Recovery

Suppose the analytics queue's access policy is accidentally changed so SNS can no longer deliver events. The notification and fulfillment queues may continue receiving new events, while analytics falls behind.

If an SNS subscription DLQ is configured correctly, failed delivery records can be retained for investigation and recovery. Without that protection, relying only on the analytics processing DLQ would not recover events that never reached its source queue.

A second failure happens when an analytics worker writes a summary record but crashes before deleting the SQS message. On redelivery, the worker must recognize the event ID and avoid applying the same update again.

Subscriber recovery should include investigating the failure, repairing permissions or processing logic, and replaying messages safely. Redrive or replay operations must preserve logical event identifiers so the recovery process does not produce duplicate business changes.

Different subscriber queues may be at different positions in the overall event history. Avoid assuming that all downstream systems become consistent at exactly the same moment.

Ready-to-Use Example: SNS Fan-Out with Terraform and Python

This example creates an SNS Standard topic and two SQS Standard queues. The notification queue receives only order.created events, while the analytics queue receives all published events.

Terraform configures subscriptions, SQS queue policies, server-side encryption, and processing dead-letter queues. Python scripts publish sample events and consume messages using SQS long polling.

1. Project Structure

Install Terraform, Python 3, and the AWS CLI. Configure AWS credentials with permissions to provision SNS and SQS resources and to publish or consume messages.

Create these files:

sns-demo/
  main.tf
  outputs.tf
  requirements.txt
  publisher.py
  consumer.py

For simplicity, both queues are owned by the same AWS account and deployed to the same Region. Cross-account subscriptions require additional resource policies and appropriate authorization.

2. Provision SNS and SQS with Terraform

Create main.tf:

terraform {
  required_version = ">= 1.5.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
}

provider "aws" {
  region = "us-west-2"
}

# -------------------------------------
# SNS topic
# -------------------------------------

resource "aws_sns_topic" "orders" {
  name = "sysdespro-orders-events-demo"

  # Use AWS-managed SNS encryption.
  # For key management requirements, configure
  # an appropriate customer-managed KMS key.
  tags = {
    Environment = "development"
    ManagedBy   = "terraform"
  }
}

# -------------------------------------
# SQS queues
# -------------------------------------

resource "aws_sqs_queue" "notification_dlq" {
  name = "sysdespro-notifications-dlq-demo"

  message_retention_seconds = 1209600
  sqs_managed_sse_enabled   = true
}

resource "aws_sqs_queue" "analytics_dlq" {
  name = "sysdespro-analytics-dlq-demo"

  message_retention_seconds = 1209600
  sqs_managed_sse_enabled   = true
}

resource "aws_sqs_queue" "notifications" {
  name = "sysdespro-notifications-demo"

  sqs_managed_sse_enabled    = true
  receive_wait_time_seconds  = 20
  visibility_timeout_seconds = 60

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.notification_dlq.arn
    maxReceiveCount     = 5
  })
}

resource "aws_sqs_queue" "analytics" {
  name = "sysdespro-analytics-demo"

  sqs_managed_sse_enabled    = true
  receive_wait_time_seconds  = 20
  visibility_timeout_seconds = 60

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.analytics_dlq.arn
    maxReceiveCount     = 5
  })
}

# -------------------------------------
# Allow SNS to deliver to the queues.
# Each policy is scoped to the topic ARN.
# -------------------------------------

resource "aws_sqs_queue_policy" "notifications" {
  queue_url = aws_sqs_queue.notifications.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "AllowOrdersTopic"
      Effect    = "Allow"
      Principal = {
        Service = "sns.amazonaws.com"
      }

      Action   = "sqs:SendMessage"
      Resource = aws_sqs_queue.notifications.arn

      Condition = {
        ArnEquals = {
          "aws:SourceArn" = aws_sns_topic.orders.arn
        }
      }
    }]
  })
}

resource "aws_sqs_queue_policy" "analytics" {
  queue_url = aws_sqs_queue.analytics.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "AllowOrdersTopic"
      Effect    = "Allow"
      Principal = {
        Service = "sns.amazonaws.com"
      }

      Action   = "sqs:SendMessage"
      Resource = aws_sqs_queue.analytics.arn

      Condition = {
        ArnEquals = {
          "aws:SourceArn" = aws_sns_topic.orders.arn
        }
      }
    }]
  })
}

# -------------------------------------
# Filtered notification subscription.
# -------------------------------------

resource "aws_sns_topic_subscription" "notifications" {
  topic_arn = aws_sns_topic.orders.arn
  protocol  = "sqs"
  endpoint  = aws_sqs_queue.notifications.arn

  # Deliver the event body directly to SQS.
  raw_message_delivery = true

  filter_policy_scope = "MessageAttributes"

  filter_policy = jsonencode({
    event_type = ["order.created"]
  })

  depends_on = [
    aws_sqs_queue_policy.notifications
  ]
}

# -------------------------------------
# Unfiltered analytics subscription.
# -------------------------------------

resource "aws_sns_topic_subscription" "analytics" {
  topic_arn = aws_sns_topic.orders.arn
  protocol  = "sqs"
  endpoint  = aws_sqs_queue.analytics.arn

  raw_message_delivery = true

  depends_on = [
    aws_sqs_queue_policy.analytics
  ]
}

The topic delivers events to two independent queues. The notification subscription matches only the order.created message attribute, while the analytics subscription has no filter and receives every event.

The raw_message_delivery = true setting means SQS receives the original published message body rather than the SNS JSON notification envelope. Message attributes remain available for compatible subscriptions, subject to documented raw-delivery constraints.

The queues' dead-letter queues protect against consumer processing failures. This demo does not configure separate SNS subscription delivery DLQs. For a business-critical system, add those independently, with queue policies allowing SNS to write delivery failures.

Create outputs.tf:

output "topic_arn" {
  value = aws_sns_topic.orders.arn
}

output "notification_queue_url" {
  value = aws_sqs_queue.notifications.id
}

output "analytics_queue_url" {
  value = aws_sqs_queue.analytics.id
}

output "notification_dlq_url" {
  value = aws_sqs_queue.notification_dlq.id
}

output "analytics_dlq_url" {
  value = aws_sqs_queue.analytics_dlq.id
}

3. Publish Events Using Python

Create requirements.txt:

boto3

Create publisher.py:

import json
import os
import uuid
from datetime import datetime, timezone

import boto3
from botocore.config import Config


TOPIC_ARN = os.environ["SNS_TOPIC_ARN"]
REGION = os.getenv("AWS_REGION", "us-west-2")

sns = boto3.client(
    "sns",
    region_name=REGION,
    config=Config(
        retries={
            "mode": "standard",
            "max_attempts": 5,
        }
    ),
)


def publish_event(
    event_type: str,
    order_id: str,
    event_id: str,
) -> str:
    # Keep event_id stable when retrying
    # publication of the same business event.
    event = {
        "event_id": event_id,
        "event_type": event_type,
        "schema_version": 1,
        "order_id": order_id,
        "created_at": datetime.now(
            timezone.utc
        ).isoformat(),
    }

    response = sns.publish(
        TopicArn=TOPIC_ARN,
        Message=json.dumps(event),
        MessageAttributes={
            "event_type": {
                "DataType": "String",
                "StringValue": event_type,
            }
        },
    )

    return response["MessageId"]


def main():
    # Expected in both queues.
    created_id = publish_event(
        event_type="order.created",
        order_id="order-1001",
        event_id=str(uuid.uuid4()),
    )

    # Expected only in analytics.
    cancelled_id = publish_event(
        event_type="order.cancelled",
        order_id="order-1002",
        event_id=str(uuid.uuid4()),
    )

    print("Created event SNS ID:", created_id)
    print("Cancelled event SNS ID:", cancelled_id)


if __name__ == "__main__":
    main()

The publisher sends two events to the same SNS topic. Because of subscription filtering, the notification queue should receive the order.created event, while the analytics queue should receive both events.

The event ID is intended to identify a business event, not an individual SNS API attempt. The demonstration generates each ID once; a production publisher should persist it with the business event so ambiguous retries reuse the same identifier.

4. Consume Filtered Events

Create consumer.py:

import json
import logging
import os
import signal
import time

import boto3
from botocore.config import Config


QUEUE_URL = os.environ["SQS_QUEUE_URL"]
REGION = os.getenv("AWS_REGION", "us-west-2")

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s %(levelname)s %(message)s",
)

logger = logging.getLogger(__name__)
running = True

sqs = boto3.client(
    "sqs",
    region_name=REGION,
    config=Config(
        connect_timeout=5,
        read_timeout=30,
        retries={
            "mode": "standard",
            "max_attempts": 5,
        },
    ),
)


def stop(signum, frame):
    global running
    running = False


signal.signal(signal.SIGTERM, stop)
signal.signal(signal.SIGINT, stop)


def process_event(event: dict) -> None:
    # Validate the event contract.
    if event.get("schema_version") != 1:
        raise ValueError("Unsupported schema version")

    event_id = event.get("event_id")
    order_id = event.get("order_id")
    event_type = event.get("event_type")

    if not all(
        isinstance(value, str) and value
        for value in (event_id, order_id, event_type)
    ):
        raise ValueError("Invalid event fields")

    # Demo processing only.
    # Production side effects must be idempotent.
    logger.info(
        "Processed event=%s type=%s order=%s",
        event_id,
        event_type,
        order_id,
    )


def consume_once() -> None:
    response = sqs.receive_message(
        QueueUrl=QUEUE_URL,
        WaitTimeSeconds=20,
        MaxNumberOfMessages=10,
    )

    for message in response.get("Messages", []):
        message_id = message["MessageId"]

        try:
            # Raw SNS delivery means the body is
            # the original event JSON.
            event = json.loads(message["Body"])

            if not isinstance(event, dict):
                raise ValueError("Invalid event body")

            process_event(event)

            # Acknowledge after successful processing.
            sqs.delete_message(
                QueueUrl=QUEUE_URL,
                ReceiptHandle=message["ReceiptHandle"],
            )

        except Exception:
            # Failed messages become visible again
            # and may eventually enter the DLQ.
            logger.exception(
                "Processing failed: message=%s",
                message_id,
            )


def main():
    logger.info("Starting SQS subscriber worker")

    while running:
        try:
            consume_once()
        except Exception:
            logger.exception("SQS polling failure")
            time.sleep(2)

    logger.info("Worker stopped")


if __name__ == "__main__":
    main()

This consumer polls one subscribed SQS queue and acknowledges messages only after successful processing. Run separate instances with different queue URLs to represent the notification and analytics services.

In production, processing logic should record completed event IDs in durable storage and use an appropriate atomicity mechanism for business updates. The logging operation in this demo does not require such protection, but email sends, billing changes, and inventory adjustments do.

5. Deploy, Test, and Clean Up

Deploy the Terraform configuration and publish the sample events:

# Initialize and validate infrastructure.
terraform init
terraform fmt
terraform validate

# Review and deploy resources.
terraform plan
terraform apply

# Configure the SNS publisher.
export AWS_REGION="us-west-2"
export SNS_TOPIC_ARN="$(terraform output -raw topic_arn)"

# Install dependencies.
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Publish two different order events.
python publisher.py

# Start a consumer for notifications.
export SQS_QUEUE_URL="$(
  terraform output -raw notification_queue_url
)"

python consumer.py

Stop the notification worker using Ctrl+C. Then start the analytics consumer with its own queue URL:

# The analytics subscriber receives all event types.
export SQS_QUEUE_URL="$(
  terraform output -raw analytics_queue_url
)"

python consumer.py

# In another terminal, inspect queue metrics.
aws sqs get-queue-attributes \
  --queue-url "$SQS_QUEUE_URL" \
  --attribute-names \
    ApproximateNumberOfMessages \
    ApproximateNumberOfMessagesNotVisible

# When finished, remove the disposable resources.
terraform destroy

The notification consumer should process only the order-created event, while analytics should process both events. Subscription filtering can take time to propagate after initial creation, so allow the subscriptions to become fully effective before using the demonstration as a strict filtering test.

The workers use long polling and continue until stopped. Terraform destruction deletes queues and their remaining messages, so verify that no messages need to be retained before cleaning up.

Security, Performance, and Cost

SNS removes messaging infrastructure management, but security and cost still depend on which principals can publish messages, which endpoints can subscribe, and how much delivery fan-out each event generates.

Applications should treat event payloads as externally distributed data contracts. Messages must contain enough information for consumers without unnecessarily exposing credentials, personal information, or sensitive internal state.

IAM, Encryption, and Endpoint Security

SNS uses IAM policies and topic resource policies to control publishing and subscription operations. An application publisher usually requires sns:Publish for a specific topic rather than broad access to every topic in an account.

When SNS delivers to SQS, the destination queue needs a resource policy permitting sns.amazonaws.com to call sqs:SendMessage, preferably restricted to the source topic ARN.

Important security practices include:

  • Use least-privilege topic and queue policies.
  • Restrict cross-account publication and subscription to authorized principals.
  • Use HTTPS for communication with AWS APIs.
  • Enable server-side encryption where data sensitivity requires it.
  • Use appropriate KMS key policies when configuring customer-managed keys.
  • Protect HTTPS subscribers with suitable endpoint authentication and verification.
  • Avoid placing credentials or unnecessary private data in event payloads.

For encrypted SNS-to-SQS delivery, verify that any customer-managed KMS keys permit the required AWS service principals to perform the encryption and decryption operations associated with delivery.

Misconfigured KMS policies can prevent delivery even when IAM and queue resource policies appear correct.

Throughput and Message Design

SNS Standard topics are designed for high-throughput fan-out. Actual service quotas, publish rates, subscription counts, and delivery characteristics depend on the Region, topic type, and destination.

Good message design has a greater long-term impact than simply maximizing publish throughput.

  • Stable event IDs: Allow consumers to detect repeated business events.
  • Explicit event types: Support filtering and independent consumer implementations.
  • Schema versions: Make compatibility and evolution manageable.
  • Small payloads: Avoid unnecessary message transfer and storage cost.
  • Business identifiers: Allow consumers to retrieve additional data when needed.
  • Consistent timestamps: Support diagnostics and latency measurement.

SNS supports message payloads up to 256 KB for typical publish operations. Store large files and datasets in S3 and publish references rather than embedding the content directly in SNS messages.

A message contract should evolve deliberately. Removing fields or changing meanings without subscriber coordination can break consumers even though SNS successfully delivers every message.

Pricing, Monitoring, and Delivery Metrics

SNS charges can depend on publish requests, notification deliveries, endpoint types, payload sizes, and applicable optional features. SMS and certain mobile or application-to-person deliveries have pricing considerations beyond ordinary application-to-application messaging.

One published event delivered to ten subscribers can generate more delivery-related usage than an event delivered to one subscriber. The complete cost of an SNS-to-SQS architecture also includes SQS requests, queue storage, worker compute, and any applicable KMS activity.

Useful operational metrics include:

  • NumberOfMessagesPublished: Publish activity accepted by the topic.
  • NumberOfNotificationsDelivered: Successful SNS delivery attempts to subscribed endpoints.
  • NumberOfNotificationsFailed: Failed deliveries requiring investigation.
  • NumberOfNotificationsFilteredOut: Messages excluded by subscription filtering.
  • Subscription DLQ depth: Events that could not be delivered.
  • Subscriber SQS queue age: Delay between delivery and actual processing.
  • Consumer processing failures: Application-level errors after successful SNS delivery.

Monitor delivery and processing separately. A topic may show successful deliveries while downstream queue backlogs increase because consumers are failing or cannot keep up.

Common Mistakes and Production Checklist

The most serious SNS mistakes involve treating event delivery as completed business processing, misunderstanding queue fan-out, or failing to protect events at the boundary between SNS and its destinations.

A reliable SNS architecture defines who owns each event, which subscribers need it, how it is routed, and what should happen when a subscriber cannot receive or process it.

Common SNS Mistakes

Common production mistakes include:

  • Confusing SNS with SQS: Expecting SNS subscriptions to behave like durable competing-consumer queues.
  • Using one shared queue for independent services: Workers compete instead of each receiving a copy.
  • Assuming Publish means processed: The producer reports downstream completion prematurely.
  • Ignoring subscription delivery failures: Messages are lost at the delivery boundary without a recovery mechanism.
  • Using only consumer DLQs: SNS-to-queue delivery failures remain unprotected.
  • Assuming Standard topics preserve order: Subscribers process related events out of sequence.
  • Assuming FIFO eliminates business duplicates: Retried consumer operations repeat side effects.
  • Misconfiguring filter policies: Required subscribers silently miss events.
  • Using an excessively broad topic policy: Unauthorized publishers can inject messages.
  • Ignoring KMS permissions: Encrypted SNS-to-SQS delivery unexpectedly fails.
  • Publishing oversized business objects: Payload limits and schema coupling increase.
  • Writing database changes and publishing separately: Crashes create missing or inconsistent events.

Production Checklist

Before relying on SNS for important business events, verify that:

  • Topic type matches ordering and endpoint requirements.
  • Events have stable IDs, explicit types, and schema versions.
  • Publishers have least-privilege IAM permissions.
  • Subscribers have the correct resource and KMS permissions.
  • Each independently operated service receives its own durable queue when necessary.
  • Subscription filters are tested against realistic event payloads.
  • SNS subscription delivery failures have a recovery strategy.
  • SQS consumer failures have separate retry and DLQ handling.
  • Consumers are idempotent for non-repeatable operations.
  • Events are published reliably after database transactions.
  • Message retention and replay procedures are documented.
  • Monitoring covers publish volume, delivery failure, filtering, and subscriber backlog.
  • Fan-out volume and delivery costs are monitored.

Frequently Asked Questions

SNS is straightforward to use for publishing notifications, but its delivery behavior differs from queues, durable event logs, and synchronous service calls.

What Is Amazon SNS Used For?

Amazon SNS is used to distribute messages and events to multiple subscribers. Common examples include microservice integration, event fan-out, infrastructure alerts, customer notifications, and routing events to independent SQS queues.

What Is the Difference Between SNS and SQS?

SNS distributes published messages to matching subscriptions. SQS stores messages in a queue until consumers receive and acknowledge them.

Use SNS when multiple independent services need copies of an event. Use SQS when work needs buffering and controlled asynchronous processing. Combine them when both behaviors are required.

Is Amazon SNS Push-Based or Pull-Based?

SNS generally uses a push-based delivery model: it sends notifications to supported subscriber endpoints.

When an SQS queue is subscribed to SNS, SNS delivers messages into the queue. Applications then retrieve those messages using the SQS pull-based API.

Does SNS Store Messages Permanently?

SNS Standard topics are not general-purpose permanent message stores for arbitrary replay. SNS manages messages during its delivery process, but applications should not depend on Standard topics as durable event history.

FIFO topics support specific message archiving and replay capabilities when configured. For independently buffered processing, SNS-to-SQS fan-out is a common architecture.

Does SNS Guarantee Message Ordering?

Standard SNS topics provide best-effort ordering. FIFO topics provide message ordering within message groups for compatible FIFO delivery paths.

The subscribed endpoint type matters. An SNS FIFO topic delivering into an SQS Standard queue does not provide the same ordered-consumption guarantees as an SQS FIFO queue.

What Is the Maximum SNS Message Size?

For typical SNS Publish operations, the maximum message payload is 256 KB. For larger data, use S3 or another storage service and publish a reference to the stored content.

Keeping messages small is generally better for throughput, cost, compatibility, and schema evolution.

Is Amazon SNS Free?

AWS may provide applicable SNS Free Tier allowances or credits to eligible accounts. Beyond those allowances, pricing depends on publish requests, delivery volume, endpoint type, and applicable optional features.

When using SNS with SQS, account for charges from both services and the consumers that process the events.

Conclusion

Amazon SNS is a fully managed publish/subscribe service that distributes events and notifications to multiple subscribers. It is particularly useful for decoupling microservices and building fan-out architectures where independent systems react to the same business event.

SNS simplifies distribution, but it does not eliminate messaging reliability challenges. Production systems must handle duplicate delivery, subscriber outages, message filtering, access policies, and eventual processing. Combining SNS with SQS provides independently buffered subscribers, while transactional outbox and idempotency patterns help maintain business correctness.

Key Takeaway: Use Amazon SNS to publish an event once and distribute it to many independent consumers. Combine it with durable queues, explicit delivery failure handling, and idempotent processing when those events trigger important business operations.

Explore these additional messaging and distributed systems articles to understand delivery guarantees, retry behavior, asynchronous processing, and event-driven architecture in production.

Comments (0)