AWS Lambda Event Sources Explained

AWS Lambda is event-driven by design. A Lambda function usually does not run by itself. It runs because something invokes it: an HTTP request, uploaded file, queue message, database stream record, schedule, or event from another AWS service.

These triggers are called event sources. Understanding Lambda event sources is one of the most important skills for building serverless applications, because the event source defines how your function receives data, how retries work, whether batching is available, how scaling behaves, and what failure handling strategy you need.

Table of Contents

• Introduction to AWS Lambda Event Sources
  • What Is an Event Source?
  • How Lambda Receives Events
  • Synchronous vs Asynchronous Invocation
• HTTP-Based Event Sources
  • Amazon API Gateway
  • AWS Lambda Function URLs
  • Application Load Balancer
• Storage Event Sources
  • Amazon S3 Events
• Messaging Event Sources
  • Amazon SQS
  • Amazon SNS
  • Amazon MQ
• Event-Driven Event Sources
  • Amazon EventBridge
  • AWS CloudTrail Events
  • Amazon EventBridge Scheduler
• Database Event Sources
  • Amazon DynamoDB Streams
  • Amazon DocumentDB Change Streams
• Streaming Event Sources
  • Amazon Kinesis Data Streams
  • Amazon MSK
  • Self-Managed Apache Kafka
• Workflow Event Sources
  • AWS Step Functions
  • AWS CodePipeline
• Monitoring and Logging Event Sources
  • Amazon CloudWatch Logs
  • AWS Config
• Event Source Comparison
• Choosing the Right Event Source
• AWS Lambda Event Source Best Practices
• Conclusion

Introduction to AWS Lambda Event Sources

An event source is anything that can invoke a Lambda function. It can be an AWS service, an HTTP endpoint, a queue, a stream, a database change, a schedule, or direct invocation through the AWS SDK.

The important part is not only what triggers Lambda, but also how the trigger behaves. API Gateway waits for a response. S3 invokes Lambda asynchronously. SQS is polled by Lambda through an event source mapping. Kinesis sends records in batches. Step Functions uses Lambda as one step in a workflow.

What Is an Event Source?

An event source produces the input event that Lambda receives. For example, when API Gateway invokes Lambda, the event may contain the HTTP method, path, headers, query parameters, and request body.

{
  "httpMethod": "GET",
  "path": "/users/42",
  "queryStringParameters": {
    "includeOrders": "true"
  }
}

For S3, the event looks completely different because the input is about a bucket and object key.

{
  "Records": [
    {
      "eventSource": "aws:s3",
      "s3": {
        "bucket": {
          "name": "uploads-bucket"
        },
        "object": {
          "key": "images/photo.jpg"
        }
      }
    }
  ]
}

Key point: the Lambda handler signature may look the same, but the event structure depends on the event source.

How Lambda Receives Events

Lambda can receive events in several ways:

• Direct synchronous invocation: the caller waits for the Lambda response.
• Asynchronous invocation: Lambda queues the event and processes it later.
• Event source mapping: Lambda polls a stream or queue and invokes the function with records.
• Service integration: another AWS service invokes Lambda as part of its workflow.

Synchronous vs Asynchronous Invocation

Synchronous invocation means the caller waits for the result. This is common for HTTP APIs. If Lambda fails, the caller usually receives an error response.

Asynchronous invocation means the event source sends the event to Lambda and does not wait for the function result. Lambda processes the event in the background and handles retries according to async invocation settings.

HTTP-Based Event Sources

HTTP-based event sources are used when Lambda should respond to web requests. They are common for APIs, webhooks, backend-for-frontend layers, internal tools, and lightweight services.

Amazon API Gateway

Amazon API Gateway is the most common way to expose Lambda as an HTTP API. It can handle routing, authentication, throttling, request validation, custom domains, stages, and integrations with other AWS services.

When to Use

• REST APIs for web and mobile applications.
• Public APIs with authentication, throttling, and custom domains.
• Webhook receivers from third-party systems.
• Microservice endpoints where each route maps to one Lambda function or a small group of functions.

Example Handler

import json

def lambda_handler(event, context):
    user_id = event["pathParameters"]["userId"]

    user = {
        "id": user_id,
        "name": "Demo User"
    }

    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "application/json"
        },
        "body": json.dumps(user)
    }

Advantages

• Good API features: routing, throttling, authorization, stages, custom domains.
• Serverless: no web server management.
• Scalable: works well for many request/response workloads.

Limitations

• Timeouts matter: not suitable for long-running request processing.
• Cold starts can affect latency: especially for heavier runtimes or VPC functions.
• API design can become messy: if routing and function boundaries are not planned.

Rule of thumb: use API Gateway when Lambda is the backend for a real HTTP API.

AWS Lambda Function URLs

Lambda Function URLs provide a simple HTTPS endpoint directly for a Lambda function. They are easier than API Gateway but have fewer API management features.

When to Use

• Simple webhooks
• Internal tools
• Quick prototypes
• Single-function endpoints

Example Handler

import json

def lambda_handler(event, context):
    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "Hello from Lambda Function URL"
        })
    }

Function URL vs API Gateway

Rule of thumb: use Function URLs for simple endpoints, not for complex public APIs.

Application Load Balancer

Application Load Balancer can also invoke Lambda. This is useful when Lambda is part of an architecture that already uses ALB routing.

When to Use

• Existing ALB-based architectures
• Path-based routing between containers, EC2, and Lambda
• Internal HTTP services
• Hybrid compute where some routes go to ECS and some go to Lambda

Example Handler

import json

def lambda_handler(event, context):
    return {
        "statusCode": 200,
        "statusDescription": "200 OK",
        "isBase64Encoded": False,
        "headers": {
            "Content-Type": "application/json"
        },
        "body": json.dumps({
            "message": "Response from Lambda behind ALB"
        })
    }

Rule of thumb: use ALB + Lambda when Lambda is one target type inside a broader load-balanced architecture.

Storage Event Sources

Storage event sources trigger Lambda when files or objects change. This is useful for import pipelines, image processing, document handling, data lake ingestion, and file validation.

Amazon S3 Events

Amazon S3 can invoke Lambda when an object is created, deleted, restored, or changed depending on the configured event type.

When to Use

• Image resizing after upload.
• CSV or JSON import into a database.
• PDF processing or metadata extraction.
• Antivirus scanning for uploaded files.
• Data lake ingestion after files land in S3.

Example S3 Event

{
  "Records": [
    {
      "eventSource": "aws:s3",
      "eventName": "ObjectCreated:Put",
      "s3": {
        "bucket": {
          "name": "my-upload-bucket"
        },
        "object": {
          "key": "uploads/report.csv"
        }
      }
    }
  ]
}

Example Handler

def lambda_handler(event, context):
    for record in event["Records"]:
        bucket = record["s3"]["bucket"]["name"]
        key = record["s3"]["object"]["key"]

        print(f"Processing s3://{bucket}/{key}")

    return {
        "processed": len(event["Records"])
    }

Important Warning

Avoid recursive triggers. If Lambda writes output to the same S3 prefix that triggered it, the output file can invoke the same function again.

Bad:
uploads/file.csv -> Lambda -> writes uploads/processed-file.csv -> Lambda runs again

Better:
uploads/file.csv -> Lambda -> writes processed/file.csv

Rule of thumb: use separate input and output prefixes or separate buckets.

Messaging Event Sources

Messaging event sources are used when work should be processed asynchronously. They are often better than doing slow work inside an HTTP request.

Amazon SQS

Amazon SQS is one of the most important Lambda event sources. Lambda polls the queue, receives messages in batches, invokes your function, and deletes messages after successful processing.

When to Use

• Background jobs
• Email sending
• Report generation
• Order processing
• Retryable external API calls
• Buffering traffic spikes

Example SQS Event

{
  "Records": [
    {
      "messageId": "msg-1",
      "body": "{\"orderId\":\"ord_123\",\"amount\":99.95}"
    }
  ]
}

Example Handler

import json

def lambda_handler(event, context):
    for record in event["Records"]:
        message = json.loads(record["body"])
        print("Processing order:", message["orderId"])

    return {
        "processed": len(event["Records"])
    }

Best Practices

• Configure a dead-letter queue for messages that fail repeatedly.
• Use batch size carefully. Larger batches improve throughput but complicate failure handling.
• Use partial batch responses when only some records fail.
• Set visibility timeout longer than the Lambda timeout.
• Make processing idempotent because duplicate delivery can happen.

Rule of thumb: use SQS when work can be done later and should survive temporary failures.

Amazon SNS

Amazon SNS is a publish/subscribe service. A publisher sends a message to a topic, and SNS delivers it to subscribers such as Lambda functions, SQS queues, HTTP endpoints, or email addresses.

When to Use

• Fan-out notifications
• One event to multiple consumers
• Simple pub/sub
• Notification pipelines

Example SNS Event

{
  "Records": [
    {
      "EventSource": "aws:sns",
      "Sns": {
        "Subject": "OrderCreated",
        "Message": "{\"orderId\":\"ord_123\"}"
      }
    }
  ]
}

Example Handler

import json

def lambda_handler(event, context):
    for record in event["Records"]:
        message = json.loads(record["Sns"]["Message"])
        print("Received SNS message:", message)

    return {
        "processed": len(event["Records"])
    }

SNS vs SQS

Rule of thumb: use SNS when multiple systems should react to the same event.

Amazon MQ

Amazon MQ is a managed message broker service for Apache ActiveMQ and RabbitMQ. Lambda can consume messages from supported Amazon MQ brokers.

When to Use

• Migration from RabbitMQ or ActiveMQ
• Enterprise messaging systems
• Legacy applications that already use broker-based messaging
• AMQP or JMS-style workflows

Example Handler Idea

def lambda_handler(event, context):
    for record in event["rmqMessagesByQueue"]:
        print("Processing broker message:", record)

    return {
        "status": "done"
    }

Rule of thumb: use Amazon MQ when you need compatibility with existing broker-based systems, not when SQS is enough.

Event-Driven Event Sources

Event-driven sources are used when systems communicate through facts that already happened, such as OrderCreated, UserRegistered, or PaymentFailed.

Amazon EventBridge

Amazon EventBridge is an event bus service. It can receive events from AWS services, custom applications, SaaS integrations, and schedules, then route those events to targets such as Lambda.

When to Use

• Domain events
• Decoupled microservices
• SaaS integrations
• Event routing by source, type, or payload fields
• Scheduled jobs

Example EventBridge Event

{
  "source": "app.orders",
  "detail-type": "OrderCreated",
  "detail": {
    "orderId": "ord_123",
    "customerId": "cus_456",
    "amount": 120.50
  }
}

Example Handler

def lambda_handler(event, context):
    detail_type = event["detail-type"]
    detail = event["detail"]

    print(f"Received event type: {detail_type}")
    print("Event detail:", detail)

    return {
        "status": "processed"
    }

EventBridge vs SNS

Rule of thumb: use EventBridge when events are part of your application architecture, not just notifications.

AWS CloudTrail Events

AWS CloudTrail records AWS API activity. CloudTrail events can be routed through EventBridge and used to trigger Lambda functions for security, compliance, automation, or governance workflows.

When to Use

• Security monitoring
• Detecting sensitive AWS API calls
• Automatic remediation
• Compliance workflows

Example Use Case

CloudTrail detects security group change
EventBridge matches the event
Lambda validates the change
Lambda sends alert or reverts unsafe rule

Rule of thumb: use CloudTrail events when Lambda should react to AWS account activity.

Amazon EventBridge Scheduler

EventBridge Scheduler can invoke Lambda on a schedule. This is useful for cron-like jobs without maintaining a server or cron daemon.

When to Use

• Nightly cleanup jobs
• Hourly synchronization
• Scheduled reports
• Periodic health checks
• Delayed one-time tasks

Schedule Examples

rate(5 minutes)
rate(1 hour)
cron(0 8 * * ? *)

Example Handler

def lambda_handler(event, context):
    print("Running scheduled job")
    run_cleanup()

    return {
        "status": "complete"
    }

def run_cleanup():
    print("Cleanup logic here")

Rule of thumb: use Scheduler when time itself is the event source.

Database Event Sources

Database event sources trigger Lambda when data changes. They are useful for audit logs, search indexing, denormalization, notifications, and event-driven projections.

Amazon DynamoDB Streams

DynamoDB Streams capture changes to DynamoDB table items. Lambda can process insert, update, and delete events from the stream.

When to Use

• Audit logging
• Search index synchronization
• Denormalized views
• Notifications after data changes
• Event-driven projections

Example Stream Event

{
  "Records": [
    {
      "eventName": "INSERT",
      "eventSource": "aws:dynamodb",
      "dynamodb": {
        "NewImage": {
          "id": {
            "S": "user_123"
          },
          "email": {
            "S": "alex@example.com"
          }
        }
      }
    }
  ]
}

Example Handler

def lambda_handler(event, context):
    for record in event["Records"]:
        event_name = record["eventName"]

        if event_name == "INSERT":
            new_image = record["dynamodb"]["NewImage"]
            print("New item:", new_image)

        elif event_name == "MODIFY":
            print("Item updated")

        elif event_name == "REMOVE":
            print("Item deleted")

    return {
        "processed": len(event["Records"])
    }

Important Warning

Avoid infinite update loops. If Lambda reacts to a DynamoDB change and writes back to the same table, that write can trigger the stream again.

Bad:
DynamoDB update -> Stream -> Lambda -> Update same table -> Stream again

Better:
DynamoDB update -> Stream -> Lambda -> Write to another table or use guard conditions

Rule of thumb: use DynamoDB Streams when you need to react to item-level changes.

Amazon DocumentDB Change Streams

Amazon DocumentDB change streams can be used to process database changes from DocumentDB collections. This is useful when your application is document-oriented and needs reactive processing.

When to Use

• Document database change processing
• Audit trails
• Search synchronization
• Data replication workflows

Example Handler Idea

def lambda_handler(event, context):
    for record in event["events"]:
        print("DocumentDB change event:", record)

    return {
        "processed": len(event["events"])
    }

Rule of thumb: use DocumentDB change streams when DocumentDB is the source of truth and downstream systems need to react to changes.

Streaming Event Sources

Streaming event sources are designed for high-throughput ordered records. Lambda reads records in batches from streams and processes them by shard or partition.

Amazon Kinesis Data Streams

Kinesis Data Streams is used for real-time streaming data such as clickstream events, logs, telemetry, and analytics pipelines.

When to Use

• Real-time analytics
• Clickstream processing
• IoT telemetry
• Log pipelines
• Ordered stream processing

Example Kinesis Event

{
  "Records": [
    {
      "eventSource": "aws:kinesis",
      "kinesis": {
        "partitionKey": "user_123",
        "data": "eyJldmVudCI6ICJjbGljayJ9"
      }
    }
  ]
}

Example Handler

import base64
import json

def lambda_handler(event, context):
    for record in event["Records"]:
        payload = base64.b64decode(record["kinesis"]["data"])
        data = json.loads(payload)

        print("Stream event:", data)

    return {
        "processed": len(event["Records"])
    }

Kinesis vs SQS

Rule of thumb: use Kinesis for streams, not simple job queues.

Amazon MSK

Amazon MSK is managed Apache Kafka. Lambda can consume records from Kafka topics through an event source mapping.

When to Use

• Kafka-based event streaming
• Existing Kafka architecture
• High-throughput event pipelines
• Event replay
• Multiple consumer groups

Example Handler Idea

import base64
import json

def lambda_handler(event, context):
    for topic_partition, records in event["records"].items():
        for record in records:
            value = base64.b64decode(record["value"])
            data = json.loads(value)

            print("Kafka record:", data)

    return {
        "status": "processed"
    }

Rule of thumb: use MSK with Lambda when your event platform is Kafka and the consumer logic is suitable for Lambda execution limits.

Self-Managed Apache Kafka

Lambda can also consume from self-managed Apache Kafka. This is useful when Kafka runs outside MSK but you still want Lambda consumers.

When to Use

• Existing self-managed Kafka clusters
• Hybrid cloud architectures
• Migration from custom consumers to Lambda
• Serverless consumers for selected topics

Important Considerations

• Networking must be configured correctly.
• Authentication and TLS require careful setup.
• Batch size and processing time must fit Lambda limits.
• Operational ownership remains with your Kafka platform.

Rule of thumb: use self-managed Kafka integration only when Kafka is already part of your architecture.

Workflow Event Sources

Workflow event sources invoke Lambda as part of a larger process. Lambda is not the whole application; it is one task inside an orchestrated workflow.

AWS Step Functions

AWS Step Functions can call Lambda functions as steps in a state machine. This is useful for multi-step workflows, retries, branches, waits, and long-running business processes.

When to Use

• Order processing
• Approval workflows
• ETL pipelines
• Multi-step business processes
• Workflows with retries and compensation

Example Workflow

Validate Order
  -> Reserve Inventory
  -> Charge Payment
  -> Send Confirmation
  -> Update Order Status

Example Lambda Step

def lambda_handler(event, context):
    order_id = event["orderId"]

    return {
        "orderId": order_id,
        "validationStatus": "valid"
    }

Rule of thumb: use Step Functions when the flow has multiple steps and the workflow itself must be observable.

AWS CodePipeline

AWS CodePipeline can use Lambda for custom actions in deployment workflows. Lambda can validate artifacts, run custom checks, notify systems, or integrate with tools not directly supported by CodePipeline.

When to Use

• Custom deployment checks
• Approval automation
• Artifact validation
• Custom notifications
• Integration with external systems

Example Handler Idea

def lambda_handler(event, context):
    job_id = event["CodePipeline.job"]["id"]

    print("Processing CodePipeline job:", job_id)

    return {
        "jobId": job_id,
        "status": "success"
    }

Rule of thumb: use Lambda in CodePipeline when the deployment process needs custom logic.

Monitoring and Logging Event Sources

Monitoring and logging event sources invoke Lambda when operational data is produced. This is useful for security monitoring, alerting, log transformation, and automation.

Amazon CloudWatch Logs

CloudWatch Logs subscriptions can send log events to Lambda. This allows you to process logs in near real time.

When to Use

• Security log scanning
• Error detection
• Log enrichment
• Forwarding logs to external systems
• Custom metrics extraction

Example Handler Idea

import base64
import gzip
import json

def lambda_handler(event, context):
    compressed_payload = base64.b64decode(event["awslogs"]["data"])
    uncompressed_payload = gzip.decompress(compressed_payload)
    log_data = json.loads(uncompressed_payload)

    for log_event in log_data["logEvents"]:
        print("Log message:", log_event["message"])

    return {
        "processed": len(log_data["logEvents"])
    }

Rule of thumb: use CloudWatch Logs subscriptions when logs themselves should trigger processing.

AWS Config

AWS Config can use Lambda for custom compliance rules. Lambda evaluates whether resources follow required policies.

When to Use

• Custom compliance checks
• Security governance
• Resource configuration validation
• Policy enforcement

Example Use Case

AWS Config detects resource configuration
Lambda evaluates compliance
Lambda returns COMPLIANT or NON_COMPLIANT

Rule of thumb: use AWS Config + Lambda when compliance logic is specific to your organization.

Event Source Comparison

Choosing the Right Event Source

The best event source depends on the problem you are solving. Do not choose a trigger only because it is easy to configure. Choose it based on delivery behavior, retry needs, ordering requirements, latency expectations, and failure handling.

AWS Lambda Event Source Best Practices

Different event sources behave differently, but several best practices apply to almost all Lambda event-driven systems.

Idempotency

Idempotency means the same event can be processed more than once without corrupting data or creating duplicate side effects. This is critical because retries and duplicate deliveries can happen.

def process_payment(event):
    payment_id = event["paymentId"]

    if payment_already_processed(payment_id):
        return "already_processed"

    charge_customer(event)
    mark_payment_processed(payment_id)

    return "processed"

Retries and Dead Letter Queues

Always understand how your event source retries failed events. For queues and asynchronous systems, configure failure destinations or dead-letter queues where appropriate.

• SQS: use a dead-letter queue for messages that fail repeatedly.
• Async Lambda invocation: use destinations or DLQ for failed events.
• Streams: handle poison records carefully because they can block shard progress.
• HTTP APIs: return clear error responses and avoid retrying unsafe operations blindly.

Batch Processing

Stream and queue sources can send records in batches. Batching improves throughput, but failure handling becomes more important.

def lambda_handler(event, context):
    failed_items = []

    for record in event["Records"]:
        try:
            process_record(record)
        except Exception:
            failed_items.append({
                "itemIdentifier": record["messageId"]
            })

    return {
        "batchItemFailures": failed_items
    }

Rule of thumb: if one bad record should not fail the entire batch, use partial batch failure handling where supported.

Concurrency Management

Lambda can scale quickly, but downstream systems may not. Databases, APIs, and legacy systems can fail if too many Lambda executions run at the same time.

Problem:
10,000 queue messages
  -> many Lambda executions
  -> too many database connections

Better:
SQS batch size + reserved concurrency + RDS Proxy + backpressure

Event Filtering

Use event filtering when possible so Lambda receives only events it actually needs. This reduces cost, noise, and unnecessary invocations.

• EventBridge rules can match specific event patterns.
• S3 notifications can filter by prefix and suffix.
• Event source mappings can filter some stream or queue events before invocation.

Monitoring and Logging

Event-driven systems can be hard to debug because one user action may produce many events. Use structured logs, correlation IDs, metrics, alarms, and tracing.

import json
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    logger.info(json.dumps({
        "requestId": context.aws_request_id,
        "eventSource": detect_event_source(event),
        "message": "Processing Lambda event"
    }))

def detect_event_source(event):
    if "Records" in event and event["Records"]:
        return event["Records"][0].get("eventSource", "unknown")

    return "unknown"

Conclusion

AWS Lambda event sources define how your serverless application behaves. The Lambda code is only one part of the design. The event source controls the shape of the input, invocation style, retry behavior, batching model, scaling behavior, and failure handling strategy.

Use API Gateway for REST APIs, Function URLs for simple endpoints, S3 events for file processing, SQS for background jobs, SNS for fan-out, EventBridge for domain events, DynamoDB Streams for database changes, Kinesis or Kafka for streaming, Step Functions for workflows, and EventBridge Scheduler for scheduled tasks.

Key takeaway: choosing the right Lambda event source is an architecture decision, not just a configuration detail. Good serverless systems are designed around clear event ownership, safe retries, idempotent processing, observability, and downstream limits.

More Articles to Read

• AWS Lambda Execution Lifecycle Explained
• AWS Lambda Cold Starts Explained
• Threads, Async, Await, and Event Loops in AWS Lambda
• AWS Lambda Architecture Patterns
• AWS Lambda Performance Optimization
• AWS Lambda Best Practices for Production
• AWS Lambda Concurrency and Scaling