Enterprise Integration Patterns with AWS Services
Published on · Updated on
Rewrote the guide to distinguish integration patterns from AWS service features, clarify delivery and workflow failure semantics, and add current AWS documentation.
Enterprise Integration Patterns describe recurring ways that independent systems exchange requests, messages, and events. AWS services provide building blocks for those patterns, but a service name is not a guarantee about the behavior of the whole workflow. A queue can buffer work without making a handler safe to retry; a router can deliver an event without proving that a business operation completed.
Start with the communication need and the failure boundary. Decide whether a caller must wait for a response, whether one message has one worker or many subscribers, whether work needs a durable backlog, and whether a workflow must coordinate several steps. Then select the AWS features that fit those requirements.
Common Integration Patterns in AWS
The same service can participate in more than one pattern, and a pattern often uses several services. This map describes common roles rather than fixed architecture recipes.
| Integration need | Pattern | Common AWS building blocks | What the pattern provides |
|---|---|---|---|
| A client needs an answer during the call | Request-reply | Amazon API Gateway and an HTTP service or synchronous AWS Lambda integration | A request path that returns a response to the caller |
| Work should wait until a worker can process it | Work queue / competing consumers | Amazon SQS | A retained backlog that separates the producer from workers |
| Several independent consumers need a notification | Publish-subscribe / fanout | Amazon SNS, or an EventBridge event bus for content-routed events | One publication can reach multiple subscriptions or targets |
| Different events need different destinations | Content-based router | EventBridge event patterns; SNS subscription filter policies | Matching events are sent to the destinations that selected them |
| A destination expects a different payload shape | Message translator | EventBridge input transformer, EventBridge Pipes, or application code | Fields are selected, renamed, or reshaped for the next integration |
| Processing needs data from another system | Content enricher | EventBridge Pipes enrichment or application code | Additional context is looked up and added before delivery |
| Several dependent actions need explicit coordination | Orchestration; sometimes a saga | AWS Step Functions with service integrations or task code | Sequencing, branching, error handling, and any business-defined compensation |
| Many records need continuous processing or independent readers | Event stream | Amazon Kinesis Data Streams | A retained stream that multiple processing applications can consume independently |
Filtering, transformation, and enrichment solve different problems. Filtering decides whether a message proceeds along a route. Transformation changes the payload sent onward. Enrichment obtains additional information, often by calling another service. Keeping those jobs distinct makes it clearer which component owns a decision, data contract, and failure.
AWS Services for Integration
Request-Reply with API Gateway and Lambda
Request-reply fits operations where the caller needs a result or acceptance response before it can continue. With an API Gateway Lambda proxy integration, API Gateway sends the HTTP request to Lambda and returns the function's response to the client. API Gateway can also proxy a request and response to a publicly routable HTTP endpoint. See the API Gateway HTTP API and Lambda integration guide and the guide to HTTP proxy integrations.
This path is synchronous from the caller's perspective; it does not make a multi-service business transaction atomic. Set timeouts for the request and its downstream calls. If the response is lost after a state change, a client may retry without knowing whether the first attempt committed. Use an idempotency key for operations that must not create duplicate effects, and return a stable result for repeated requests where the application can do so. AWS explains this approach in its guidance on safe retries with idempotent APIs.
Publish-Subscribe with SNS
Amazon SNS topics implement a push-based fanout pattern: a publisher sends a notification to a topic and SNS delivers it to the topic's subscribers. This works when several recipients need an event or notification without the producer calling each one. SNS supports subscriptions such as SQS queues, Lambda functions, and HTTP/S endpoints; see the SNS subscription guide. For asynchronous workers that need their own backlog, subscribe one SQS queue per consumer; SNS fans out the notification, while each queue lets its worker poll and process at its own pace. AWS describes this combination in its guide to SNS fanout to SQS for asynchronous processing.
An SNS subscription filter policy can limit which messages that subscriber receives. Policies can match message attributes or, when configured for it, properties in a JSON message body. A filter decides delivery; it does not rewrite or enrich the message. A message rejected by one subscription can still be delivered to other subscriptions whose policies match. See SNS message filtering.
Durable Work Queues with SQS
Amazon SQS is a durable queue for decoupling producers from workers. A consumer receives a message, which remains in the queue but becomes temporarily invisible while the consumer processes it. The consumer deletes it after successful processing. If it is not deleted before the visibility timeout expires, it can become available for another attempt. Queue retention, visibility timeout, and dead-letter queue settings govern different parts of that lifecycle; configure them for the work rather than assuming a queue completes or repairs failed work. See the SQS message lifecycle and visibility timeout guidance.
Standard queues provide at-least-once delivery, so a message can be received more than once. Make a worker's side effects idempotent, especially when it calls a database or external API.
FIFO queues add ordering within each message group and deduplication for repeated sends within a defined interval. The group key is the unit of ordering: one group is processed sequentially, while different groups can make progress independently, so a single busy group limits parallel work. FIFO queues also have separate throughput quotas and a high-throughput mode; check current quotas for the queue and Region before sizing them.
AWS describes FIFO queue processing as exactly-once, but that does not make a payment, database update, or other external effect an atomic exactly-once business transaction. If a consumer performs an effect and fails before deleting the message, the message can be received again after its visibility timeout. Use FIFO when its ordering and deduplication behavior fits the workflow, and still make effects safe to repeat. See AWS documentation for standard queue delivery, FIFO ordering and throughput guidance, and FIFO message groups and deduplication.
Content-Based Routing with EventBridge
EventBridge event buses route events by matching event patterns. An event can match multiple rules or subscribers, so a bus can support one-to-many routing as well as content-based routing. The default event bus and Custom Event Bus - Classic use rules and targets. For example, a Classic rule can select events from an order service whose detail-type is OrderPlaced and send matching events to a queue or function.
EventBridge's models have changed. The enhanced Custom Event Bus, announced on September 24, 2026, uses consumer-created subscribers with their own filters and optional transformations, and retains events for replay. Custom Event Bus - Classic remains available with its rules-and-targets model. Do not apply a Classic bus limitation or configuration description to the newer bus without checking the documentation for that model. The AWS launch announcement summarizes the change.
Failure handling is also bus-specific. A Classic rule can retry target delivery and use an SQS dead-letter queue for events that could not be delivered. The enhanced Custom Event Bus writes failure details and event identifiers to its SQS failure queue, not a copy of each original event; recovery replays the event from the bus while it is still retained. In either model, delivery to a target and successful processing by that target are separate outcomes. See AWS guidance for Classic target retries, Classic target dead-letter queues, and Custom Event Bus retries and failure recovery.
EventBridge Pipes serve a different shape of integration: a point-to-point connection from one supported source to one target, with optional filtering, enrichment, and transformation. A pipe is useful when a direct source-to-target flow needs those steps without a general event bus. See the EventBridge Pipes guide.
For a closer comparison of routing, retained events, competing workers, and the two DLQ boundaries, read our EventBridge vs. SQS guide.
Transformation Is Different from Enrichment
A transformer maps data that already exists. For a Classic EventBridge rule, an input transformer can select values from the original event and build a target-specific payload. A translator function can do more complex schema conversion when a template is insufficient. Neither operation necessarily adds new business facts.
An enricher obtains data that was not present in the original message—for example, looking up a customer's shipping address before starting fulfillment. EventBridge Pipes can call a supported enrichment service and then send the enriched result to its target. That lookup introduces a new dependency with its own latency, permissions, error handling, and freshness. Keep it explicit in the flow. See AWS documentation for EventBridge input transformation and Pipes enrichment.
Workflow Orchestration with Step Functions
AWS Step Functions provides a state machine to coordinate tasks, branches, waits, and service integrations. Task states can define Retry and Catch behavior. A retry runs the task again, so the task must tolerate repeated execution or use an idempotency mechanism when it changes state. Step Functions coordinates the workflow; it does not supply a universal transaction across the systems called by each task. See Step Functions error handling.
A saga is a distributed workflow made of local transactions. If a later step cannot continue, the workflow may issue compensating business actions for earlier completed steps—for example, release a stock reservation or refund an authorized payment. Those actions must be designed and implemented; they are not automatic rollback. Compensation can itself fail, and a refund does not erase the original charge. AWS's saga guidance explains orchestration and choreography tradeoffs.
Event Stream Processing
A queue holds work for a consumer to take. A stream is useful when records form a continuous feed that multiple processing applications may read independently, such as application logs, clickstream records, or operational metrics. Amazon Kinesis Data Streams supports multiple applications consuming the same stream, allowing separate processing and archiving paths. Choose a stream when retained records and independent readers are part of the requirement; use queue or event-bus delivery semantics when the problem is dispatching work or routing business events. See What is Amazon Kinesis Data Streams?.
Integration Pattern Example: Order Processing
An order workflow can combine request-reply, routing, a work queue, and orchestration without making them interchangeable:
- Accept the request. An API service validates the order and returns an appropriate response to the client. If fulfillment continues asynchronously, the API can acknowledge acceptance without keeping the client connected for every later step.
- Publish a business event. Once the application has accepted the order, it emits an
OrderPlacedevent with a stable identifier and documented fields. EventBridge can route that event by its type and contents; SNS can fan out a notification to known subscribers. - Buffer independent work. A fulfillment queue can hold work while workers are busy or unavailable. Consumers delete messages after success, handle duplicate delivery safely, and send persistent failures to a configured dead-letter queue for investigation.
- Coordinate dependent actions. If inventory reservation must precede payment, a Step Functions workflow can make that sequence and its retry policy explicit. The business defines what happens if payment fails after reservation, such as releasing the reservation. That compensation is a new action with its own failure handling.
- Adapt data at boundaries. A target input transformer can pass only the fields a notification target needs. If fulfillment requires a current address lookup, an enrichment step fetches it; filtering the event would not perform that lookup.
If saving an order and publishing its event are separate writes, a crash between them can leave an accepted order with no event, or an event without the expected committed record. A transactional outbox can record the order and a pending event in one local transaction, then publish asynchronously. Its relay can still retry, so consumers need idempotency.
The example does not imply that every order system needs all of these services. It shows how to assign each responsibility to a boundary and reason about what happens when a call times out, a target is unavailable, or a message is delivered again.
Design Around the Failure Boundary
Choose a pattern by the communication and recovery behavior the application needs: a waiting response, a durable work queue, fanout, content-based routing, payload adaptation, enrichment, or explicit workflow coordination. Then verify the delivery, ordering, retention, retry, and replay behavior for the exact AWS feature in use. In particular, distinguish a successful delivery from successful business processing and make retried side effects safe. For a focused guide to retries, queues, idempotency, and failure recovery across service boundaries, see AWS Cross-Service Resilience: Retries, Queues, and Disaster Recovery.