Managing Data Across Multiple Services

Data management becomes one of the hardest parts of microservices architecture. Inside a monolith, multiple modules can share a database, execute joins, and update related records inside a single transaction. Once business capabilities become independent services, those assumptions no longer hold.

A scalable microservices architecture usually gives each service clear ownership of its data. That improves autonomy and isolation, but cross-service transactions, queries, consistency, reporting, and schema evolution must be handled explicitly.

Table of Contents

• Data Ownership in Microservices
  • Database per Service
  • Shared Database
• Consistency Across Services
  • Local Transactions
  • Eventual Consistency
• Distributed Business Transactions
  • Saga Pattern
  • Reliable Event Publication
• Querying Data Across Services
  • API Composition
  • Materialized Read Models
• Production Design Example
  • Distributed Order Processing
• Common Mistakes
• Production Checklist
• Conclusion
  • Key Takeaway
• More Articles to Read

Data Ownership in Microservices

The fundamental data-management rule in microservices is that business state should have one authoritative owner. Other services can consume or replicate that information, but they should not bypass the owning service's business rules by modifying its storage directly.

For example, Inventory should own stock levels, Payments should own payment state, and Ordering should own order lifecycle state.

Order Service            Inventory Service          Payment Service
     |                          |                          |
     v                          v                          v
 Order DB                  Inventory DB                 Payment DB
     |                          |                          |
     | authoritative           | authoritative            | authoritative
     | order state             | stock state              | payment state
     |                          |                          |
     +----------- APIs / Events / Messages --------------+

Ownership is primarily a logical boundary. Separate services do not necessarily require separate physical database servers. Several services can use the same database cluster while owning isolated schemas or databases, provided cross-service access is prevented.

Database per Service

The database-per-service pattern gives each service exclusive control over its persistent state. Other services communicate through APIs or events instead of querying its tables.

This provides several advantages:

• Independent schema evolution: internal tables can change without coordinating every consumer.
• Deployment autonomy: database migrations remain part of the owning service's release process.
• Failure isolation: expensive queries from one service are less likely to affect unrelated workloads.
• Technology flexibility: different workloads can use relational, document, key-value, or search storage when justified.
• Security boundaries: database credentials can restrict access to owned data.

The cost is that joins and ACID transactions no longer naturally span business capabilities.

Database-per-service should therefore be treated as an ownership model, not a requirement to deploy hundreds of independent database servers.

Shared Database

A shared database allows multiple services to access the same schema or tables directly.

This can simplify migrations from a monolith because existing joins and transactions continue working, but it creates strong coupling:

A shared database can be reasonable during incremental decomposition, but ownership should still be explicit. For example, separate schemas and database roles can prevent one service from writing another service's tables.

The dangerous form is not simply sharing database infrastructure. It is sharing responsibility for the same business state.

Consistency Across Services

When each service owns its data, consistency must be considered at two levels. Strong consistency can usually be maintained inside one service, while workflows crossing service boundaries often require temporary inconsistency.

This changes system design from asking how to create one global transaction to asking which invariants actually require immediate consistency.

Local Transactions

Each service should use normal ACID transactions to protect invariants inside its ownership boundary.

For example, Inventory can atomically validate stock and create a reservation:

BEGIN;

-- Lock the inventory record while checking availability.
SELECT available_quantity
FROM inventory
WHERE sku = 'SKU-42'
FOR UPDATE;

-- Reserve stock only inside the Inventory ownership boundary.
UPDATE inventory
SET available_quantity = available_quantity - 2,
    reserved_quantity = reserved_quantity + 2
WHERE sku = 'SKU-42'
  AND available_quantity >= 2;

INSERT INTO reservations (
    reservation_id,
    order_id,
    sku,
    quantity,
    status
)
VALUES (
    'res_4821',
    'ord_7281',
    'SKU-42',
    2,
    'reserved'
);

COMMIT;

The transaction does not attempt to update the Order or Payment databases. Those services maintain their own invariants.

Keeping strong consistency local reduces the need for distributed locking and allows each service to recover independently.

Eventual Consistency

Eventual consistency means related representations can temporarily disagree while changes propagate between services.

Suppose Ordering creates an order and Inventory receives the corresponding event 200 milliseconds later. During that interval, Ordering knows about the new order while Inventory does not yet know about the reservation request.

Time -------------------------------------------------------->

Ordering:
Order Created ----------------------------------------------->

Broker:
              OrderCreated ---------------------------------->

Inventory:
                                  Reservation Created -------->

Read Model:
                                                   Updated --->

This is not necessarily a failure. It becomes a failure only when business logic assumes every representation changes atomically.

Systems using eventual consistency should model intermediate states explicitly:

pending
   |
   +---- inventory reserved + payment authorized ----> confirmed
   |
   +---- inventory rejected --------------------------> rejected
   |
   +---- payment rejected ----------------------------> payment_failed

User-facing and internal APIs should expose these states instead of pretending asynchronous work has already completed.

Distributed Business Transactions

A business transaction can span several services even though no single database transaction covers the entire workflow. Creating an order may require inventory reservation, payment authorization, and fulfillment preparation.

Distributed two-phase commit can provide atomic coordination in some environments, but it introduces coordinator dependencies, locking, availability constraints, and operational complexity. Microservice architectures commonly prefer local transactions combined with explicit workflow coordination.

Saga Pattern

A saga represents a business transaction as a sequence of local transactions. If a later step fails, previously completed steps can be reversed using compensating operations where the business allows it.

Create Order
     |
     v
Reserve Inventory
     |
     v
Authorize Payment
     |
     v
Confirm Order
     |
     v
Create Shipment


Failure during payment:

Payment Rejected
     |
     v
Release Inventory
     |
     v
Reject Order

Compensation is not a database rollback. A released inventory reservation or refunded payment is another real business operation with its own possible failures.

Sagas can use orchestration, where one component explicitly controls the workflow, or choreography, where services react to events without a central coordinator.

Choreography works well for a few loosely coupled reactions. Explicit orchestration becomes easier to operate when workflows contain many ordered steps, timeouts, retries, and compensations.

Reliable Event Publication

Events introduce another consistency problem. Updating a database and publishing a message are two independent operations.

1. INSERT order
2. COMMIT
3. publish OrderCreated

              CRASH
                ^
                |
      between steps 2 and 3

The order exists, but consumers never receive the event.

Publishing first does not solve the problem because the event can become visible before the database transaction commits.

The transactional outbox pattern stores business state and an event record in the same local transaction:

BEGIN;

INSERT INTO orders (
    id,
    customer_id,
    status,
    total_amount
)
VALUES (
    'ord_7281',
    'cus_381',
    'pending',
    14990
);

-- The event is committed atomically with the business state.
INSERT INTO outbox_events (
    id,
    aggregate_id,
    event_type,
    payload,
    created_at
)
VALUES (
    'evt_8291',
    'ord_7281',
    'OrderCreated',
    '{"order_id":"ord_7281","customer_id":"cus_381"}',
    CURRENT_TIMESTAMP
);

COMMIT;

A separate publisher reads the outbox and delivers records to the message broker. Publication may occur more than once, so consumers should normally be idempotent.

More about reliably coordinating database changes and message publication can be found here: Transactional Outbox Pattern for Reliable Messaging.

Querying Data Across Services

Database ownership makes writes easier to reason about but complicates queries that require data from several domains. A customer order screen may need order state, payment status, shipment information, and product descriptions owned by different services.

Allowing Reporting or Ordering to join tables from every service destroys data ownership. Instead, cross-service reads usually use API composition or purpose-built read models.

API Composition

API composition retrieves information from several services and combines it at request time.

                       Order Details API
                         /     |      \
                        /      |       \
                       v       v        v
                   Orders   Payments  Shipping
                       \       |        /
                        \      |       /
                         +-----+------+
                               |
                               v
                      Combined Response

This works well when the number of dependencies is small and fresh data is important.

The main cost is runtime coupling. End-to-end latency depends on downstream services, and the aggregator must define behavior for partial failures.

import asyncio


async def get_order_details(
    order_id: str,
    order_client,
    payment_client,
    shipment_client,
) -> dict:
    # Independent reads execute concurrently instead of adding
    # their latency sequentially.
    order_task = order_client.get_order(order_id)
    payment_task = payment_client.get_payment_by_order(order_id)
    shipment_task = shipment_client.get_shipment_by_order(order_id)

    order, payment, shipment = await asyncio.gather(
        order_task,
        payment_task,
        shipment_task,
    )

    return {
        "order": order,
        "payment": payment,
        "shipment": shipment,
    }

Parallel execution reduces latency, but three dependencies still increase the probability that at least one request fails. Timeouts and partial-response behavior should therefore be explicit.

Materialized Read Models

When queries require many services or high throughput, a dedicated read model can maintain a denormalized representation optimized for the query.

Services publish domain events and a projection consumer updates the read database:

Order Service ------- OrderConfirmed ------+
                                            |
Payment Service ----- PaymentCaptured ------+----> Projection Consumer
                                            |              |
Shipping Service ---- ShipmentCreated ------+              v
                                                     Order View DB
                                                          |
                                                          v
                                                     Query API

The read model intentionally duplicates data:

CREATE TABLE customer_order_view (
    order_id VARCHAR(64) PRIMARY KEY,
    customer_id VARCHAR(64) NOT NULL,
    order_status VARCHAR(32) NOT NULL,
    payment_status VARCHAR(32),
    shipment_status VARCHAR(32),
    tracking_number VARCHAR(128),
    total_amount BIGINT NOT NULL,
    updated_at TIMESTAMP NOT NULL
);

CREATE INDEX idx_customer_order_view_customer
    ON customer_order_view (customer_id, updated_at DESC);

This database is not authoritative for payments, shipments, or orders. It is a projection that can be rebuilt from authoritative sources or event history when the architecture supports replay.

Read models reduce synchronous dependency chains and make expensive cross-domain queries efficient. The trade-off is eventual consistency and additional infrastructure for projection processing, recovery, and reconciliation.

Production Design Example

Consider an order platform with Ordering, Inventory, Payments, and Fulfillment services. Each capability owns its database and publishes events after important state transitions.

The design needs to support reliable checkout without introducing a distributed ACID transaction across all four databases.

Distributed Order Processing

The workflow begins by creating an order in the pending state. Ordering commits the order and an outbox record atomically.

                        Order Service
                             |
                   Order + Outbox Commit
                             |
                             v
                        OrderCreated
                             |
                             v
                       Message Broker
                       /            \
                      /              \
                     v                v
              Inventory Service   Payment Service
                     |                |
             Reserve Inventory   Authorize Payment
                     |                |
                     v                v
            InventoryReserved   PaymentAuthorized
                     \                /
                      \              /
                       v            v
                         Order Saga
                             |
                             v
                       Confirm Order
                             |
                       OrderConfirmed
                             |
                             v
                    Fulfillment Service

Every service changes only its own database. Inventory cannot set an order to confirmed, and Ordering cannot directly mark a payment as authorized.

The order becomes confirmed only after the required business outcomes have been observed.

A simplified state transition function might look like:

from dataclasses import dataclass


@dataclass
class OrderProgress:
    inventory_reserved: bool = False
    payment_authorized: bool = False
    inventory_failed: bool = False
    payment_failed: bool = False


def resolve_order_status(progress: OrderProgress) -> str:
    # Permanent downstream rejection determines the business outcome.
    if progress.inventory_failed:
        return "inventory_rejected"

    if progress.payment_failed:
        return "payment_failed"

    # Confirmation happens only when all required conditions hold.
    if progress.inventory_reserved and progress.payment_authorized:
        return "confirmed"

    # Temporary inconsistency is represented explicitly.
    return "pending"

If payment fails after inventory has been reserved, the workflow requests a compensating ReleaseInventory operation. If a message is delivered twice, the receiving service uses event IDs or business idempotency keys to prevent duplicate side effects.

Customer-facing reads do not need to call every operational service. A projection consumes OrderConfirmed, PaymentAuthorized, ShipmentCreated, and related events to build a query-optimized order view.

This architecture intentionally accepts temporary inconsistency. In return, each service keeps clear data ownership, local transaction guarantees, independent scaling, and isolated schema evolution.

Common Mistakes

Data problems in microservices usually appear when service boundaries exist at the application layer but are ignored at the persistence and consistency layers.

Production Checklist

Data architecture should make ownership, consistency, and recovery behavior explicit before services are independently deployed.

• Assign one authoritative owner: document which service is allowed to change every important business state.
• Restrict database permissions: prevent services from writing schemas or tables owned by another service.
• Keep invariants local: place data requiring immediate atomic consistency inside the same ownership boundary.
• Model intermediate states: represent pending and partially completed distributed workflows explicitly.
• Define compensation: document how completed steps are reversed when later saga operations fail.
• Make consumers idempotent: protect business side effects against duplicate message delivery.
• Publish state changes reliably: use transactional coordination such as an outbox for database-to-message transitions.
• Define event schemas: treat published events as versioned contracts rather than serialized internal models.
• Choose query strategies intentionally: use API composition for small fresh-data queries and projections for complex high-volume reads.
• Track consumer lag: measure how far replicated state is behind authoritative state.
• Monitor saga duration: detect workflows stuck in pending or compensating states.
• Provide reconciliation: periodically detect differences between authoritative state and important derived projections.
• Plan projection recovery: support replay, rebuilding, or checkpoint restoration when read models become inconsistent.
• Design schema evolution: allow old and new producers, consumers, and database versions to coexist during deployments.
• Test partial failures: verify behavior during duplicate delivery, delayed messages, unavailable databases, failed compensation, and consumer restarts.

Conclusion

Managing data across microservices requires replacing implicit database coordination with explicit ownership and workflow design. Each service should protect its own invariants using local transactions while cross-service operations use APIs, events, sagas, and purpose-built read models.

Data duplication and eventual consistency are often deliberate consequences of service independence. The architecture remains reliable when authoritative ownership is clear, intermediate states are modeled, events are delivered reliably, and derived data can be recovered or reconciled.

Key Takeaway

Keep strong consistency inside service boundaries and coordinate cross-service state explicitly. One authoritative owner per business state, local transactions, reliable messaging, idempotent consumers, and query-specific projections provide a practical foundation for distributed data management.

More Articles to Read

• Microservices Explained: Benefits, Challenges, and Trade-Offs
• Defining Service Boundaries with Domain-Driven Design
• REST vs gRPC vs Messaging Between Microservices
• Designing APIs for Microservice Architectures
• Microservices Best Practices for Production Systems