What's the difference between an engine replica and shard in appian?
Answer
An Appian engine replica is an exact copy of an engine instance used to balance read traffic and lower latency, while a shard is a horizontal partition that divides unique process data across multiple separate pieces to process work in parallel
Engine Replicas
- Purpose: Balances read-heavy traffic across multiple instances to improve system response times and reduce user delay.
- Data Handling: Serves as an additional copy of the engine to handle read requests without altering how the underlying data is partitioned.
- Scaling: Added to alleviate processing bottlenecks when users experience delays waiting for data reads.
Engine Shards
- Purpose: Distributes heavy computational and memory loads by splitting data into parallel execution lanes.
- Data Handling: Stores unique, distinct subsets of process data (meaning different shards hold different process instances/records).
- Scaling: Expanded when execution and analytics engines run out of breathing room. Note that in Appian, adding shards is irreversible and they cannot be removed once configured
Example: To see how this works in practice, imagine a large global logistics company that handles 10 million shipping orders per day using Appian.
The Engine Replica Example (Copying the Data):
Now, suppose thousands of customer service agents are constantly refreshing their dashboards to view reports and check order statuses. This creates a massive amount of "read-only" traffic on Shard 2.
To keep Shard 2 from slowing down, the company creates 2 Replicas of Shard 2.
- Shard 2 (Primary): Handles all the heavy lifting, like advancing workflows, updating delivery times, and writing new data.
- Shard 2 (Replica A) & Replica B: They look exactly like Shard 2, but they are read-only.
When an agent refreshes their screen to check Order #5,000,000, Appian routes that request to Replica A or Replica B. This keeps the Primary Shard 2 free to process incoming shipments without any lag.
The Shard Example (Splitting the Data):
Instead of forcing a single Process Execution Engine to hold and run all 10 million active workflows, the company uses 3 Shards.
- Shard 1 holds and processes orders 1 to 3,333,333.
- Shard 2 holds and processes orders 3,333,334 to 6,666,666.
- Shard 3 holds and processes orders 6,666,667 to 10,000,000.
If a customer tracks Order #5,000,000, Appian goes specifically to Shard 2 to update it. Sharding ensures the server memory doesn't crash from trying to hold all 10 million orders at once.