The challenge
Available-to-sell inventory is the number a marketplace cannot get wrong: too high and it accepts orders it cannot fulfil, too low and sellable stock is invisible. The existing design pushed availability from order management into commerce through a synchronous service wrapped in a circuit breaker. It worked, but its ceiling was roughly 20,000 updates in 24 hours — orders of magnitude below what the catalogue and seller base had grown into. Retries and circuit-breaking masked the backlog rather than clearing it, so the storefront could be hours behind reality without any component reporting a failure.
Approach
- Reframed the problem as architectural rather than tuning: a synchronous call between two systems that must scale independently has a ceiling no amount of tuning removes.
- Made the event stream the system of record for change, so the storefront consumes availability rather than being pushed it.
- Decoupled producer from consumer so each could scale, fail and be released on its own schedule.
- Reproduced behaviour under real cluster concurrency before committing to the design — the failure modes only appear with multiple consumers.
- Validated against production-scale data volumes rather than a representative sample.
Innovation
- Treated event-driven design as a decoupling decision first and a performance decision second — the throughput gain followed from removing the coupling, not from optimising the call.
- Made backlog observable. The previous design's worst property was that it degraded silently; the replacement surfaces lag as a first-class signal.
- Kept the existing commerce platform in place — the re-architecture sat around it rather than requiring its replacement.
Outcome
- Throughput moved from roughly 20,000 availability updates in 24 hours to around 4 million in an hour.
- Propagation became near real-time, so the storefront reflects availability rather than trailing it.
- Order management and commerce decoupled — each can now be scaled and released independently.
- Backlog made observable, so degradation is detected rather than inferred from symptoms.
Recommendations
- A circuit breaker in front of a saturated integration is a symptom, not a solution. If it is tripping routinely, the design has reached its ceiling.
- Silent degradation is worse than failure. If a backlog cannot be seen, it will be discovered by the business, not by engineering.
- Decouple first, optimise second. Most of the throughput gain here came from removing a synchronous dependency, not from making calls faster.
- You rarely need to replace the commerce platform. Re-architecting the integration around it is cheaper, faster and far less risky.
In their words
Wish I could have you for all my projects — we can turn global escalations into reference global customers.
Global executive, platform vendor · 2016-09
Technology
- SAP Commerce (hybris)
- SAP Commerce OMS
- Apache Kafka
- Spring Boot
- Hystrix (replaced)
- Clustered application tier