Should retries and timeouts live in your application or your service mesh?
This debate comes up constantly.
Should resiliency live in the platform components, or should the application own it?
Like most things in distributed systems, the answer is: It depends.
Infrastructure Understands Traffic
Service meshes, API gateways, and load balancers are great at handling generic resiliency concerns.
Things like:
- Connection timeouts
- Automatic retries
- Circuit breakers
- Request-level failover
The advantage is obvious. You remove complexity from the application and apply resiliency consistently across services.
For many scenarios, this is the right answer.
Infrastructure Doesn’t Understand Intent
The challenge is that platform services only understand traffic. They don’t understand why the request is made.
They don’t know whether a request is reading customer profile data, reserving inventory, transferring money, or uploading cat videos.
To the service mesh, they’re all just requests. Some requests might even have custom timeouts and retries configured. Most of the time, that’s fine.
Sometimes, it’s not.
Context-Aware Resiliency
Some resiliency decisions require application context.
Consider a financial transaction. Should a timeout trigger a retry? Maybe.
Does the request have an idempotency key? If yes, blindly retrying might be safe. If not, things become more complicated.
Did the request reach the downstream system? Was it partially processed?
Do you need a compensating transaction before retrying?
At this point, the retry is no longer a networking decision. It’s a business decision. And business decisions belong in the application.
The Real Answer
The best architectures usually use both approaches.
Use platform resiliency whenever the decision can be made without application context. But when correctness depends on understanding the request itself, move that logic into the application.
There should be a strong preference toward offloading complexity when possible. Just don’t offload decisions that require business context.
Final Thoughts
I often see teams over-index on platform-level resiliency. And while reducing application complexity is valuable, it isn’t free.
Infrastructure understands traffic and request characteristics. Applications understand business intent.
When resiliency decisions depend on business intent, they belong in the application.