Ingestion relief is pulling against the trace evidence needed for rare failures.
Lower retained volume protects platform capacity while richer tail coverage protects engineers during outages.
Who you’d be doing this for
“The failures happen once in a while, and those are exactly the traces I need when leadership asks why.”
Shahram Demir · Site Reliability Engineer
Runs a high-volume distributed payments platform and investigates intermittent failures through retained traces.
What is at stake
Replay data shows the proposed default cuts ingestion but drops rare confirmed error paths. You have to weigh capacity relief against evidence engineers cannot recover after an incident.
Why it isn’t already fixed
Every obvious fix costs something else. That’s the part you’d have to decide.
- capacity headroom vs incident evidence
- uniform control vs request-aware sampling
- automated replay labels vs verified incident truth
- cross-language consistency vs rapid relief
Why Datadog
APM SDKs and the Event Platform jointly determine which customer request paths remain available for investigation at scale.
Written with these in mind
Not your kind of problem? 40 more at Datadog, or browse every organization.
This is the setup. The work is inside.
Running it puts you in the room: the full situation and its constraints, stakeholders who push back in their own words, and the decisions that are yours to make. What you produce becomes a Day One Plan — work you can show someone instead of describing.