Reliable status depends on delivery paths that rarely attract attention until they fail.
Fast recovery matters, but false completion signals can create a second incident.
Who you’d be doing this for
“I can’t tell whether this run is stuck or whether the status just never arrived.”
Meera Tiwari · Data Engineer
Operates declarative ETL pipelines whose downstream teams depend on accurate run state.
What is at stake
4.8% of completed runs are still displayed as active after a traffic change. You have to weigh fast restoration against creating false terminal states.
Why it isn’t already fixed
Every obvious fix costs something else. That’s the part you’d have to decide.
- rapid restoration vs state integrity
- retry availability vs duplicate execution
- narrow rollback vs root-cause evidence
Why Databricks
Lakeflow pipelines depend on reliable cross-cloud service communication to report execution state accurately.
Written with these in mind
Not your kind of problem? 34 more at Databricks, or browse every organization.
This is the setup. The work is inside.
Running it puts you in the room: the full situation and its constraints, stakeholders who push back in their own words, and the decisions that are yours to make. What you produce becomes a Day One Plan — work you can show someone instead of describing.