Retention changes become difficult when the same record persists through many useful forms.
Compliance boundaries and reproducible analysis can pull in opposite directions.
Who you’d be doing this for
“I need to show what was used and why, without keeping customer-level records forever.”
Chen Nishida · Compliance Analytics Manager
Must provide quarterly evidence that regulated operational analysis used approved, retained data pathways.
What is at stake
The contract now caps customer-level records at 30 days, but copies sit in bronze tables, checkpoints, aggregates, and 17 downstream paths. You have 45 days to make deletion reach all of them and prove the audits still add up.
Why it isn’t already fixed
Every obvious fix costs something else. That’s the part you’d have to decide.
- data deletion vs. audit reproducibility
- fixed compliance deadline vs. costly verification
- automated discovery vs. verified scope
- streaming recovery vs. retention boundaries
- legal assurance vs. analytics continuity
Why Databricks
For governed data platforms, retention decisions often test whether lineage can support both controlled deletion and accountable reuse.
Written with these in mind
Not your kind of problem? 34 more at Databricks, or browse every organization.
This is the setup. The work is inside.
Running it puts you in the room: the full situation and its constraints, stakeholders who push back in their own words, and the decisions that are yours to make. What you produce becomes a Day One Plan — work you can show someone instead of describing.