The reported-content model must catch an emerging harm pattern without turning uncertain reports into enforcement.
Trust & Safety needs faster removal of harmful content, while Policy protects members from irreversible incorrect actions.
Who you’d be doing this for
“I woke up to people asking why the report did nothing, and the message was still there.”
Pekka Jensen · Volunteer Moderator
Runs moderation for a large gaming server and relies on automated review when reports arrive faster than the moderator team can inspect them.
What is at stake
Reported harmful messages are being marked low risk and remain visible in large servers. You must weigh faster removal against irreversible incorrect enforcement.
Why it isn’t already fixed
Every obvious fix costs something else. That’s the part you’d have to decide.
- harm removal speed vs incorrect enforcement
- model recall vs policy context
- automated scale vs verified evidence
- moderator relief vs appeals capacity
Why Discord
Automated review decides most reported-content cases, so a threshold change reaches members and moderators at platform scale.
Written with these in mind
Not your kind of problem? 16 more at Discord, or browse every organization.
This is the setup. The work is inside.
Running it puts you in the room: the full situation and its constraints, stakeholders who push back in their own words, and the decisions that are yours to make. What you produce becomes a Day One Plan — work you can show someone instead of describing.