Three Layers of Moderation: solving one of the hardest problems in tech

LifeSim's mission is to enable every human being to explore their infinite latent potential.

When I think about LifeSim, I think about me at 15 years old, daydreaming about who I could become, about how I will explore life. That's what this platform is for — exploring, imagining, living lives you'll never get to live. It's not about explicit graphic, or other harmful content.

That has no place here.

This isn't hyperbole. Roblox has 150 million daily active users, over half of them under 13, a multi-billion dollar company, and thousands of employees. And for years, their AI content filters were easily bypassed — blocking harmless messages while missing actual slurs and predatory behavior. Users exploited code words, text manipulation, and creative workarounds faster than the system could adapt. It got so bad that multiple countries and U.S. states launched investigations or outright banned the platform.

As of March 2026, Roblox is shutting down roughly 5,000 game instances per day that violate their community standards — and they're still catching up.

That's a company with virtually unlimited resources. Content moderation at scale is genuinely one of the hardest unsolved problems in technology. Anyone who tells you otherwise hasn't tried it.

I say to set the context and show the magnitude of the problem.

The Sims has been the gold standard for life simulation for over two decades. One of the things they nailed is the content rating. The Sims is rated Teen by the ESRB, and most people don't even think about it — it just feels right. Romance exists. Conflict exists. But you never see anything that would make you uncomfortable if your kid was playing next to you.

That's the bar we set for LifeSim. Not "family friendly" in a sanitized, boring way. Teen-appropriate in a way that still lets you tell real, interesting, human stories.

Here's what that means in practice:

And here's what's never allowed:

No gray area on those. They get blocked, every time.

Here's what makes LifeSim fundamentally different from most games when it comes to safety: everything is free-form. There are no pre-written dialogue trees. No fixed storylines. No asset library that someone reviewed by hand before shipping. The world, the characters, the conversations — all of it is generated by AI in real time, shaped by whatever the player decides to do.

That's what makes LifeSim magical. It's also what makes moderation incredibly hard.

Because when you give people an open-ended, unlimited creative sandbox, some of them will stress test it. That's just human nature. People will try to say things the system doesn't expect. They'll find creative workarounds. They'll probe for edges. The battle isn't against a list of bad words — it's against the entirety of human creativity. And human creativity is relentless.

A traditional game can moderate by reviewing every asset before release. We can't do that. Every single interaction is new, never seen before, generated on the fly. So we had to build something that thinks, not just something that pattern-matches.

So we built a three-layer moderation pipeline that runs on every piece of user-generated content before it reaches the simulation.

Layer 1: Keyword and obfuscation detection.

This is the fast, blunt layer. It catches explicit language — and it also catches people trying to sneak things through with asterisks or creative spelling. It runs instantly on every input.

Layer 2: AI content scoring.

Every message gets scored by OpenAI's moderation model across categories like sexual content, violence, harassment, and self-harm. We set the thresholds conservatively — especially for sexual content, where we use a lower trigger point than the default.

Layer 3: Contextual Teen classifier.

This is the one I'm most proud of. It's a dedicated AI classifier that reads the full conversation context — not just the latest message, but the recent thread — and evaluates whether the direction of the conversation exceeds a Sims-style Teen rating. It understands nuance. It knows the difference between two characters having a romantic evening and a scene that's crossing a line. It can detect escalation patterns, grooming behavior, and power dynamics that a simple keyword filter would miss completely.

All three layers run on every input. If any layer flags something, the content is blocked before it ever reaches the game.

Some content doesn't clearly cross a line but raises a flag. Maybe someone's testing boundaries. Maybe the phrasing is ambiguous. For those cases, we built a borderline detection system that logs the content for review without blocking the player.

This matters because the hardest moderation problems aren't the obvious ones. They're the subtle ones — an adult character slowly building an inappropriate dynamic with a younger one, or language that reads differently depending on context. Our system watches for those patterns even when no single message would trigger a block on its own.

I could have shipped LifeSim without any of this and added moderation later. It would have been faster — and much better for the metrics. But LifeSim is about wholesome exploration, experiences, and the beautiful human experience.

The whole point of LifeSim is that you get to explore who you could become. That's a deeply personal thing. And for it to work, the space has to feel safe — not just technically safe, but genuinely safe. The kind of safe where a parent can hand their teenager the phone and not worry.

We're not perfect. No moderation system is. But we're not treating safety as an afterthought. It's baked into how LifeSim works, at every layer, and it gets better every week.

I want to be upfront about this: some content that gets blocked shouldn't be. A message that's completely fine gets flagged because the system errs on the side of caution. That's frustrating, and I know it.

Players already report false positives when they hit them, and we use every single one of those reports to improve the system. Each false positive teaches us where the line is drawn too tight, so we can loosen it without compromising safety.

We're getting better at this every day. The system today is meaningfully better than it was a month ago, and a month from now it'll be better again. That's the commitment.

If you have questions about any of this, if you hit a false positive, or if you ever encounter something in LifeSim that doesn't feel right — email me directly at [email protected]. I read every message.

Cris