The customer has been anonymised at their request. Quotes below are from the platform's Head of Trust & Safety.
The Challenge
The platform is a global gaming and virtual world environment with over 50,000 users, many of them minors. Like most platforms of its size, it had grown rapidly, and its safety tooling had not kept pace.
Their existing moderation system relied on keyword-based detection: a manually maintained blocklist of words and phrases. This approach had three critical limitations:
- Invisible behavioural threats. Grooming does not happen in a single message. It unfolds across dozens of interactions, building trust, isolating the child, and escalating gradually. Keyword filters cannot detect this because no single message contains a flagged word.
- No image or voice detection. Users could share images and communicate via voice chat with no safety analysis at all. The keyword filter only covered text.
- No age verification. The platform had no mechanism to verify user ages, making it impossible to apply age-appropriate moderation thresholds.
- Moderator overload. Because the keyword filter generated a high volume of false positives, moderators spent most of their time clearing false alarms rather than investigating genuine threats.
"We knew we had gaps, but the scale of what we were missing only became clear after we switched to Tuteliq."
The Solution
The platform evaluated several options, including building in-house and using US-based moderation APIs. They chose Tuteliq for three reasons: developer-friendly documentation, the speed of integration, and EU data sovereignty.
Integration
Using Tuteliq's open-source SDK, the platform's engineering team completed full integration in under 48 hours. The API was added to their existing message pipeline, image upload flow, and moderation dashboard. No infrastructure changes were required, since Tuteliq runs as a real-time API call within their existing architecture.
What they deployed
- Grooming detection. Detects multi-message grooming patterns, trust-building, isolation tactics, and escalation, which were previously not detected.
- Bullying detection. Identifies harassment, exclusion, threats, and coordinated pile-on behaviour, replacing partial keyword coverage.
- Unsafe content. Flags self-harm, violence, drug references, and explicit material, age-calibrated, replacing a basic blocklist.
- Image analysis. Detects unsafe imagery, nudity, violence, and synthetic content in user-uploaded images, previously absent.
- Emotion analysis. Monitors emotional distress signals for early intervention.
- Moderation dashboard. Centralised view of all flagged content with severity, evidence, and recommended actions, replacing manual spreadsheet tracking.
The Results
Within three months of deployment, the platform saw measurable improvements across every safety metric:
- 300 threats detected and blocked that the previous keyword system would have missed entirely. These included grooming sequences, coordinated bullying, and unsafe image uploads.
- 500K+ API calls per month processed in real time, covering text messages, image uploads, and moderation actions. Average response time 1.1 to 1.5 seconds.
- Dramatically reduced false positives. Tuteliq's behavioural analysis understands context, reducing moderator noise and allowing the team to focus on genuine threats.
- New moderator onboarding cut to hours. The dashboard provides severity levels, evidence trails, and recommended actions for every flagged incident.
- Coverage expanded from text-only to multimodal. For the first time, the platform has safety analysis on images, not just text messages.
"The biggest surprise was how much we were missing. We thought our keyword filter was doing a reasonable job. Tuteliq showed us it was catching maybe 10% of what was actually happening."
Why They Chose Tuteliq
- Developer-friendly. Open-source SDKs, comprehensive documentation, and a 14-day free trial for testing. The engineering team had a working prototype the same afternoon.
- Fast to production. From first API call to full production deployment in 48 hours. No infrastructure changes, no ML expertise required, no model training.
- EU data sovereignty. All processing runs on Google Cloud Vertex AI in EU regions. Zero content retention by architecture. Critical for GDPR compliance and parent trust.
- Behavioural detection. The ability to detect grooming and coercive control across message sequences, not just individual flagged words, was the decisive technical differentiator.
- One API for everything. Text moderation, image analysis, and age verification through a single API with a unified dashboard.
Looking Ahead
The platform is now evaluating additional Tuteliq capabilities including voice analysis for in-game voice chat, synthetic content detection for user-uploaded avatars, and identity verification for age-gating premium features. They are also working toward compliance certification under the EU Digital Services Act, with Tuteliq's KOSA 9/9 category coverage as the foundation.
Stay updated
Get notified when we publish new case studies.
Newsletter
Stay informed on child safety regulation & technology.