In brief
As the demand for online safety grows, social media moderators face an uphill battle. Working behind the scenes, they often encounter the most horrific content imaginable. Despite their crucial role in ensuring a safer online environment, many have reported significant mental health consequences. With AI technology on the rise, the future of this high-stress job is uncertain, but moderators express pride in their position as protectors of users.
In an age of pervasive social media, the vital role of content moderators often remains overshadowed by the platforms themselves. These individuals are the gatekeepers who stand between users and the most nightmarish content Internet can offer—videos of violence, hate speech, and child exploitation. It’s a daunting job described by moderators as haunting and filled with trauma, forcing them to confront the darkest corners of human behavior.A recent exploration by the BBC uncovered the grim reality faced by these moderators, particularly those working in East Africa. Many of them have since left the industry due to the severe psychological toll the work exacts. They share stories that paint a frightening picture of their daily routines, consumed by the need to remove disturbing footage while battling their own mental anguish. "I was moderating, in the hundreds, horrific and traumatising videos," Mojez, a former TikTok moderator, reflected on his harrowing experience. "I took it upon myself... so that general users can continue going about their activities on the platform."The legal implications of the mental health struggles faced by these workers are becoming increasingly spotlighted. Several moderators have sought avenues to unionize and seek justice for their suffering. Martha Dark, who leads a campaign group advocating for these workers, emphasizes the grim reality: “The only thing that’s between me... and watching a beheading, is somebody sitting in an office, watching that content for me.”In a poignant admission, Selena Scola, a former US moderator, likened their role to being the “keepers of souls” as they witness the final moments of lives captured in graphic detail. The work has left many former moderators plagued by insomnia, anxiety, and a disconnect from their loved ones. Despite the traumatic nature of the job, interviewees expressed pride in their duty to protect users from online dangers.Interestingly, even amid the strain, there is a strong sentiment against fully automating the moderation process. Many moderators recognize the limitations of AI in understanding the subtleties of human experiences. Former OpenAI safety head, Dave Willner, acknowledged the potential of AI-based moderation tools, but cautioned that they may not entirely replace the nuanced understanding human moderators bring to the table. Respresentatives from the major social media platforms have stated they are aware of the difficulties facing their moderators. TikTok conveyed that it strives to create a supportive working environment, while OpenAI expressed gratitude for the critical role human workers play in training AI models to identify harmful content. Meta has also ensured that companies involved in their moderation process offer extensive mental health resources to their employees.As public scrutiny of social media platforms intensifies, the future of content moderation hangs in the balance. Ongoing discussions around technological solutions versus human oversight highlight the need for a balanced approach to online safety, ensuring the welfare of those tasked with the critical responsibility of moderating our digital experiences. The documentary series “The Moderators” presents these complexities, shedding light on the unsung heroes of the digital world.
