Audio Moderation

Audio communication is everywhere: in the videos we watch, in chat rooms, and even in our work communication tools. With it comes the need to protect community standards, users, and service providers who deliver audio content.

This is where audio moderation comes in.

What Is Audio Moderation? 

Audio moderation is the process of reviewing and filtering audio content and voice calls to ensure they adhere to safety, quality, and community guidelines. It's one of the many forms of content moderation companies employ to protect their and their users' interests.

Other types of moderation include text, image, and live stream video moderation. Audio moderation is one of the more complex types due to the inherent difficulties of analyzing audio.

Audio moderation is a three-step process. It involves: 

1. Detecting audio input and analyzing it for verbal content (spoken text, sung lyrics, etc). 
2. Transcribing the speech audio into text.
3. Analyzing and classifying the text using moderation techniques.

How Does Audio Moderation Work? 

Audio moderation is a complex and highly technical process. To understand how it works, let's explore the basic definition provided above.

Audio Analysis

Audio input detection and analysis is the first and most difficult step. It aims to identify audio and then isolate the spoken words within it. 

A video or live broadcast may contain multiple simultaneous sound sources, so pinpointing the correct audio stream can be challenging.  

To start, most audio moderation algorithms begin with basic audio filtering techniques, such as: 

• Detecting and mitigating distortion and excessive volume.
• Filtering out frequency bands unnecessary for speech.
• Removing extraneous sounds like background noise and music.

Once the preliminary audio analysis steps above are complete, an additional subtask often involves further segmenting the spoken audio into multiple audio streams to identify different speakers. 

In cases where users connect from separate devices, like a multi-computer Zoom meeting, the process can be simplified by detecting audio from different devices.  However, more complex audio separation algorithms are required to detect multiple speakers from a single audio input.

Transcription

The second phase of the audio moderation process, transcription (speech-to-text conversion), brings us one step closer to actual moderation.  Transcription converts detected speech audio into written text, which can then be analyzed and filtered.

Today, AI is improving the speech detection process considerably. However, factors like background noise, music, and audio distortion can still make it difficult to separate speech audio streams. Even without unwanted sounds complicating the process, speech analysis itself is a bit tricky.  

There are often myriad different languages and accents to account for. Dialects, regional slang, and cultural communication differences make speech analysis difficult for an automated system trained only on "standard English."

Text Moderation

After the audio is analyzed and converted to text, the moderation phase begins.  

Here, the resulting text is processed and filtered using text moderation techniques. Its purpose is to analyze and filter user-generated text, discovering and weeding out text that violates community standards or other company guidelines.  

Text moderation typically filters for:

• Profanity
• Hate speech
• Threats
• Spam
• Custom lists

Text moderation is broad enough in reach that it can be tailored for use in almost any field where text is used. More complex media formats like audio and video can often be boiled down to their speech/text component and moderated using text alone.

Audio Moderation Use Cases 

Text moderation is a content moderation approach with many use cases, including community forums, gaming, other online chat spaces, consumer protection, and more.

Social Media, Gaming, and Dating Platforms

Many social media platforms, gaming communities, and dating apps include voice communication features.  With a broad spectrum of users spanning various age ranges and backgrounds, users are often quick to resort to disrespectful and harmful speech.  The partial anonymity of communicating online contributes to people's ways of interacting in these forums. 

• Voice/Audio Messaging: Like the older unidirectional formats of text and email, voice messages are sent one at a time as a single message (in this case, using audio instead of text). Like any message, they can include unacceptable content. These messages can be moderated responsively (in response to a user-submitted report) or proactively by screening for unacceptable audio content in real time before a message is sent.

• Livestream Events: Live streaming has become a key part of social media campaigns, with almost all major social media platforms offering real-time streaming capabilities to their users.  It's a great creative outlet for marketing and user interaction, proving to excel in increasing user engagement compared to older methods.  However, as a source of real-time, unfiltered audio, there is the potential for unacceptable messages to be shared.  This is why AI moderation is used during livestream events to filter audio in real time.

• Gaming: In-game chat rooms have some of the highest levels of verbal harassment and disrespectful speech.  The combination of anonymity, competitiveness, and disproportionate male participation (which often leads to toxic environments) all underscore the need for in-game chat moderation.  So, in addition to text moderation, audio moderation is frequently used on platforms that include voice chat to prevent harassment, hate speech, and other forms of toxic behavior.

• AI Voice Agent: AI-powered voice agents enhance real-time audio moderation by actively intervening in voice-based interactions. These agents can detect harmful or inappropriate speech and take immediate action, such as issuing warnings, muting offenders, or escalating violations for human review. Unlike traditional reactive moderation, AI voice agents provide proactive enforcement, helping to maintain a safe and respectful environment across various platforms, including customer support systems, gaming voice chats, and live-streamed conversations.

Virtual Classrooms

Remote learning environments are one of the areas where content moderation is most important.  When you consider the difficulties associated with managing in-person classroom behavior, the potential issues are only amplified with the added anonymity of the online setting.

With this in mind, use audio moderation tools and other content moderation approaches to monitor and protect students and create safe learning experiences. Audio moderation can detect profanity, hate speech, bullying, and other unacceptable speech; however, it can also be employed for more straightforward sound filtration, used to remove unwanted background noise and other types of interruptions.  

SDKs can help developers interested in creating online learning apps build a virtual classroom app with video and voice chat features.

Telehealth 

Telehealth is one field in which audio moderation is used for a purpose other than preventing harassment and disrespectful speech.  

In the healthcare industry, content moderation is used primarily to protect the consumer's personal data.  Here, custom moderation lists can be used to detect and ensure users' protected health information (PHI). Additionally, audio moderation can be used to auto-detect other personal information such as email addresses, phone numbers, social security numbers, mailing addresses, etc. 

Special APIs, such as the HIPAA-compliant video API, can be used to add compliant video calling services to your app. You can also add audio-only telehealth services to your app for users who don't have a camera or have other personal reasons for preferring audio-only communication.

Benefits of Audio Moderation 

Audio moderation has wide-ranging benefits. It positively impacts user trust and safety and offers additional benefits for service providers.

Enhanced User Safety and Well-being

The primary goal of audio moderation is to protect the user.  In our daily use of apps and online forums, we subject ourselves to the opinions and judgments of countless users who we'll never meet in the real world; this anonymity can sometimes amplify the voices of bad actors. 

Audio moderation, like all content moderation, is designed to safeguard users from being on the receiving end of potential hate speech, profanity, and other forms of harassment.  It often uses AI to assist in filtering out unacceptable speech before it reaches its intended target, thereby mitigating harm and reducing the potential for future trauma.

Improved Brand Reputation and Trust

It should come as no surprise that the same service providers who go out of their way to protect their users via content moderation also want to protect their reputation.  Audio moderation is a key tool for filtering out content that might reflect poorly on a brand. 

By ensuring adherence to community standards, plus additional company-set guidelines, brands maintain a positive image and ensure that their best customers have a safe place to interact and do business.  

By implementing audio moderation on their platforms, companies avoid potential legal and regulatory risks, which could be very costly if not prevented.

Increased User Engagement and Satisfaction

When users trust the brand they're interacting with to provide a consistent, positive experience, they're more likely to return to their platform.  Feeling safe from bullying, harassment, and inappropriate language and having a way to report issues when they arise are all critical to maintaining customer engagement and user satisfaction.   

Challenges of Moderating Audio 

Audio can be quite complex, and the moderation process isn't without its challenges.

Filtering Complex Audio Sources 

With multiple sound sources occurring in one moment.  With real-time audio, the challenge of managing this complexity increases.  Live sound sources can include:

• Background noises
• Surges of volume
• Simultaneous speakers
• Multiple languages

Audio moderation tools must include functional filtering algorithms that remove unwanted sounds and separate the sounds they want to keep and analyze.

Deciphering Lyrics in Songs

Song lyrics are a special case for AI-based speech detection. Although humans can usually understand song lyrics when listening to music, if you step back, you'll notice that many words are pronounced completely differently than in everyday speech.  

The rhythm and emphasis of syllables may change when sung. Consonants are diminished, whereas vowels are amplified and extended — not to mention all the breathiness.  All of this makes it quite challenging for computers to understand sung lyrics.

Nuance of Meaning

The final step of audio moderation is classifying text and assessing its appropriateness or inappropriateness.  

Unfortunately, subtlety is not AI's strong suit, and certain text can often be mislabeled. AI has trouble understanding nuance, including sarcasm and user approval or disapproval.  

For example, a user discussing her disapproval of an unsavory topic may be seen as supporting that view simply by speaking at length about it. 

To improve their reliability, AI moderation tools require careful sentiment analysis, LLM prompt engineering, and AI model training and refinement.

Frequently Asked Questions