Quality control for recorded live content presents a new automation challenge. While detecting something unusual in a recording is relatively straightforward, determining whether it is actually a problem is much harder.
Every live recording needs to be checked before it can be published. An operator is charged with identifying black frames, freezes, visual breakups and flicker, listening for dead audio, determining where the session actually begins and ends and verifying that captions accurately reflect what was said.
Automating part of that process is already possible with traditional signal analysis. Deterministic tools can measure a recording and flag potential issues at scale. The challenge is that those tools understand signals, not meaning. A sequence of near-identical frames, for example, can be detected as a freeze. But the same measurement could come from a title card that is supposed to remain still. Stadium camera flashes can trigger flicker detection, and confetti can look like impulse noise. Each potential false positive still requires an operator to seek the finding, watch the relevant content and decide what actually happened.
Southworks has developed an AI-powered quality control pipeline solution on Microsoft Azure that brings automated analysis and AI-assisted review into the post-live workflow. Built to address the gap between detecting potential issues and determining which ones require action, the solution combines deterministic media analysis with Azure Speech and Azure OpenAI to automate more of the first review.
The pipeline runs automatically a configurable number of minutes after a session’s scheduled end time to accommodate live stream extensions without requiring manual initiation. Video, audio and closed captions are ingested directly from the live stream DVR and analyzed in parallel. Preliminary findings are surfaced in the UI as they are generated, giving reviewers visibility before the full analysis is complete. Once processing is complete, an AI agent evaluates the combined findings, validates potential issues and provides actionable insights for final reviewer sign-off.
Previously, reviewers could spend more than 20 minutes manually checking a one-hour recording for potential issues. With dozens of sessions running in parallel, this created delays between the end of a live session and VOD publication. The new pipeline automatically identifies and surfaces potential issues, allowing reviewers to focus on validating findings rather than searching through the entire recording. This is designed to streamline the review process and help accelerate VOD approval and publication.
The approach is based on a clear division of responsibilities: signal processing measures what happened, AI helps interpret what those measurements mean, and the operator makes the final decision. Together, these capabilities turn a manual, finding-by-finding investigation into an automated first pass that an operator can confirm rather than perform from scratch.
Four Challenges That Measurement Alone Can’t Solve
Automating quality control is not just a matter of detecting more anomalies. The workflow presents four distinct challenges:
· Signal detectors are statistically correct but semantically blind – Tightening detection thresholds can reduce false positives, but it can also remove real faults. The core difficulty lies in distinguishing between an accurate measurement and a genuine content problem.
· Session boundaries are a matter of meaning, not signal – Live recordings are often padded before and after the actual program. A waveform alone cannot reliably distinguish a presenter settling the room from beginning the session, or determine whether applause, a conclusion or a Q&A should mark the end.
· Caption quality has no built-in reference point – Captions can be shifted, disappear while someone is speaking or inaccurately represent what was said. Evaluating them requires comparison against the audio.
· Volume can make review the constraint – A problematic feed can generate hundreds of findings. Giving operators another long, unranked list of alerts does not remove the manual work; it moves it elsewhere.
Matching Each Part of the Workflow to the Right Technology
The solution is built on a modular Azure architecture that combines Microsoft AI services and deterministic media processing, with each component handling a specific part of the review.
Microsoft Foundry provides the core multimodal AI capabilities, including Azure OpenAI models and Speech AI. Azure Speech produces a time-aligned transcript from the recording's audio, providing the reference for downstream language analysis, session boundaries and caption quality. Azure OpenAI handles language and vision analysis, from interpreting the transcript and comparing captions to providing a second opinion on visual findings.
Azure Container Apps Jobs and Azure Service Bus provide the queue-driven processing architecture. Each stage performs its task independently and publishes work to the next stage.
Azure Blob Storage and Azure Cosmos DB provide persistence. Blob Storage retains media derivatives and the AI review frames, while Cosmos DB holds the structured results presented to operators.
Azure DevOps pipelines automate build and release workflows, while modular Bicep templates provide consistent, repeatable infrastructure provisioning through Infrastructure as Code.
FFmpeg remains the deterministic measurement layer, handling black, freeze, flicker, structural breakup, broadcast-range violations, silence, abrupt level changes and frame extraction for the AI review.
“We did not set out to simply add AI to quality control. We took the opportunity to identify which parts of the process require measurement, which require interpretation and which still require human judgment,” said Johnny Halife, CTO, Southworks. “The pipeline we created keeps measurement with deterministic tools, uses Azure AI to add context where interpretation is required and keeps the final decision with the operator.”
Three AI Passes Turn Findings into a More Informed First Review
Once a recording enters the pipeline, a single media pass creates the audio and video derivatives needed by later stages and captures the channel's alarm history for the same window. Video, audio and closed captions are analyzed in parallel to reduce overall processing time. From there, processing branches into deterministic detection and three distinct AI passes, each designed to answer a different question.
The first pass focuses on the session. Azure Speech produces a time-aligned transcript, and Azure OpenAI analyzes it alongside periods of silence. Rather than looking for the first words spoken, the model applies the operator team's working rules to distinguish pre-session activity from the point where the speaker begins the session.
This analysis produces proposed entry and exit points, classifications of how the session begins and ends, identified Q&A sections, confidence information and reasoning. It can also surface problems that signal analysis cannot identify by name, such as a dropped microphone or a presenter waiting on a failed demonstration during a prolonged period of near-silence.
The second pass looks at caption quality. Timing offsets are established by matching caption lines with the transcript, while gaps are evaluated based on whether captions disappear while someone is still speaking. Azure OpenAI handles the semantic question: whether the captions accurately represent what was said. It compares time-aligned caption and transcript segments and can report potential mistranscription, garbling, truncation, missing content and formatting problems.
The third pass gives visual findings a second opinion. Instead of treating a detector's output as a final result, the pipeline treats it as a hypothesis. It extracts the frames an operator would have inspected and uses Azure OpenAI to evaluate the finding before it reaches the operator.
For a suspected freeze, the model looks for details such as a ticking clock or scrolling ticker that could show the picture is not actually frozen. For flicker, it considers the difference between scattered camera flashes and large-area repeating flicker. For a potential breakup, surrounding frames help distinguish corruption from a normal transition between clean shots.
Frame selection changes with the question as well. A duration issue requires frames spread across the affected window, while a single-frame glitch requires consecutive frames. Segment defects receive context from immediately before and after the event.
This is also where the review makes an important distinction between whether the detector's specific claim was correct and whether there is a genuine defect that requires attention. A feed cutting to black might be incorrectly reported as flicker, for example. The AI review can refute the original label while still identifying a real problem. Likewise, a held end slide may genuinely be static without representing an unintended freeze. Keeping those questions separate prevents an incorrect label from automatically becoming a dismissed finding.
Three Opinions, One Decision
When the analysis reaches the operator, the workflow keeps three distinct layers intact:
· Suggested action: what the pipeline recommends based on its measurements
· AI verdict: what the frames and AI review indicate
· Review status: what the operator ultimately decides
Only the operator's decision is binding for downstream actions such as publishing, trimming, re-editing or rejecting the recording.
The AI narrows what needs to be reviewed, explains its assessment and shows the evidence behind it. It does not make the final decision.
From Performing the First Review to Confirming It
By combining deterministic signal analysis with Azure Speech and Azure OpenAI language and vision capabilities, Southworks turned a manual, per-finding review into an automated first pass that an operator can confirm rather than perform from scratch.
The new Southworks solution brings that approach into a deployable Azure-based pipeline, automating the workflow from post-session ingestion through analysis and AI-assisted review, giving operators richer context and clearer evidence while keeping them responsible for the final decision.
Southworks is a Microsoft Solutions Partner helping media organizations bring AI into production workflows. To discuss how AI-powered quality control could streamline your content review process, get in touch.
