The internet has dramatically changed content consumption by continuously introducing new forms of media. Today, television is just one of the many ways people watch content. Audiences can watch what they want, when they want to, on whatever device they want to watch it.
As mobile devices and 5G networks pave the way for diversified content consumption across the globe,
the new global media trend poses new challenges to traditional broadcasting companies.
For instance, a global broadcaster based in South Korea produces a wide variety of content spanning international news, economy, culture, and entertainment programs. Since 1997, the broadcaster has been transmitting its contents over cable, satellite, and IPTV to audiences worldwide.
To keep pace with the fast-moving media trend and deliver information rapidly, the company turned to the cloud for transforming its media production system and providing leading-edge services to its audience. The customer adopted state-of-the-art, cloud-based media technology, and undertook an industry-leading digital transformation.
MegazoneCloud, with its expansive technology offerings and expertise in providing cloud solutions, was a natural partner. MegazoneCloud is an AWS Partner Network (APN) Premier Consulting Partner and Managed Service Provider (MSP) with AWS Competencies in Digital Customer Experiences, Financial Services, SAP, and others.
The project described in this post, powered by AWS Media Services, began with the aim to build a state-of-the-art prototype for media production using the AWS Cloud.
About the Live News Creation and Distribution Process
Swift delivery of global online content has always been critical to broadcasters. However, this has not been
easy to realize as the content production process usually takes hours, if not days, to complete. In the traditional methodology, the content had to go through a series of labor-intensive processes before transmission.
This process started with recording using broadcasting equipment, included a manual editing process, and was followed by additional preparations such as transcription and translation for online distribution. Therefore, automating these major time-consuming procedures was the key to creating an efficient cloud-based media production system.
In most cases, live news is recorded for 40 minutes, and then goes through manual editing and clipping to turn the continuous video stream into shorter, topic-based news segments. This was a painstaking task in which the editor needed to watch the video closely and decide where the video should be split.
In fact, this was the method most broadcasting companies, including our customer, used to edit its content to be uploaded on the internet.
Figure 1 – The production process following the recording.
Testing and Prototyping the Solution
Our customer sought to test three main service scenarios:
- Expedite news production by implementing real-time, automated, live news clipping.
- Automatically transform the contents of the segmented newsclip into text using speech to text (STT).
- Automate the process for creating captions and translation into other languages.
To attain these innovative services, cloud-based media and machine learning (ML) services needed to be incorporated into the media production process. Building this complex architecture required a close collaboration and in-depth discussions between MegazoneCloud and our customer in Korea.
To fulfill the customer’s first goal of automating news clipping, the customer and MegazoneCloud decided to employ two Amazon Web Services (AWS) sets of technologies: AWS Media Services and AWS Machine Learning Services.
They wanted to automate the role of editors in judging where a news story starts and ends. By doing this, the time-consuming manual editing process that was repeated throughout the day could be eliminated altogether to create an efficient workflow for faster news distribution.
This diagram shows the prototype’s full architecture:
Figure 2 – AWS architecture for auto-clipping.
Deploying AWS Media and Amazon Rekognition
The first step for the prototype was to upload the news contents to AWS for broadcast-grade live video processing using AWS Elemental MediaLive, a solution optimized for media processing.
Building an architecture enabling the step-by-step processes of transcoding, machine learning, transcription, and translation services was no small feat. The main challenge in building this system was the need for it to work in real-time.
As expected, enabling this process for live video was a whole different story from applying it for video-on-demand. The two companies found the answer to this problem by bringing Amazon Rekognition into the picture, which allows users to leverage deep learning even without any in-depth knowledge of machine learning.
By employing this solution, the customer could store a collection of anchorperson or frequently appearing faces under index Faces, and then use AWS Lambda and Amazon DynamoDB to identify and compile a database of where the anchor appears on the video. That would indicate the start of a new story.
Figure 3 – AWS Elemental MediaLive configuration.
An important element that needed to be added to this architecture was the logic for determining where to split the continuous news stream. Although an automated facial recognition analysis can assess the time and location of where the anchor’s face appears in the video, it is not necessarily an exact indicator of where the story begins and ends.
For a more definitive judgement of the news section, a decision logic needed to be set up.
At the same time, the customer wanted to build a serverless architecture. To conform to this requirement, MegazoneCloud decided to execute the logical programming on AWS Lambda. After intensive efforts to run the logic on Lambda, the team succeeded in applying the logic in real-time on the timeframe information stored in Amazon DynamoDB. By leveraging Lambda, the architecture was also built to reduce errors and respond to exceptions.
After the setup, Lambda automated a dynamic series of actual news clipping and editing processes using AWS Elemental MediaConvert. This image shows the auto clipping process from recognizing the anchor’s face to the final clipping.
Figure 4 – Auto-clipping process
The video contents were saved in 1080p and 30fps format using AWS Elemental MediaLive and Real-Time Messaging Protocol (RTMP) to maintain broadcast-quality resolution. AWS Elemental MediaLive and AWS Elemental MediaConvert also allowed the live news recording and video on demand (VOD) news clip service to be provided in the high quality video the customer needed.
By deploying these cloud services, a seamlessly automated live clipping process was brought to life. Now, high definition data streams could be uploaded in real-time to be processed with Amazon Rekognition, and then edited to produce news segments without delay.
Automating Transcription and Multi-Language Translation
The second goal of this project was to execute STT conversion on the processed news contents to produce captions. For this purpose, Amazon Transcribe was activated immediately following the news clipping. When the VOD news segment was saved in Amazon Simple Storage Service (Amazon S3) by AWS Elemental MediaConvert, Lambda would invoke Amazon Transcribe to perform the STT function.
Since the transcription produced by Amazon Transcribe was saved in a JSON file, it needed to be changed into caption formats such as WebVTT, SRT, or SAMI so it could be run simultaneously with the video clip. The WebVTT format was chosen because of its high compatibility with most video players.
To enable the caption to be displayed on the news clip on the video player, a Lambda function was set up to execute the commands to convert the STT data from JSON to WebVTT. After that, the Lambda function would allow the converted caption file to be played with the video clip prepared in HLS format on S3 to be played on an HTML5 Player.
For efficient delivery of the news contents, Amazon CloudFront was employed in the final distribution process. The architecture also allowed the caption to be displayed at the bottom of the screen as the news is played in the customer’s test demo player.
For the third new service, AWS Lambda was configured to run the Amazon Translate function on the text saved as WebVTT. (See Figure 2.) Since the news was reported in English targeting the global audience, the test process was able to produce high-quality STT results and translations.
As seen in the image below, the prototype tested for English to Chinese translation to be displayed in the demo player in sync with the news segment.
Figure 5 – Automated news clipping, transcription, and translation process.
Creating an Efficient, Time-Saving, Live, News Clipping Workflow
The prime benefit of leveraging AWS for the broadcasting process is that it uses managed service platforms on a serverless cloud environment.
Figure 6 – Major functions in the broadcasting process.
Automated Clipping Reduces Content Creation and Distribution Time
Broadcasting and media is a technology-intensive industry that employs a wide array of media solutions and IT infrastructure. The news clipping service introduced in this case study was enabled on a serverless architecture without the need for additional infrastructure.
It demonstrates how a serverless, fully managed service on the cloud can transform the broadcasting process, and possibly become a key to leading the broadcasting industry to a new level of growth.
This project demonstrates the potential of the cloud to:
- Introduce innovative services that adopt the latest media trends.
- Enhance workflow efficiency.
- Enable news to be transmitted swiftly through an automated distribution platform.
Conclusion
As a result of this pilot, MegazoneCloud’s customer was able to reduce the time spent on editing the news clip to enable the distribution of the news within minutes. Overall, the automated process created by leveraging ML services improved the broadcaster’s workflow efficiency by eliminating time-consuming manual operations.
The key to implementing these services in such a short time was the infrastructure technology available on the cloud. In fact, AWS Media Services and AWS Machine Learning Services stacks were the major contributors in building the leading-edge services for the customer.
Had this service been built with a systems integration approach, the project would have taken at least a few months. By leveraging the power of AWS, MegazoneCloud was able to produce the desired results in just one month.
As is evident in this example, AWS managed services allow businesses to focus on identifying their needs and building the optimal service platform to realize their goals. By implementing the cloud in the broadcasting process, our customer was able to add a new competitive edge to its services.
We hope this case will become a benchmark for news producers around the world who seek to not only survive but thrive in the digital world.
Please contact us to find out more about MegazoneCloud services.
This content is written by Yo-Han Choi, Sr. Media Solutions Specialist at MegazoneCloud, Seung-Ryong Kim, Commercial Sales at MegazoneCloud and Jin-Ho Jeong, Sr. Media Pre-Sales at MegazoneCloud.
Original post : https://aws.amazon.com/ko/blogs/apn/how-a-global-broadcaster-deployed-real-time-automated-news-clipping-with-aws-media-services/