OTT streaming has been with us for several years now, but only recently, have companies begun to research what makes one service better than the other and how to attract more consumers. This research results in a better understanding of consumer quality preferences, such as what is causing most people dissatisfaction and frustration of low quality or poor streaming services. By understanding these quality factors, we are able to define a better approach to identify and prevent the causes of low-quality streams; improve video streaming quality, and most important our customer satisfaction.
A lot has been said and written about video quality and monitoring. As the industry is slowly approaching the next phase in the OTT era, video providers are fighting to survive in this highly competitive market. Many have figured out their best monetization approach and have deployed, or are in the middle of deploying live and VOD services, and now they need to understand how to position themselves against their competitors.
In their market research report “State of Online Video 2019”, Limelight has clearly stated that “Viewers expect high-quality online experiences.” What does it mean for us? If years ago, content was king; today, a focus on just content is not enough. Today people want not only content but high-quality content and high-quality service. The same report found that Video rebuffering “remains the most frustrating aspect of online viewing.”
What can be done to achieve the goal of highly improved quality? To answer any theoretical question, we need to have a good dataset to work on. That means we need to start collecting data, monitor our services, and analyze the result. But where do we start? Should we focus on the end consumer and understand their experience, or should we monitor our own networks and devices? Well, the real answer is “all of the above.” A lot has been said and written about video monitoring – what the most effective ways to monitor are and what the best KPIs are. When asked, customers identified the following aspects, as the most annoying and cause the most frustration during their video watching experience:
Video startup time – the actual time it takes from the moment one presses the play button for the video to start playing.
Video quality – the actual picture and sound quality, as perceived by the end-user. The video must be clear of any visual and audio artifacts and have a high visual perceptual quality.
Video availability – the basic fact of being able to receive a video stream without interruption and being able to play it on the user’s end device.
Rebuffering – being able to play the video without pausing or stopping to refill the video player buffer. I.e., prevent the player from fully drain its buffer from video frames to play.
Intuitively, one can say, “Ok, let us get the player statistics and see how the video is performing on my customer devices.” While device monitoring is critical and extremely important, it fails to provide some of the most critical answers that the operators are looking for. When things are not working, the first questions being asked by the people in charge of video delivery and quality, are: What happened, Where is the source and Why is this happening? I.e., The 3 Ws of the video monitoring. There is also a fourth question asked, but this time by the engineering teams: How do I monitor to prevent or be alerted of any failures in my services?
To get the most comprehensive and complete dataset for our baseline analysis and real-time alarming, we would need to focus on more than a single collection source. Data has to come from all parts of the video delivery chain: from content creation (filming; encoding; transcoding), through delivery and streaming, to consumption and actual consumer experience. Add to this, the fact that video network complexity these days creates a lot more handover points in the chain of delivery, silos, and too many “finger-pointing situations.” Without “Real End-to-End” monitoring, operators will be blind to what is affecting their video customer’s experience, which can lead to higher churn without them even knowing why.
To build this “Real End-to-End” monitoring solution, there is a need to focus on all the aspects of video streaming which touch the four main problems mentioned before. Let’s try and apply our “WWW” concept to the main three aspects of video monitoring.
Content – Quality of the video and audio content. How good is my picture looking, and how decent does my audio sound? Video should look perfect and be without any visual artifacts or flaws in the encoding.
What to monitor – Content quality applies to both live streaming content and VOD file-based content. The content quality check can be as deep as the video structure itself, from a single frame down to a single block. All the way from compression, scene changes, and even using AI to identify visual impairments.
Where to monitor – Since the focus on this aspect is the video/audio itself, check the quality at every point the content gets modified or changed. Will it be the encoder or transcoder, statmux, or any other device that may change the video/audio payload?
Why – Encoding is a very complex process that involves many different algorithms and predictions. Not every encoder is equal, and not every piece of content has the same complexity. The same video scene may produce different results when transcoded, even with the same encoder. Therefore, it is critical to ensure that content is produced with the minimum satisfying level of quality.
How – Deep quality analyzes (Quality of Experience) for all video and audio content using perceptual, subjective, or computer calculated algorithms, such as VMOS, PVQ, and others. For live streams, it’s best to apply non-reference base algorithms that enable higher processing speed and scalability, while for VOD, reference-based algorithms offer more detailed and precise comparison (PSNR, SSIM, VMAF, and others).
Delivery – Quality of delivery of video/audio content via IP/RF networks. How fast and errorless can my network deliver the payload I just created and checked for Quality of Content? Delivery is a very broad aspect that touches numerous types of networks, LAN, WAN, CDN, Public Internet, and others.
What to monitor – Any video stream that goes throughout our network. Be it a MPEG Transport Stream (TS) or an Adaptive protocol, such as HLS, MPEG DASH, or others.
Where to monitor – Depends on what part of the entire chain of delivery you own. It could be the LAN in your HeadEnd, or the CDN or anything in between.
Why – To ensure a properly functioning network that is capable of delivering content without any errors, delays, or loss. To ensure that your video network components (such as packager, origins, load-balancers, and others) are stable and responsive.
How – There are many different methods to monitor for Quality of Service in video streams. In the case of MPEG TS, the most common one is joining multicast using IGMP protocol or just listening to traffic coming off of span port on a switch. For ABR traffic, a proactive method of requesting URL for analysis could be used in combination with a passive method for deep network traffic analysis and detection.
Consumer Experience – Measure the video streaming experience and quality as perceived by the customers. Not only should we focus on how the user experienced their video consumption, but also whether this behavior is consistent across multiple devices, players, operating systems, and Settop boxes.
What to monitor – Two things can give us an indication of the actual experience. One is the customer themselves, or to be more precise, their device or the application. The other one is a Syntethic Client application that can proactively probe and test your streaming servers to detect failures or predict possible streaming issues.
Where to monitor – On the devices where the video is played, or at the very edge of the network, as close as possible to the streaming servers or CDN edge.
Why – First, it helps us better analyze the actual customer behavior, how they react to certain programs, types of content, and levels of streaming quality. Second, more preemptive way to predict or prevent issues from affecting our consumers. For example, if we detect a failure in one of the bitrate renditions, we could resolve it, before any user is affected.
How – Client behavior and their experience should be measured as close as possible to the clients themselves. One of the very popular methods to do so is by using tools that integrate into your player application, or the device OS with dedicated SDKd to collect streaming performance. Another way is to deploy synthetic clients in critical networks and geographic locations to probe and test your network or streaming servers directly from the edge of the CDN.
Video monitoring has evolved in the last several years, and today the focus is not only on video streams going through provider’s HeadEnd but also covers 3rd party components, outside network segments, and even public cloud infrastructure.
With the shift to more cloud-based streaming, lots of companies are practically “flying blind” without having any visibility into their video streams. It should not be like this. When designing your new video service, remember the WWW concept and try to apply it over every segment or demarcation point your video stream goes through. Using new tools available today on the market can help you get the visibility you need to run smooth, high quality, and highly profitable service.
Written by Michael Demb | Director, Solution Architecture, Strategic Sales at Telestream.