What Is Experience Metrics?

Experience metrics are quantitative measures designed to evaluate the quality and effectiveness of user interactions within app and video streaming services. They play a crucial role in understanding user behavior, preferences, and satisfaction levels, because they represent how users interact with a platform, giving digital businesses valuable insights on optimizing their features, content delivery, and overall performance. 

The success or failure of a digital business can hinge on the quality of user experience they deliver, making experience metrics essential for assessing and improving service performance. A positive experience, characterized by seamless interactions and high satisfaction levels, encourages users to sign up, subscribe, and continue using the service. A negative experience can lead to user frustration, prompting users to seek out other platforms and services. Customer experience metrics are thus essential for setting business KPIs, evaluating the performance of digital platforms, and for driving user engagement.

Conviva differentiates itself from other monitoring and analytics solutions thanks to its ability to offer out-of-the-box and custom-built experience metrics that represent both high-level business KPIs and business-critical steps in the customer journey. Activating this information depends on the ability to calculate stateful metrics. Conviva provides this capability with its unique Time-State technology, which correlates and contextualizes all ingested data and events into user sessions. Conviva’s stateful technology also offers core experience metrics and advanced experience metrics, which provide granular insights into specific events within customer journeys, laying the foundation for calculating higher-level business KPIs and customer journey metrics. Core experience metrics are stateful metrics that enable companies to improve the quality of experience (QoE) within any given step of a customer journey. Lower-level advanced experience metrics include both stateful and non-stateful data that allow companies to further fine-tune QoE.

Core Experience Metrics

Core experience metrics provide essential insights into what users are actually experiencing. As stateful metrics, they capture data that is continuously updated based on ongoing interactions, offering a dynamic look at user experience. With Conviva, companies can see what is going wrong, where it’s going wrong, and respond while users are still in session.

Monitoring a range of metrics to measure user experience helps companies provide seamless and satisfying experiences by adapting to evolving interactions. Conviva’s ODP comes with many core experience metrics already built in and empowers businesses to customize their desired metrics almost infinitely. Our intuitive no-code visual builder makes it easy for any business to tailor metrics to their specific needs. With Conviva, experience metrics can include (but of course are not limited to):

1. Minutes Per User

This tracks the amount of time each user spends on a platform. More minutes per user indicates higher individual engagement. Tracking minutes per user offers insight into how well content retains viewer attention. This metric allows organizations to zero in on content effectiveness and user preferences, allowing for the development of targeted content strategies and personalized user experiences. 

2. Minutes Per Device

This metric measures the average amount of time content is viewed on each device. It provides insight into device-specific engagement patterns, which can inform targeted optimizations for different platforms. This metric focuses on the performance and suitability of content across various devices. More minutes per device suggest that the content is well-suited for the devices being used. Identifying the kinds of devices that viewers use most can help companies prioritize application development for the devices that generate the most traffic. 

3. Exits Before App Launch

This metric measures the instances where users exit an app before it fully launches. It helps identify potential issues during the app loading phase, such as long load times or crashes, which can negatively impact the initial user experience. 

4. Exit Before Video Start

This measures the number of times users leave before a video begins. This metric helps identify issues causing impatience or dissatisfaction, such as potential delays in startup time or initial buffering. Reducing this metric is essential for maintaining viewer retention and engagement.

5. Video Start Failures

Video start failures (VSF) count the instances where a video fails to begin after a user clicks play. Reducing these failures is essential for retaining users and providing a reliable viewing experience. This metric also can help companies looking to enforce shared subscriber accounts, segmenting out errors related to that—and freeing operations teams from spending time on problems not related to streaming or network issues.

6. Connection Induced Rebuffering Ratio (CIRR)

CIRR, short for connection induced rebuffering ratio, is a metric that assesses the impact of connection-induced rebuffering incidents. These are unexpected pauses in video playback, caused by segments of the video downloading to the player or “buffering” more slowly than the average play rate. CIRR helps diagnose and address connectivity-related disruptions, ensuring a more stable streaming experience for users. By identifying connection-induced rebuffering, service providers can optimize network performance to improve reliability. This metric is crucial for ensuring a seamless viewing experience and minimizing user frustration caused by interruptions.

7. Average Bitrate

Average bitrate measures the data transfer rate during streaming, reflecting the quality of video playback. Higher average bitrates, meaning more bits transmitted per second, contribute to better video quality and resolution, enriching the overall user experience: the higher the bitrate, the higher the play rate. By continuously monitoring and optimizing average bitrate, streaming platforms can ensure that users enjoy high-quality video content with crisp images and smooth playback.

8. Average App Startup Time (Mobile)

Quick launch or start-up times leave a positive impression on users and increase the likelihood of repeat visits. By closely monitoring app responsiveness metrics, organizations can identify areas for improvement and optimize an app to deliver a better experience.

9. Playback Failure Rate

Playback failure rate monitors the frequency of errors or failures during video playback. A lower failure rate is critical for maintaining user trust and engagement. Playback errors quantify instances where playbacks are prematurely terminated because of errors that occur after the initial rebuffering phase. Mentoring this rate and proactively responding can ensure a smooth streaming experience for users.

Advanced Experience Metrics

Advanced experience metrics combine both stateful and non-stateful data to offer a more granular understanding of QoE. Non-stateful metrics provide snapshots of data that capture specific moments in time. These complement stateful metrics by highlighting specific issues or trends, allowing companies to calculate overarching customer journey metrics or business KPIs. Together, all of these metrics combine to provide a holistic view of user interactions and behaviors across various touchpoints.

Core and advanced experience metrics are crucial for understanding the entire customer journey, from initial engagement to long-term retention. These metrics provide insights that help identify strengths and weaknesses in the user experience, enabling companies to make data-driven decisions that increase user satisfaction and engagement. 

Integrating experience metrics with customer journey metrics allows companies to pinpoint critical moments that influence user behavior. Collectively, these metrics contribute to a broader understanding of how users navigate and engage with a platform.

Conviva’s advanced experience metrics provide the granular data needed to support business KPIs that align with a company’s goals, such as increasing user retention rates, optimizing ad placements, or improving content discovery.

Content Watch Time

This metric tracks the total duration that users spend watching content. Conviva can track events per user and events per device. Higher watch times indicate more engaging and compelling content that holds the attention of viewers, which is critical for user engagement and retention.

Ad Experience Metrics

Ad experience metrics evaluate the performance and impact of advertisements within content. These metrics are crucial for balancing ad revenue with viewer satisfaction, ensuring that ads do not detract from the overall experience. Optimized ad placements maintain a seamless viewing experience.

Sign-Up Metrics

Sign-up metrics measure the effectiveness of the onboarding process by tracking the number of new users registering on the platform. High sign-up rates can indicate successful marketing campaigns and an attractive initial user experience. Sign-up metrics can serve as early indicators of user retention. Users who complete the sign-up process quickly and smoothly are more likely to continue using the platform. Conviva’s detailed analytics provide a clear view of the onboarding process, helping to identify and eliminate points of friction.

Login Metrics

Login metrics monitor user retention and engagement by tracking the frequency and consistency of user logins. Frequent logins suggest strong user interest and satisfaction, while irregular login patterns could signal engagement issues.

Discovery/Search Metrics

Discovery and search metrics gauge how effectively users can find the content they are looking for. These metrics track search queries, the number of successful searches, and the time it takes for users to find content. High discovery efficiency contributes to a better user experience by reducing frustration and enhancing content accessibility. Conviva’s advanced search analytics ensure users can easily access desired content, improving overall satisfaction.

Understanding and utilizing core and advanced experience metrics helps companies optimize QoE by providing actionable insights that help improve user engagement, satisfaction, and retention. The analysis and application of these metrics leads to timely, data-driven decisions.

Industry-Specific Metrics

Travel

Experience metrics can be tailored to specific industries and user flows to pinpoint critical issues that directly impact revenue and user satisfaction. For example, in the travel industry, critical experience metrics might include:

• Time from Flight Selection to Purchase Click: This metric measures how long users take to proceed from selecting a flight to initiating the purchase, indicating the efficiency of the booking interface.
• Time to Process Booking: This tracks the system’s speed in processing a booking after the user clicks purchase, which can be a key factor in user satisfaction.
• Purchase Click Success Rate: This measures how often users successfully complete a purchase after clicking the purchase button, helping identify issues in the final stages of booking.
• Purchase Success Rate: This overall metric tracks the success rate of purchases from start to finish, offering insight into the effectiveness of the entire booking process.

These metrics are important because issues in the booking process don’t always manifest as full-scale outages—but they can still significantly impact user experience and revenue. For example, a sudden increase in the Time to Process Booking metric could indicate a backend issue that’s slowing down transactions. Even if this affects only a small percentage of users, it could lead to abandoned bookings and lost revenue. By alerting teams to issues when they happen, Experience-Centric Operations (ECO) enables quick resolution before issues escalate into larger problems affecting more users.

Online Retail and E-Commerce

When it comes to online retail and e-commerce, experience metrics could include:

• Add to Cart Conversion: This metric measures the percentage of site visitors who add an item to their cart out of the total visitors. This helps evaluate product and website appeal.
• Homepage to Checkout Conversion Rate: This tracks the percentage of users who navigate from the homepage to the checkout page, indicating the effectiveness of the site’s user journey.
• Time from Login to Add to Cart: This measures the time it takes for a logged-in user to add an item to their cart. This metric can offer insights into user engagement and ease of navigation.
• Time to Checkout: Time to checkout tracks the time it takes for a user to complete the checkout process from the moment they add an item to their cart. This metric can reveal potential bottlenecks or friction points in the purchasing process.

These metrics provide valuable insights into the customer journey, from browsing to purchase completion. Monitoring these KPIs makes it possible to identify bottlenecks, optimize user flows, and enhance the overall shopping experience. Tracking these critical metrics in real time allows for quick adjustments to improve conversion rates and ultimately drive revenue growth.

Gaming

Gaming is a unique digital experience where user engagement is highly dependent on seamless performance and immersive interactions. Even minor delays or interruptions can significantly impact user satisfaction and retention. Therefore, specific metrics are crucial to understanding and optimizing the user experience, ensuring smooth gameplay, and maximizing player retention.

• Game Load Time: Measures the time it takes for a game to fully load from the moment the player initiates the game until they can start playing. This metric is critical in retaining players, as long load times can lead to frustration and potential drop-off.
• Operator Game Begin to End Conversion Rate: This rate tracks the percentage of games that players start and finish, offering insights into engagement levels and a game’s ability to maintain player interest.
• Operator Game Begin to End Average Time: This metric measures the average duration from when a player starts a game to when they finish it. It helps in understanding gameplay pacing and can be used to optimize game design for better player engagement.
• Supplier Transaction Begin to End Conversion Rate: This metric tracks the percentage of transactions initiated by a player that are successfully completed. It provides insight into the efficiency and reliability of in-game purchases or other transactions, highlighting areas that may need optimization to reduce drop-offs.
• Supplier Transaction Begin to End Average Time: Measures the average time it takes for a transaction to be completed, from initiation to confirmation. This metric is key for understanding the speed of in-game transactions, which can impact player satisfaction.
• Transaction Response Time: This metric tracks the time it takes for the system to respond to a player’s transaction request, such as making a purchase or redeeming an item. Quick response times are crucial in maintaining a smooth gaming experience, ensuring that players remain engaged.

Financial Services

In the financial services sector, every interaction needs to be smooth, secure, and efficient. Some key experience metrics for financial services include:

• Average Time from Transfer Start to Completion: This metric measures the average time it takes for a user to complete a money transfer, from initiation to confirmation. It helps identify any delays or bottlenecks in the transfer process, which is crucial for user satisfaction.
• Average Time to Complete Credit Card Application Page 1: This tracks the average time users spend filling out the first page of a credit card application. It can offer insights into the complexity of the form and potential drop-off points in the application process.
• Average Time from Credit Card Application Submit to Approval: This metric measures the duration between when a user submits a credit card application and when they receive an approval decision. It offers visibility into the efficiency of the backend approval process.
• Credit Card Approval Check Average Network Request Duration: This tracks the average time it takes for the system to process a network request during the credit card approval check. It can help identify backend performance issues that might be slowing down the approval process.

By closely monitoring these or similar metrics, financial institutions can enhance their digital services, streamline processes, and provide a more user-friendly experience.

Conviva’s ECO AI Alerts system is designed to fire when critical experience metrics spike, even if only a small number of users is affected. This allows businesses in all sectors to address issues quickly, minimizing revenue risk and maintaining a high-quality user experience.

System Performance Metrics

System performance metrics also play a crucial role in understanding how the technical aspects of a platform impact QoE. Monitoring KPIs such as server response times, throughput, and error rates allows companies to identify and resolve performance bottlenecks. This ensures a smooth and reliable streaming experience, reducing buffering and enhancing overall user satisfaction. By integrating system performance metrics with experience metrics, companies can gain a holistic view of their service quality and make more informed decisions to optimize their infrastructure and ensure smooth and uninterrupted service.

When companies continuously monitor the performance of servers, networks, and applications, they can detect and address potential issues before they affect users. This proactive approach minimizes downtime and interruptions, maintaining a high standard of service quality. Combining experience and system performance metrics essentially turns them into quality of experience metrics, providing a comprehensive view and enabling companies to deliver exceptional QoE.

Significance of Experience Metrics

Experience metrics are the foundation of user-centric services, helping businesses identify areas for improvement and make data-driven decisions about how to act to enhance the overall user experience. Monitoring QoE through experience metrics can inform strategic planning and resource allocation, ensuring that solutions that will have the biggest impact on user engagement and loyalty are prioritized. Keeping existing users engaged is crucial for long-term success.

• Continuous Improvement

Continuous monitoring of experience metrics allows providers to identify and address issues promptly, optimizing the service to meet user expectations and preferences. Measuring metrics such as drop-off points helps companies understand where to focus their efforts.

• Competitive Edge

Understanding and leveraging experience metrics provide a competitive edge, allowing providers to differentiate their offerings by delivering a superior and consistently positive user experience.

• User Satisfaction

Experience metrics prioritize the user, offering a data-driven approach to enhance user satisfaction and engagement. Identifying areas for improvement is key to ensuring a seamless and enjoyable interaction with the app or video streaming service.

• User Retention

Users are not patient when they experience negative issues—there are a lot of other video streaming services out there. When users are frustrated by any experience on the digital platform, they become more likely to give up, unsubscribe, and switch to a competitor platform, which decreases revenue. A positive user experience therefore contributes to higher user retention rates, fostering loyalty and encouraging users to continue using the app or streaming service.

• Proactive Problem Resolution 

Experience metrics enable the proactive identification and resolution of issues, minimizing disruptions that could negatively impact user satisfaction. Proactive problem resolution helps in maintaining a high-quality user experience, fostering user satisfaction and loyalty. By leveraging experience metrics to anticipate and resolve issues, organizations can differentiate themselves from competitors and build a reputation for reliability and quality, ultimately driving user retention and business growth.

Observing Experience Metrics with Conviva’s Operational Data Platform

A number of crucial experience metrics impact users every day. Tracking and trending them is a key way to gauge how users perceive experiences. Conviva’s Operational Data Platform is a pioneering and exclusive tool that monitors and analyzes complex system performance, establishing a direct connection to business outcomes at internet scale and in real time. 

12 of the top 15 largest global media companies trust Conviva to oversee experience quality for their users, customers, and viewers. Conviva’s platform processes 5 trillion daily events from 7 billion SDKs, transforming this huge volume of data into immediate, actionable insights that enhance the viewer experience. Conviva’s platform also offers a comprehensive perspective on backend issues affecting application performance. This enables in-depth root cause analysis and the proactive resolution of issues that directly influence real-time user experience. 

Conviva’s solution uses an innovative Experience-Centric methodology, collecting comprehensive user data and prioritizing any issue that influences the user experience. It offers teams insight into how performance issues correspond to the user experience in real time. This enables teams to prioritize effectively. Experience-Centric Observability (ECO) gives teams a clear view of how the quality of experience they offer impacts customers’ long-term engagement.

One of the primary reasons for the emphasis on quality of experience is its profound impact on customer retention. Conviva’s data reveals that mature video publishers may lose up to 1.7 million subscribers annually because of unresolved QoE issues. Unlike outages, which occur infrequently and impact a large number of viewers at once, QoE issues happen around the clock and can affect millions of viewers per year per publisher. They have an overwhelming influence on customer retention.

Traditional observability and application performance monitoring (APM) tools have historically focused on monitoring quality of service (QoS). What makes Conviva’s ECO so groundbreaking is that QoE takes center stage instead. Traditional observability tools primarily monitor latency, traffic, errors, and saturation, but ECO enables companies to focus on what customers are experiencing. 

By measuring customer experience and connecting their engagement to backend performance across the entire user journey, Conviva’s platform offers unparalleled insight into the moments that matter most. From application launch to video playback, or any interaction in between, Conviva’s Experience-Centric Observability ensures that organizations can detect and resolve issues that impact user satisfaction and engagement.

With Conviva’s ECO service, organizations gain access to advanced capabilities such as census-level data ingestion, advanced event contextualization, no-code server-side metric creation, real-time data visualization, and AI-driven alerting that includes root cause analysis. By leveraging these capabilities, businesses can prioritize issues based on their cumulative impact on QoE.