What is User Experience?

User experience is a complex subject. Many digital businesses, big data platforms, and observability and analytics solutions have tried to find a way to represent it effectively, but the effort, manpower, and budget required to get even a small glimpse of what users are encountering on the front-end of digital platforms has been unsustainable.

Conviva, however, has put decades of research into translating vast quantities of data into an intelligible representation of true, real-time user experience. Our learning and practice working with leading brands in the industry has allowed us to identify the two aspects of experience which matter most for your business and therefore must be surfaced by your observability and analytics solution if you are to achieve true Experience-Centric Observability:

    • Business-critical user journeys, or the user’s ability to successfully complete critical business journeys within your platform;
    • Application interactiveness, or the user’s visual experience in terms of performance and smoothness of interactions.

These challenges are all rooted in the same underlying problem. Existing Observability tools and the teams that use them are disconnected from what really matters to the business: user experience and engagement.

 

Let’s use some real-world examples to illustrate what we mean.

A. Business-critical user journeys

If subscribers can’t log in to an app, it’s impossible for them to utilize many services or consume paid content. No matter how great video quality or app performance is, a failure in the authentication service will drive away users, leading to steep declines in engagement, spikes in churn and, eventually, critical impact on company revenue. Flaws in subscription and payment processes lead to similar results: if users can’t pay, your business loses money, nor can you guarantee that your user won’t just find an alternative app and never return.

B. Application interactiveness

Let’s imagine a video app: loading the top-content gallery takes three seconds; scrolling down to reach the recommended content gallery takes another three; navigating to the episode details takes another five. Search returns limited 3 results and takes eight seconds to respond and render. From ‘play button’ press to actual play – another five seconds. If we consider a user might run several searches or explore multiple galleries, your average user will spend three to four minutes from session start to first content play, where 80% of that time is spent waiting on the app, but just 20% is the user’s interaction time. This type of experience won’t stop your user from reaching desired content, but the lag is annoying, meaning that competitive apps with similar content and comparable price can pull users away by offering a faster, smoother experience.

Being able to track either of these aspects in real time and at scale is the holy grail of observability. Legacy solutions spend too much money on too many data sources, without delivering enough value, meaning that most companies fall back on basic performance monitoring, without any real awareness of how it correlates to and affects user experience.

Conviva, on the other hand, has recently launched its transformative Time- State Technology to bridge the gap between performance and experience, immediately surfacing root-case analysis of issues affecting both business-critical user experience or application interactiveness. Instead of taking hours, days, even weeks to pinpoint problems, your reaction speed can be instantaneous. Our platform translates all data into a common language for every team in your business, helping them focus on what matters: user experience and critical paths for your business within the user experience. This empowers you to actively safeguard engagement, retention, growth, revenue, and other business goals as you solve problems for your users in real time.

Keep reading to discover the real questions and concerns we have fielded from CTOs and VPs of engineering while discussing how our groundbreaking new technology helps them understand what’s broken for their users and fix it as quickly as possible.

 

PROBLEM 1: Measuring User Experience

QUESTION

So you’re telling me you’re capable of providing a solution for user-centric experience measurement in our app or streaming platform. But how do you even reduce user experience to something that can be measured effectively?

ANSWER

As we’ve established, user experience can be broken down into two aspects that matter for your business: business-critical user journeys and application interactiveness. Conviva is able to model both of these thanks to two unique aspects of our Operational Data Platform.

First, we have built a very powerful client sensor.

Our sensor team is dedicated to delivering the client SDK for each app platform (iOS, Android, Roku, etc.) and solving the observability task within the device, with a full development cycle from design to end-to-end QA. The current state of the sensor enables our customer to get value immediately after a very simple integration: add the sensor SDK into the app, do the initial call specifying your account customer key provided by Conviva, and start collecting a range of relevant events, covering the entire app lifecycle, including but not limited to:

    • app launch
    • screen / page views
    • backgrounding / foregrounding
    • button and link clicks
    • app crashes
    • device / geo comprehensive metadata
    • connection type
    • importantly – network requests, with response code and extra tags from headers or response body available as metadata dimensions for instant filtering and response duration as a metric

Second, we have created an entirely new approach to experience metrics.

It’s important to note that client measurements are NOT yet directly experience metrics. A count of button clicks isn’t a replacement for experience itself. When you see 100 login button clicks, you don’t actually know what the experience was. Rather you need a metric—login success rate, login page view to completion time, login to first content play, etc.—to confirm whether login is functional or has become a hurdle for your users.

Notably, these metrics are stateful: they are based on the concept of a session (continuous flow of data from a single client) where what matters is the previous state/condition that triggered the transformation. Conviva Time-State computation technology addresses this challenge. Traditional SQL-based, table-and-count solutions weren’t designed for that.

Therefore, we have built two major unique components for our solution:

    • Semantic Mapper: this tool allows you to translate raw events into business events—that is, common-sense events that have an impact on your business operations. This happens in Conviva’s back-end, where you can manipulate variables through an extremely intuitive UI, so that you can surface desirable information immediately, instead of having to search it out manually from the raw data.

    • Metric Builder: this tool allows you to create Time-State metrics using the business events set up in the Semantic Mapper. You can connect important events into a key metric for your business, and then measure conversions, conversion rates, time intervals, and/or successful interactions for the workflow between those events. With custom metrics, you can now collect data on which workflows are critical to the success of your application and measure in the moment how well your customers are achieving those workflows.

This is insight beyond just the technical availability of your app. Instead, you are being empowered to immediately surface important deviations in business-critical experience metrics and to finetune responsiveness and performance issues for each stage of the user journey between app launch and content play. With this kind of detail, Conviva enables you to proactively pursue your company’s experience and usability goals in real time.

 

PROBLEM 2: Limiting Sensor Load & Safeguarding Performance

QUESTION

All that sounds great, but from our past experience, attempting to collect all this data will cost us a fortune. We’re also concerned that if you collect every event as you described, especially the network requests, it will slow down our devices and create unnecessary additional load.

ANSWER

This is a fair concern, which we worked on addressing at a very early phase in the development of our product, thanks to what we had learned directly from our customers. We have two aspects of our sensor which handle these concerns.

First, our sensor is designed so that it is actually not doing any significant processing. Instead, it is a lightweight client which wraps data into a simple JSON format and sends it out. It doesn’t keep any large cache. The format is minimalistic. During objective side-by-side tests of real-world apps with and without the Conviva SDK, executed on devices at the lower end of performance ratings, no significant difference was discovered between the apps from a CPU and memory perspective. Specifically, CPU difference during the tests was demonstrated to be statistically insignificant.

Second, we have implemented intelligent remote configuration of the sensor, via the Conviva dashboard. This means that you can block particular event types or patterns of network requests from being sent by the sensor. Additionally, the data is sent intelligently, preserving network usage and device performance.

This ability to configure the sensor allows you to be very smart about what you capture, only pulling in what’s relevant and necessary for your business. If you notice concerning performance issues, we always suggest reviewing noisy, high-frequency event reporting and configuring the sensor to mute anything irrelevant which might bog down the sensor. Since our platform is priced on data volume, you are fully empowered to control cost inflation for your observability and analytics solution.

 

PROBLEM 3: High Development Resource Requirements

QUESTION

The main problem with other solutions we tried so far is that the dev team is required to implement all kinds of measurements, thinking up front about use cases. If they miss any, it takes time to add those, QA, then release to production. We don’t want the dev team to spend time & effort on that; their job is to build the product, not analytics. How do you address this challenge?

ANSWER

Conviva has been working to address this exact challenge. Our solution’s unique capability is that it allows you to distill and dynamically create virtual, mapped events from observed raw data points through our Data Modelling dashboard. This enhancement happens in a very dev-friendly way: the arguments to pass are just the name and any JSON object providing context for that event. Your developers don’t need to consider business logic upfront; no forward-thinking or alignment across multiple different platforms is required.

Once the Conviva sensor has automatically fed key data to our intuitive and user-friendly Conviva dashboard, you can sanitize the data, create metrics, and reconstruct the business logic of your app by mapping event attributes to important user milestones. The mapping process also cleans and translates the data into a common language, enabling multiple teams across your business to access key events instantaneously without needing to know query languages, by means of our interactive multi-dimensional drill down.

These features allow Conviva to immediately reduce time to use/value for your dev team.

 

PROBLEM 4: Alerting for Metric Deviations

QUESTION

The ability to create metrics sounds great; however, we won’t be able to sit and watch the dashboard waiting for something to happen. Do you have alerts?

ANSWER

Yes, our platform provides AI Alerts, which is based on the foundation of well-established, practically proven alerting technology, originally built by Conviva for video streaming metrics.

Conviva AI Alerts takes a fundamentally different approach to monitoring, focusing on user experience at the granular level. Unlike other services, Conviva measures the experience impact on every device and every viewer, ensuring that alerts are only triggered when users are genuinely affected.

Our approach also ensures you’re only alerted to the issues relevant for your business, so you can take action before revenue is impacted. Conviva AI Alerts surface issues like a sudden increase in average network request duration, which can slow down login, sign up, or search, and directly impact customer experience and revenue.

Whether alerting on average network request duration, average screen load time, average page load time, or a sudden drop in active devices, the incident will escalate from Info, through Warning, to Critical level, depending on the scale and severity of the issue. Every AI Alert also arrives with full context and in-depth reasoning for why it was fired, including a timeline and a spotlight on impacted metrics. This gradation and contextualization of alerts enables teams to respond appropriately to any issue, whether it’s a complete outage that needs immediate attention or a small issue slowing down backend processes.

Conviva’s approach eliminates noise and decreases alert fatigue, helping your teams focus on protecting the moments that matter most to users in your app.

 

PROBLEM 5: Debugging Unwanted Events

QUESTION

I want to debug after a crash or some other unwanted event and understand why it happened. How can I do that? Can I build a funnel and see what led to the event?

ANSWER

Conviva makes it very simple to drill down across multiple metrics and dimensions to understand the context and root cause of any unwanted event. By filtering key information such as App Crash or Event Count by event name, or a combination of event name plus event tag values, you can determine both when the unwanted event happened and whether it was spiky in nature or happening consistently over time.

Another benefit of debugging with Conviva is that our dashboard allows you to compare and contrast the metric of interest against other metrics—which immediately surfaces any correlation with other metrics in scope. This allows you to generate a holistic picture of device performance and user impact before, during, and after the event. In particular, it is always very important to run diagnostics looking at Active Devices or Active Users as a secondary metric. This combination will immediately contextualize any unwanted event with respect to user impact.

 

Conviva’s platform also sessionizes all data—that is, it keeps the stateful track of event sequence. This allows you to isolate the diagnostic space down to only impacted sessions, so you can pinpoint any correlation to other events— without the need to build a funnel manually.

Finally, when identified users are being impacted by an event under investigation, you can even drill down to the most granular level of an individual user’s session details (User Timeline). There, you will see all events for that user, allowing you to trace the steps which led to the incident, check on follow-up actions (e.g., app restart, another attempt, etc.), and help them solve their issue during a live customer support session.

CONCLUSION

At Conviva, we believe that understanding user experience shouldn’t be an insurmountable obstacle for our clients. That’s why we’ve put decades of research, experience, and development into perfecting an Operational Data Platform that allows you to protect important moments for your users, resolve their issues in real time, and reap the benefits for your business along the way. With our unique sensor and proprietary Time-State technology, we empower you to create a truly flawless experience for your customers. Our intuitive, streamlined platform supports you in prioritizing resources across your teams, protecting budget, cutting costs, simplifying your tech stack, and encouraging audience and revenue growth. It may sound too good to be true when we say our technology is transformative, but for B2C companies, the direct access to user experience which we provide is the missing link for unlocking your business’s full potential.

 

Ready to discover the transformative impact of Experience-Centric Observability?

Talk to Conviva