Insights overview
A guide to the Insights pages, which explain what Otis records about your product, what it computes, what it delivers, and how it comes to know your product.
The Insights section explains how Otis works. Each page covers one topic, such as tasks, segments or the daily briefing. Read these pages when you want to know what a number or a finding in Otis refers to, and how far you can rely on it.
The pages are in four groups. The first three follow the path from your app's telemetry to a published finding. The fourth covers what Otis knows about your product and the checks behind what it reports.
The SDK section covers how your app sends the telemetry that these pages describe.
What Otis records
Your app sends telemetry as spans. A span is one recorded operation, such as an AI call, a tool call, a traced function or a custom event. Otis groups spans by user and by session, and divides each session into tasks.
For each of these, Otis keeps what your app reported and adds what it works out for itself: what the user was trying to do, how the attempt turned out, and whether the user's messages showed frustration or delight. It also keeps a dated record of changes to your product, and of what your AI calls cost.
Users, groups and accounts
The people who use your product, and the groups and paying accounts they belong to.
Spans
The unit of telemetry that your app sends, and what a span holds.
Surfaces
The named parts of your product that users work in.
Sessions
The activity that shares one session ID, and what Otis writes about it.
Tasks
One piece of work that a user set out to do, and how Otis decides whether it worked.
Documents
The lasting thing a user works in, such as a project or a file.
Signals
The judgments Otis makes about individual messages, such as frustration or delight.
Feedback
The explicit reactions your app sends about an AI response or a tool call.
Errors
The operations that failed, and the cause Otis gives each one.
Intent
The description Otis writes of what a user was trying to do.
Changes
The dated record of what changed in your product, such as deploys and feature flag variants.
Cost and revenue
What your AI calls cost, and what each account pays.
What Otis computes
From those records, Otis computes values over many users, sessions and tasks at once. It takes named measurements of each user. It forms the groups that it compares: segments and cohorts of users, and clusters of tasks with similar goals. It counts progress through the funnels you declare, and it places each account in a commercial stage.
Two pages in this group cover how Otis connects a number to an explanation. Candidate causes describes how Otis tests whether a recent change made a measure worse. Evidence describes how Otis picks the specific cases and quotes behind a finding.
Measurements
The named values Otis computes for each user. The Otis app calls a measurement a lens.
Segments and cohorts
The groups of users that Otis compares, including the four built-in segments.
Intent clusters
The groups of tasks in which users were trying to do similar things.
Funnels
The ordered stages you declare, and how Otis counts progress and outcomes.
Lifecycle
The commercial stage of each account, and its movements between plans and stages.
Candidate causes
The tests of whether a deploy, a flag variant or a model change made a measure worse.
Evidence
The specific cases and quotes behind a finding, and what Otis checks before showing them.
What Otis delivers
Otis investigates the data itself, and publishes what it can support as insights. An insight states what Otis found, shows the evidence, and suggests what to do. The daily briefing is a short summary that Otis posts to Slack each morning.
Your team has controls over what Otis analyzes, what it publishes and how it writes. Customization lists each one and where it is set.
Insights
What an insight contains, how Otis checks one before publishing it, and how to read its confidence.
Briefings
What the daily briefing contains, how Otis chooses what goes in it, and how to set it up.
Customization
The controls over what Otis analyzes, what it publishes and how it writes.
How Otis knows your product
Telemetry shows what users did. It doesn't say what your product is for, who it is sold to, or what your team considers a good result. Otis learns those things during onboarding, keeps them as memories and documents, and brings them to each piece of work it does.
This group also covers how far you can rely on what Otis reports. Statistical tests lists the rules Otis applies before it reports a difference. Data health and gaps explains what Otis does when telemetry is missing, late or changed.
Onboarding
The steps that take a new project from creation to its first insight.
Memories and knowledge base
What Otis knows about your product, where that knowledge comes from, and how to correct it.
Context
Which of the things Otis knows reach each part of its work, and what they can change.
Statistical tests
The statistical rules Otis applies before it reports a difference, and what they don't cover.
Data health and gaps
What Otis accepts and drops, how it notices a break in telemetry, and how long it keeps each record.