How Otis knows your product
Onboarding
The steps that take a new project from creation to its first insight, what Otis checks along the way, and when each kind of result first appears.
Onboarding is the set of steps that gives Otis what it needs to analyze your product: an understanding of the product, telemetry from your code, and a check that the telemetry is usable. You do it in a chat with Otis and with a coding agent in your repository.
This page explains what happens at each step, what the progress bar measures, and how soon after your first telemetry each kind of result appears.
The two halves of onboarding
Otis needs two things that it can't get from each other. It needs to know what your product is for, which only your team can say. It also needs events from your code, which only a change to your code can provide. The chat covers the first, and the coding agent covers the second.
Onboarding steps
- Create a project. A team admin enters the product's name. Otis opens the onboarding chat, which every member of the team can see.
- Describe your product. Otis asks five questions, one at a time: what the product does, its key features, how people use it, the stage of the business, and the measure you care about most. It saves each answer as a memory. It also asks whether your users need to consent before analytics cookies are set.
- Analyze your codebase. The chat shows how to install the Otis skills into your coding agent. You then run the
otis-analyzeskill. The agent profiles your code and scores how well each measurement fits your product. - Review the recommended measurements. The chat shows them in a panel titled "Review lenses". A strong fit is selected for you, a good fit is left for you to choose, and a poor fit isn't shown.
- Instrument your code. You run the
otis-instrumentskill. The agent installs the SDK, adds the events that your chosen measurements need, and opens a pull request. - Deploy, and let Otis verify. Once telemetry arrives from production, Otis checks it, as described below.
- Add more context. After instrumentation, Otis asks about your ideal customer, your challenges, what you want from analytics, and your plans and pricing. Lifecycle describes the plan setup.
- Connect GitHub. This is the last step, and you can skip it.
Agent skills describes the coding-agent steps in detail.
Background research
When you create a project, Otis researches your company on the public web. It starts from the domain of your work email address, and does nothing for a personal email address. The research covers your company and product, what your founders have said publicly, who your customers appear to be, your competitors, and recent changes to your product or pricing.
The report goes into your project's knowledge base. Otis treats it as unverified. In the chat, it checks that the report describes the right company and goes through it with you. It saves a claim as a memory only after you confirm it.
Skipping steps
You can skip the cookie question, the review of measurements, the teammate invitation and the GitHub step. You can run otis-analyze again at any time, and measurements you have already kept or skipped keep their status.
Onboarding doesn't gate analysis. Otis starts analyzing as soon as production telemetry arrives, and you can open an ordinary chat with Otis at any time. A project with little context still gets insights. Otis has less to judge them by, so they are less well targeted.
The progress bar
The onboarding chat and the project's home page show a progress bar with the name of the current stage, such as "Gathering context", "Instrumenting", "Verifying ingest" or "Fixing issues".
The bar fills from four things:
- Context. How many of the questions Otis asks have an answer saved.
- Instrumentation. Whether the measurements were reviewed, the SDK is installed, identified users are arriving, and verification found no blocking problem.
- GitHub. Connected or skipped.
- Depth. How specific your answers are, as judged by an AI model.
A saved answer counts whatever it says, so the depth score is what separates a thorough answer from a one-word one. When the bar is nearly full, the chat shows "Otis's got what it needs!".
"Spans arriving" means that Otis has seen an identified user in the last seven days. Telemetry with no user ID doesn't count, because Otis can't attribute it to anyone.
Verification
After a measurement is instrumented, Otis checks the last seven days of telemetry for it:
- Whether the events arrive. Each event that the setup agent declared should appear.
- Whether they carry a user ID and a session ID.
- Whether your app identifies users and sets user and account properties.
- Whether artifact events carry an artifact ID.
Otis checks every 15 minutes just after a change to your instrumentation, and less often as time passes without one.
Problems are graded. A blocking problem turns the home page card red and holds the stage at "Fixing issues". Examples are events with no user ID, an app that never identifies users, and a declared event that has gone silent while a similarly named one is arriving. A warning doesn't block. An instrumented measurement with no events yet is only a warning, because Otis can't tell an event that was never shipped from a feature nobody has used yet.
When every declared event has arrived and no blocking problem remains, Otis marks the measurement as verified. It never removes that mark automatically.
Verification results
- The home page card shows "Telemetry issues detected" for a blocking problem, and "Review telemetry" for a warning.
- Ask Otis in a chat for the list of problems. It gives each one with a suggested fix.
- Run the
otis-verifyskill in your coding agent to see the same list next to your code.
First results
Otis runs more often for a new project, and returns to its usual pace over the first two weeks.
| Result | What it waits for |
|---|---|
| Counts on the Ingestion tab | Nothing. They update while the tab is open. |
| Sessions and tasks | The session has been idle for 30 minutes and is large enough to analyze. |
| Measurements | The first run, within a few hours. |
| Segments | A funnel success or a reported task success somewhere in the project. |
| Intent clusters | Enough recent tasks to group. |
| First tested findings | Seven days of telemetry. |
| First insight | The next early-morning investigation that has a candidate. |
| First briefing | The next morning with something to report, if Slack is connected. |
Data health and gaps lists what each result waits for.
Changing your answers later
- To correct what you told Otis, say so in any chat in the Otis app. The new answer replaces the old one.
- To change which measurements are kept, return to the onboarding chat, which stays available, or use the setup agent.
- To change plans and lifecycle settings, ask Otis in a chat in the Otis app.
- To change tone and focus, use the Otis Guidelines tab in settings. Customization lists every control.
Related
- Agent skills covers the setup commands that run in your coding agent.
- Memories and knowledge base covers where your answers and the research report are kept.
- Measurements covers what you choose in the review step.
- Data health and gaps covers what Otis does when telemetry is missing, late or changed.