Funnels

A funnel is an ordered list of stages that something in your product moves through, such as a document that is created, edited and then published. You declare the stages, and Otis counts how far each entry gets and how it ends.

This page explains how Otis counts progress and outcomes, so that you can read a funnel's rates correctly. Funnels and artifacts in the SDK guide covers how your app sends the events.

Purpose of funnels

Most of what Otis infers about success comes from what users say and how operations end. A funnel adds evidence that your product reports directly: this document was published, this onboarding was completed, this draft was discarded.

A funnel therefore does two jobs. It shows where entries stop along a path. It also gives Otis a definite success or failure for the work, which feeds task outcomes and the built-in segments.

The funnel definition

A project can have up to five funnels. Each one declares:

• The path. Between 2 and 20 stages in order. The first stage is the entry.
• Success stages. One or more stages that mean the work was finished, such as publish or share.
• Failure stages. Optional stages that mean the work was given up, such as discard.
• What moves through the funnel. An artifact, a user, or an account. An artifact is one thing a user works on, such as a document, and is the default.
• A conversion window. The number of days an entry has to finish, counted from its first event. The default is 14.

One funnel can be marked as the scoring funnel. If none is marked, the first funnel is the scoring funnel. Otis uses it to decide the success side of the built-in segments.

The definition is written when a coding agent sets up Otis in your codebase. Customization describes how to change one.

Funnel entries

An entry is one thing that reached the first stage. For an artifact funnel, each artifact ID is one entry. For a user funnel, each user is one entry, and for an account funnel, each account.

Three conditions apply:

• The first stage is required. Something that never sent an event for the first stage isn't counted at all, even if it sent later stages.
• Events need a user. A stage event with no user ID is ignored.
• The owner must be recently active. Otis counts an entry only if its owner was active in the last 30 days. For an artifact, the owner is the user on the artifact's first stage event.

Events from other users on the same artifact still count toward that artifact's stages.

Stage progress

Otis credits stages in the declared order. An entry reaches a stage only if it reached every earlier stage first, and within the conversion window.

Stages can't be skipped. An entry that goes from the first stage straight to the third is credited with the first stage only.

The count at each stage is cumulative. An entry that reached the third stage is counted at the first, second and third stages.

Stage names must match the definition exactly. The SDK lowercases the stage names your app sends, so declare stage names in lowercase.

Entry outcomes

Every entry has one of three outcomes:

The three outcomes always add up to the number of entries.

Two rules are easy to miss:

• The outcome doesn't depend on the path. A success stage counts whether or not the entry passed through the stages before it. An artifact that is created and then shared has succeeded, even if the path lists an edit stage between the two. The number of successes can therefore be higher than the count at the last stage of the path.
• Success wins over failure. An entry that sent both inside the window is a success.

Unknown covers two cases together: entries that are still in progress, and entries that ran out of time. An entry runs out of time when its conversion window closes, or when it has sent no stage event for 14 days by default. Only the funnel's own stage events count as activity for that test.

Entry dates

Otis files each entry under the date of its first event, and not under the date of a later stage or of the outcome.

A daily or weekly funnel series is therefore a series of starting groups. The figure for a week says how many of the entries that started in that week have reached a stage so far.

Recent groups are still in progress. Their rates rise on later runs, and their share of unknown outcomes is high. A drop in the most recent period usually reflects how young those entries are.

To compare groups fairly, Otis also measures each starting group at a fixed age: 1 day, 7 days, and the length of the conversion window. A group has a 7-day figure only once more than seven days have passed since it started. A missing figure means not yet measured.

Funnel figures

For each funnel, Otis records the number of entries that reached each stage and the number with each outcome, by starting date. From these it works out:

• Reach at each stage, and the drop-off between one stage and the next.
• Two success rates. One is successes as a share of all entries. The other is successes as a share of the entries that have ended in success or failure.
• The share that ran out of time.

A funnel can also declare one attribute to split by, such as the artifact's type. Otis then reports each figure for each value of that attribute.

For an artifact funnel, Otis can also report the AI cost of each successful artifact and the share of AI cost spent on artifacts that didn't succeed. Both need your AI calls to carry the artifact ID. Artifact context describes how to attach it.

Otis recomputes funnels about every 12 hours for a project that is sending telemetry. Each run counts again every entry that started in the last 30 days, so a late event is picked up on the next run. The period is 90 days for a funnel with a stage that happens only once for each user.

Funnels in other analysis

• Task outcomes. An event for a success stage marks the task it belongs to as a success, and an event for a failure stage marks it as struggled.
• Segments. The first rule for success in the built-in segments reads the scoring funnel.
• Segments you define. A rule can use the stage a user reached, or the outcome of a user's entries, in any funnel.
• Insights. Otis looks for a step that loses a larger share of entries than the others, for a starting group that succeeds at a different rate than the one before it, and for a group of users whose funnel rates differ from everyone else's.

Reading funnels

• In an artifact funnel, the counts are artifacts. One user can own many artifacts. The number of people is a separate count, and it doesn't add up across stages.
• A skipped stage reads as a drop-off. If users can reach a later stage without an earlier one, the path undercounts them. Put on the path only the stages that every entry passes through.
• A funnel with no failure stages always succeeds among ended entries. With no failure stage, every entry that has ended is a success, so that rate is 100%. Read successes as a share of all entries for such a funnel.
• A change to the definition changes past figures. After an edit, Otis counts the last 30 days again under the new definition. Figures from before the edit and after it measure different things. Otis records each edit as a change.
• Keep every funnel in the definition valid. Otis computes a project's funnels together, so one invalid funnel holds up the others.
• Account funnels follow current membership. An account funnel counts each user under the account they belong to now, so past figures move when a user changes account.

Funnels in Otis

Funnels reach you through insights and through answers from Otis in chat, in Slack, or through the MCP server. You can ask where entries drop off, how a rate compares between two periods, or how one group of users compares with another. Otis names a funnel by the ID in its definition.

Related

• Funnels and artifacts covers the SDK calls that send stage events.
• Tasks covers how a stage event becomes a task's outcome.
• Segments and cohorts covers how the scoring funnel decides the built-in segments.
• Lifecycle covers the commercial stages of an account, which Otis tracks separately from funnels.