How we measure
Last updated: October 2026
Important limitation. Quotalis measures AI answers through the models' official developer APIs — except ChatGPT, Claude, Gemini, Perplexity, which are currently routed through the OpenRouter gateway (a third-party aggregator, not the provider's official API) — it does not measure the consumer chat apps of the engines listed below, and API results do not necessarily match what a consumer sees in those apps. Engines measured without web-search augmentation reflect base-model responses. Treat our numbers as a consistent, comparable signal — not as a literal reading of consumer behavior.
1. Measurement design
- Each prompt × engine cell is run 3 times per measurement cycle (independent API calls).
- Visibility % for a cell = mentions ÷ completed runs. Dashboard visibility is the mean across cells, always shown with the min–max range (e.g. “67% (range 2–2 of 3 runs)”). We never report bare point scores.
- Starter tier runs weekly; Growth and Pro run weekly as well — daily measurement would cost more in model calls than any sustainable price.
- A cycle missing >20% of its runs is flagged partial and retried — incomplete data is never silently presented as complete.
2. Engines and models
| Engine | API | Model | Web access |
|---|---|---|---|
| ChatGPT | OpenRouter gateway → openai/gpt-4o | openai/gpt-4o (configurable) | OpenRouter web search plugin (unified) |
| Claude | OpenRouter gateway → anthropic/claude-sonnet-4-5 | anthropic/claude-sonnet-4-5 (configurable) | OpenRouter web search plugin (unified) |
| Gemini | OpenRouter gateway → google/gemini-2.5-flash | google/gemini-2.5-flash (configurable) | OpenRouter web search plugin (unified) |
| Perplexity | OpenRouter gateway → perplexity/sonar | perplexity/sonar (configurable) | OpenRouter web search plugin (unified) |
Models are pinned per deployment and shown on every stored run. We never scrape logged-in chat UIs — that would violate their terms of service and produce flaky data.
3. Mention & citation detection
- A run counts as a mention when your brand name, any alias you provide, or your domain appears in the response text. Matching is Unicode-aware and word-boundary aware: a Chinese alias like “wu的智脑团” only matches as a whole — a lone “wu” or “agent” elsewhere in the text does not count.
- A run counts as a citation when a source URL from your domain appears in the engine's citation list. When your domain includes a path (e.g. a social profile like
x.com/DavidAgentic), the citation URL must point at that path or below it — other links on the same host do not count. - Citation share = citations to a domain ÷ all citations in the cycle.
- Share of voice = runs mentioning a subject ÷ all completed runs in the cycle, across your brand and tracked competitors.
4. What we don't do
- No consumer-app scraping. No claim that API behavior equals app behavior.
- Sentiment scoring is experimental (not in MVP): per independent research, LLM-judged sentiment flips ~6.7× more often than mention counts, so it is never our headline KPI.
- We don't fabricate data: every dashboard number traces to stored API responses, which you can inspect run-by-run in the response viewer.
5. Cost & fairness
Model API calls cost us money, so each tier has a monthly measurement budget. If a cycle would exceed your budget, tracking pauses automatically and we notify you — we never silently burn budget.