Methodology

How every number is collected, computed and qualified

AI assistants don't return the same answer twice. Every number in Huginnova is an estimate from a sample of real answers, so we show how it was collected, how it was computed and how certain it is. Nothing here is a black box.

How answers are collected

For each tracked prompt we ask every selected engine the question exactly as a buyer would type it, without mentioning the brand. We store the full answer text and every source it cites, so every metric can be traced back to the answers behind it.

EngineCollection method
ChatGPTThe real ChatGPT interface, as a signed-out user sees it, including the sources it cites.
PerplexityPerplexity Sonar, which searches the web for every answer.
GeminiThe real Gemini interface, including the sources it cites.
Google AI OverviewsThe AI Overview on a live Google results page. If Google shows none, that is recorded as an answer without your brand.
Google AI ModeLive Google AI Mode results, with their references.
ClaudeClaude with web search on (add-on).

Each project sets a market (country) and language. The country is passed as the user location to engines that support it, and the language selects the Google interface language. Results are only representative of that market.

Where an engine offers a consumer interface we collect from it, because API answers can differ from what people see (model version, search behaviour, interface features). Data is collected through a specialised provider; every stored answer records which method produced it.

Repeated sampling

The same prompt can produce different brands, orderings and sources from one run to the next. A single answer per week is an anecdote, not a measurement. We therefore ask each prompt several times per engine on every run (3 by default) and report rates across all of those answers.

More samples mean narrower confidence intervals and earlier detection of real changes, at a proportional collection cost.

Mentions and citations

A mention is the brand's name (or one of its aliases or its domain) appearing in the answer as a whole word, case-insensitively. The order in which tracked brands first appear gives each one a position (1 = named first).

A citation is a source URL the engine attached to the answer or included in its text. URLs are normalized (tracking parameters removed, duplicates collapsed) and attributed to your site, a competitor's site, or a third party by domain.

Metric definitions

AI visibility: the share of collected answers that mention the brand.

visibility = answers mentioning the brand ÷ total answers

Share of voice: the brand's share of all mentions of tracked brands (yours and your competitors'). It depends on which competitors you track.

share of voice = brand mentions ÷ mentions of all tracked brands

Cited as a source (citation rate): the share of answers that cite at least one page on your domain.

Average position: the mean position of the brand across the answers that mention it. Answers that don't mention the brand are excluded.

Sentiment: how each answer portrays the brand, scored from −1 (very negative) to +1 (very positive) in five steps. When an Anthropic API key is configured, a Claude model rates the answer's own language about each brand (recommendation strength, praise, caveats, criticism), not the model's opinion of the brand. Otherwise a keyword lexicon is applied to the sentence around the mention. The excerpt behind every score is stored and shown.

Confidence intervals

Every rate is shown with a 95% Wilson score interval, which stays accurate for small samples and for rates close to 0% or 100%. In charts, the shaded band is the interval and the marker is the point estimate.

center = (p + z²/2n) ÷ (1 + z²/n), margin = z·√(p(1−p)/n + z²/4n²) ÷ (1 + z²/n), z = 1.96

Read it as: if we kept collecting answers the same way, the true rate would fall inside this range 95% of the time.

Is a change real?

Every “vs previous period” comparison runs a two-proportion z-test. A change is labelled significant only when p < 0.05; otherwise it is labelled “not significant” and should be read as normal variation between answers. This prevents reporting noise as wins or losses to clients.

z = (p₂ − p₁) ÷ √(p̂(1−p̂)(1/n₁ + 1/n₂)), p̂ = (x₁ + x₂) ÷ (n₁ + n₂)

Measuring action impact

When an action in the plan is marked done, we record the date. Its impact compares the brand's visibility in the 28 days before with the 28 days after (or up to today), restricted to the prompts the action targets, or all prompts for site-wide actions, using the same significance test.

This is a before/after comparison, not a controlled experiment: engine updates or competitor activity in the same window can also move the numbers. Comparing targeted prompts against the rest of the project helps separate the two.

Demo data

Engines without credentials can run on synthetic answers so the product can be explored before connecting data sources. Synthetic answers are stored with the source “demo”, and every page and report that includes any of them shows a visible demo-data notice. Production workspaces disable demo data entirely.

Limitations

  • Visibility reflects the prompts you track. A different prompt set gives a different picture, so choose prompts that match real buyer questions.
  • Engines personalize answers for signed-in users. We measure a neutral, signed-out experience for the chosen market.
  • Name matching can miss paraphrases (“the Swedish furniture giant”) and can over-count generic names. Add aliases to improve recall.
  • Engines change models and interfaces without notice. Sudden shifts across every brand at once usually indicate an engine change rather than a change in your position.
  • Sentiment is an automated judgement. Check the stored excerpts before acting on it.