Methods · Version 1
How we measure
local AI visibility.
A transparent record of what we observe, how we calculate it, and what these numbers can tell us.
Study scope
A research market is one metropolitan statistical area and one Pantora-supported industry. The industry list is the same catalog used across pantora.ai. Metro definitions are pinned to the 50 largest US metropolitan statistical areas by Census Vintage 2025 population, using OMB July 2023 delineations. A metro label describes the intended market; city-targeted searches do not measure every neighborhood or municipality within it.
Business universe and panel
Representative Google Maps searches discover a candidate universe; only businesses whose Google category matches the industry can join a panel. A stable panel targets 10 businesses: four from the market’s Maps leaders (the top twelve by mean reciprocal rank across the prompt set), three middle, and three weaker performers, sampled deterministically within those strata. Businesses that AI names but Maps did not surface are recorded as recommendations outside the panel and shown alongside it. Panel cohorts last about 90 days; closed, invalid, and duplicate businesses can be removed earlier. This is a stratified, non-probability sample, not a census of local businesses.
Stable prompts
Prompt sets combine general recommendations, reputation, booking intent, and service terms from Pantora’s industry knowledge. Every prompt explicitly names the metro. Prompt versions are immutable. Changing a term creates a new version; trend comparisons across different prompt meanings are treated as a methodology break. No individual business is named in discovery prompts. The Query Explorer shows each prompt verbatim with the ordered answers each provider returned.
Businesses AI recommends
When an AI answer recommends a business that Google Maps did not surface for the prompt set, that business is looked up in Google Maps by name and city. When the returned listing’s name and category match, its place, rating, review count, coordinates, and category are folded into the dataset, so a business that AI recommends but that does not rank in the local Maps results still carries real listing data and can be measured. Businesses that cannot be matched to a confident listing remain recommendation-only.
Website signals
Each measured business’s homepage is read once a month with the Jina reader. From the returned text we record content length (word count), the presence of trust content (FAQ, testimonials, credentials, guarantees, financing, emergency service, online scheduling, contact options, service areas), and the density of the industry term. Keyword relevance (service coverage) is the share of the industry’s ten study service topics the homepage names, allowing for word forms. Technical signals are measured directly from the business’s own domain, also monthly: whether /llms.txt returns a plain-text file (HTML soft-404s and redirects do not count); whether robots.txt lets the major AI assistant crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, Google-Extended, PerplexityBot, ClaudeBot, Claude-SearchBot) fetch the homepage; the Google Lighthouse mobile performance score from PageSpeed Insights; schema.org LocalBusiness and FAQ structured data in the homepage HTML; the number of page URLs in the site’s XML sitemaps (counted up to 1,000); and domain age from the public registry (RDAP) registration date. Directory and social profiles are not treated as a business’s website. A blocked, failed, or timed-out check is recorded as unavailable, never as absent. Homepage checks do not describe the whole site.
Sources and location
New AI observations use OpenAI with web search and Gemini grounded with Google Search. Each response records its actual model, sources, and API channel. Earlier scraper observations retain their original channel labels. API responses are measurements of those model APIs, not reproductions of the ChatGPT or Gemini consumer interfaces. Google Maps and organic observations come from SERP data for the same prompts. Google AI Overviews are not currently collected.
Entity resolution
Provider place identifiers take priority. Domain, phone, address, name, and location provide supporting evidence. A shared chain domain is not enough to merge locations. Ambiguous and newly mentioned businesses remain provisional until validated. Mention extraction keeps source evidence and omits fields that cannot be supported by the response.
Collection and evidence
Visibility and profile measurements are scheduled weekly; authority checks are monthly. Individual observations preserve timestamps, prompt versions, parser versions, raw-response references, and costs. Monthly snapshots retain their actual observation dates.
Visibility measures
AI visibility (mention rate) = completed eligible answers recommending a business divided by completed eligible answers. Top-3 rate uses the first three recommendations. Mean position excludes answers that did not mention the business. Share of voice divides a business’s mentions by all resolved business mentions. Maps visibility is mean reciprocal rank, with absence scored as zero in completed Maps results. Organic visibility uses the same convention for matched domains. In the dashboard, Maps, review, and authority positions are shown as percentiles among the businesses in view.
Overlap and agreement
AI / Maps overlap compares the businesses an AI answer recommended with the first ten Google Maps results for the same prompt, as the size of the intersection divided by the size of the union. Provider agreement compares ChatGPT and Gemini on the same prompt the same way. Prompts where neither side returned any business are excluded. Missing or failed observations are not treated as zero overlap. Stability is the share of recommended businesses retained between consecutive periods measured with the same sources.
Relationships
Each signal is compared with AI visibility on its own. For numeric signals we compare the average mention rate of the top quarter of businesses on that signal with the bottom quarter; for yes/no signals, businesses with the feature against those without. The ratio is reported only when each group has at least eight businesses; otherwise a yes/no signal reports how many sites have it. Strength labels come from rank-based correlation with tie-aware ranks over all measured businesses, reported from 30 complete pairs with variation in both measures: weak below 0.3, moderate below 0.55, strong above. When a city or industry view has too few businesses to compare signals, its comparisons use every published market and say so. When markets are pooled, each business is compared within its own market first: quarters are set within each market and mention rates are read against that market’s average, so a market with more businesses (and so lower rates per business) cannot create a difference on its own. Pooled views can repeat businesses across markets. Signals travel together (businesses with more reviews also tend to have older domains and more links), so results are exploratory associations without causal or significance claims. Plots display at most 500 points.
Publication criteria
A market is published only with at least 10 verified panel businesses, at least 80% of planned prompts completed, coverage from both AI providers, and observations no older than 21 days. Unresolved recommended businesses block publication. Development fixtures are never eligible. Views without a current sample are not indexed and are omitted from the labs sitemap. Aggregate views summarize only published markets and should not be read as full coverage of the target universe.
Limitations and editorial review
AI outputs vary across runs, interfaces, models, and time. Rankings are observations rather than endorsements. Ratings, reviews, and authority may be incomplete or stale between collection dates. Visibility is not a professional recommendation, safety assessment, or medical, legal, or financial advice. Candidate findings require human review before they appear on the dashboard. Conclusions use observational language and do not establish causation.
Census metro population source · OpenAI web search documentation · Gemini grounding documentation