# How often do AI search engines get answers and citations wrong? > As of 17 November 2026, no single study establishes a universal error rate for AI search. Source: https://llmsearchnews.com/ai-answer-accuracy-studies-2026 · Publisher: LLM Search News · Published: 2026-11-17 · Updated: 2026-11-17 ## TL;DR - The 2025 EBU/BBC study found significant issues in 45% of 2,709 evaluated news responses, while 81% contained some minor or significant issue.[7] - The Tow Center found that more than 60% of 1,600 controlled source-identification queries produced incorrect results, with Grok 3 incorrect on 94% of its test queries.[1][3] - Academic studies distinguish factual accuracy, citation faithfulness, URL validity and source selection rather than treating them as one error category.[4] - A live citation can resolve without supporting the claim, so URL validity is not proof of citation correctness.[4] - The available evidence does not establish one universal 2026 AI-search error rate, and the 2025 EBU/BBC and Tow Center figures should not be presented as current platform ratings.[3] As of **17 November 2026**, no single study establishes a universal error rate for AI search. The strongest available evidence instead measures different problems: the EBU/BBC found significant issues in 45% of evaluated news responses in 2025, while a separate Tow Center test produced incorrect results for more than 60% of controlled source-identification queries.[1] ## What happened in the latest AI search accuracy studies? Recent studies have examined four related but distinct questions: - **Does the answer accurately represent the news or evidence?** The EBU/BBC study evaluated news responses for accuracy, context and sourcing problems.[2] - **Can the system identify the correct source?** The Tow Center tested whether tools could recover an article and its metadata from an excerpt.[3] - **Does a citation resolve and support the claim?** Academic studies separately measured URL validity, citation faithfulness and source selection.[4] - **Can an assistant answer questions about newly published facts?** LiveBrowseComp tested information published within the previous 90 days.[5] These studies use different units—responses, queries, definitive answers or URLs—so their percentages cannot be combined into one AI-search accuracy score.[1] | Study | Engines or systems | Sample | Key finding | Date | |---|---|---:|---|---| | EBU/BBC *News Integrity in AI Assistants* | AI assistants used for news questions | 2,709 core responses, plus 353 custom responses | 45% of core responses had at least one significant issue; 81% had some issue when minor and significant issues were combined.[2][6] | Responses generated 24 May–10 June 2025; report published October 2025.[7] | | Columbia Tow Center source-identification test | ChatGPT Search, Perplexity, Perplexity Pro, DeepSeek Search, Copilot, Grok 2, Grok 3 beta and Gemini | 1,600 excerpt-based queries | More than 60% of results were incorrect overall; Perplexity was incorrect on 37% and Grok 3 on 94% of queries.[3] | 2025.[3][8] | | Citation-faithfulness study | LLM-based information-retrieval systems | Reported experiments on citation attribution | Up to 57% of citations lacked faithfulness, meaning the citation did not reliably represent the evidence used to produce the answer.[1][4] | 2025. | | Chinese factual-search benchmark | Nine search engines, standalone LLMs and AI Overviews | Evidence-derived Chinese yes/no questions | Conditional accuracy ranged from 73.2% to 78.9%, while end-to-end accuracy was lower for several systems when uncertainty counted as an error.[1] | 2026.[1] | | Reference-hallucination study | Ten commercial LLMs and deep-research agents | DRBench citation outputs | 3.0%–13.3% of citation URLs were hallucinated and 5.4%–18.5% were non-resolving.[4] | 2026. | | Google AI Overviews citation audit | Google AI Overviews | Citation-selection analysis | AI-generated documents were cited more frequently than human-authored documents after retrieval rank was controlled, largely because of non-retrieved citations.[9] | 2026.[9][4] | | LiveBrowseComp | Search-augmented agents | 335 questions about facts published within the previous 90 days | All evaluated agents scored below 2% closed-book accuracy; search-augmented scores were 25–40 points below BrowseComp.[5] | 2026. | ## Why does this matter? AI search answers can fail in more than one way. A response may state a false fact, omit necessary context, identify the wrong publisher, provide a broken URL, cite a live page that does not support the claim, or rely on a source without faithfully reflecting it.[4] The distinction is especially important for news. In the EBU/BBC study, sourcing was the most common significant problem category, affecting 31% of core responses. Accuracy issues affected 20%, while insufficient-context issues affected 14%.[6] The study’s 81% figure also requires careful wording. It means that 81% of evaluated responses contained at least one issue when minor and significant issues were included; it does not mean that 81% of answers were seriously false.[7] Citation presence is not equivalent to citation correctness. A URL can resolve successfully while failing to support the statement attached to it, while a citation-faithfulness test asks whether the source genuinely supports the generated claim rather than merely being attached after the answer was formed.[4] Freshness creates another difficulty. LiveBrowseComp used questions about facts published during the 90 days before benchmark construction, and the evaluated agents performed poorly on that time-sensitive task.[5] A system can therefore appear reliable on established facts while struggling with newly reported events. ## What do we know about the EBU/BBC findings? The EBU/BBC report brought together 22 public-service media organizations across 18 countries and 14 languages. In total, 271 journalists reviewed 2,709 core-question responses and 353 custom-question responses.[3] The core responses were generated between 24 May and 10 June 2025.[7] Reviewers assessed news answers for categories including accuracy, sourcing and context. The principal results were: - 45% of core responses contained at least one significant issue.[2] - 81% contained an issue when minor and significant issues were counted together.[7][2] - Significant sourcing issues affected 31% of responses.[2] - Accuracy issues affected 20% of responses.[6][2] - Context issues affected 14% of responses.[7][2] The report also describes a BBC-to-BBC comparison in which the share of responses with significant issues fell from 51% in an earlier round to 37%.[6] The report cautions that the broader multi-publisher results cannot be directly compared with the first BBC-only study.[6] The evaluation was not adversarial and was not designed to compare countries or languages.[4] A like-for-like 2026 EBU/BBC result with a new comparable percentage is not verified in the primary material available for this article. ## What did the Tow Center test actually measure? The Tow Center tested eight generative search tools: ChatGPT Search, Perplexity, Perplexity Pro, DeepSeek Search, Microsoft Copilot, Grok 2, Grok 3 beta and Google Gemini.[3] Researchers selected ten articles from each of 20 publishers, supplied excerpts and asked the systems to identify the headline, original publisher, publication date and URL.[3] This produced 1,600 queries.[1] The scoring required the system to identify the correct article, publisher and URL. More than 60% of answers were incorrect overall.[3] Perplexity was incorrect on 37% of its queries, while Grok 3 was incorrect on 94%.[3] DeepSeek misattributed the source of excerpts 115 times out of 200.[3] More than half of Gemini’s and Grok 3’s responses cited fabricated or broken URLs, and 154 of Grok 3’s 200 citations led to error pages.[3] The test does not provide a general accuracy rate for open-ended AI search. It measured a controlled source-identification task in which systems had to recover a known article and associated metadata from an excerpt.[3] The researchers also cautioned that the task might not reflect typical user behavior, that each excerpt was queried once and that the findings should not be extrapolated to all models or news organizations.[3] A comparable 2026 Tow Center re-test is not verified in the primary Tow Center material used here.[3][8] ## What do academic studies add? ### Citation faithfulness is different from citation correctness A 2025 study distinguishes citation correctness from citation faithfulness. Correctness asks whether a cited document supports a statement; faithfulness asks whether the system genuinely relied on that document rather than producing an answer first and attaching a plausible source afterward.[4] The study reported that up to 57% of citations lacked faithfulness.[10] That result concerns citation attribution in LLM-based information retrieval and is not a live audit of every consumer AI search product.[1] ### URL validity is not evidence support A 2026 study of ten commercial LLMs and deep-research agents tested citation outputs on DRBench.[4] It found that 3.0%–13.3% of citation URLs were hallucinated and 5.4%–18.5% were non-resolving.[4] The study defined a non-resolving link as a URL that returned an HTTP error or failed to connect. A hallucinated URL was a non-resolving URL with no archived Wayback Machine snapshot.[4] Those measurements concern whether URLs exist and resolve. They do not directly measure whether a working page supports the claim made in the answer.[4] ### Source selection can also be biased A 2026 audit of Google AI Overviews found that AI-generated documents were cited more often than human-authored documents even after retrieval rank was controlled.[9] The difference was driven mainly by citations to documents that were not retrieved in the underlying search process.[9] This is a source-selection and provenance finding, not a claim that every affected citation was false or an overall percentage of incorrect citations.[4] ### Fresh factual information remains difficult LiveBrowseComp contains 335 human-authored questions about facts published in the 90 days before the benchmark was constructed.[5] All evaluated agents scored below 2% on the closed-book version, and search-augmented results were 25–40 points lower than BrowseComp scores.[5] The benchmark measures performance on recent information. It does not establish citation correctness across search products.[1] ## What is confirmed, and what is not? ### Confirmed by the cited studies - The EBU/BBC’s 2025 news study found significant issues in 45% of 2,709 core responses.[6] - The same study found some issue, minor or significant, in 81% of core responses.[1] - Significant sourcing issues were the largest reported category, affecting 31% of core responses.[2] - The Tow Center’s 1,600-query source test produced incorrect results in more than 60% of cases overall.[4][3] - Citation studies measure separate failure modes, including URL resolution, citation faithfulness and source selection.[4] - Fresh-fact benchmarks show that current information can be substantially harder for agents than established information.[7] ### Not confirmed by this evidence - There is no single overall 2026 error rate for all AI search answers.[3][4] - The EBU/BBC 2025 percentage is not a current November 2026 platform rating.[7] - The Tow Center’s result is not a general factual-accuracy percentage for every search query.[1][7] - A resolving URL does not prove that the cited page supports the answer.[4] - A citation-selection difference does not prove that every selected citation is incorrect.[3] - The available material does not verify a like-for-like 2026 EBU/BBC or Tow Center re-test.[3] ## How should AI search accuracy be reported? A technically meaningful report should identify the task, sample, denominator, date and failure definition. 1. **Name the task.** State whether the study measured news quality, source identification, factual correctness, citation support, URL validity or freshness. 2. **Name the denominator.** Specify whether the percentage refers to responses, queries, claims, definitive answers or URLs. 3. **Give the collection date.** AI-search outputs change over time, and the EBU/BBC responses, for example, were generated during a defined 2025 period.[1][7] 4. **Separate conditional and end-to-end accuracy.** A system can be accurate when it answers definitively but perform worse when uncertain or missing answers count as failures.[1] 5. **Do not treat citations as binary proof.** Check whether the URL resolves, whether the page supports the claim and whether the response faithfully reflects the source.[4] 6. **Avoid a blended headline percentage.** The current evidence does not support one universal 2026 AI-search error rate.[1][4] Background: why AI search accuracy studies disagree AI-search systems generate answers through different combinations of retrieval, ranking, model generation and citation selection. A benchmark that asks a system to identify a known article therefore tests a different capability from one that asks whether a newly published fact is correct. The denominator also changes the result. A study counting invalid URLs can report a URL-level failure rate even when the surrounding answer is correct. A study counting response-level significant issues can include sourcing, context and accuracy problems in the same response. A benchmark that counts uncertain answers as wrong will produce a lower end-to-end score than one that evaluates only definitive answers.[1] For that reason, percentages should always remain attached to the study design that produced them. ## FAQ ### How accurate are AI search engines in 2026? There is no verified universal 2026 accuracy rate for AI search engines. Available studies report different results because they test different tasks, including news quality, source identification, citation validity and fresh-fact retrieval.[4] ### What percentage of AI news answers were wrong in the EBU/BBC study? The 2025 EBU/BBC study found that 45% of 2,709 core responses had at least one significant issue. When minor and significant issues were combined, 81% had some issue; the study did not say that 81% of answers were seriously false.[7] ### What did the Tow Center find about ChatGPT citations? The Tow Center tested ChatGPT Search alongside seven other generative search tools in a controlled source-identification task. More than 60% of all results were incorrect, but the published overall figure covers article, publisher and URL identification across the full test rather than ChatGPT’s general factual accuracy.[4][3] ### Does a working AI citation prove that an answer is correct? No. A working URL only shows that the address resolves; it does not establish that the page supports the claim. Citation-faithfulness research separately examines whether the answer genuinely reflects the cited evidence.[1][4] ### Why do some AI accuracy studies report higher numbers? Studies may count only definitive answers, while others count uncertain or missing answers as errors. In the Chinese factual-search benchmark, conditional accuracy was higher than end-to-end accuracy because the end-to-end measure penalized uncertainty and incomplete answer coverage.[1] ## Sources [1] Evaluating Reliability Asymmetries in Chinese Factual Search and AI Answers — https://arxiv.org/html/2602.22221 [2] Largest study of its kind shows AI assistants misrepresent news content 45% of the time – regardless of language or territory — https://www.bbc.com/mediacentre/2025/new-ebu-research-ai-assistants-news-content [3] AI Search Has a Citation Problem - Columbia Journalism Review — https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php [4] Detecting and Correcting Reference Hallucinations in Commercial LLMs and Deep Research Agents — https://arxiv.org/html/2604.03173 [5] [2605.28721] LiveBrowseComp: Are Search Agents Searching, or Just Verifying What They Already Know? — https://arxiv.org/abs/2605.28721 [6] News Integrity in AI Assistants — https://www.bbc.co.uk/aboutthebbc/documents/news-integrity-in-ai-assistants-report.pdf [7] Search Engines in the AI Era: A Qualitative Understanding to the False Promise of Factual and Verifiable Source-Cited Responses in LLM-based Search | Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency — https://dl.acm.org/doi/full/10.1145/3715275.3732089 [8] Tow Center's Latest Report on AI Search Engines | Columbia Journalism School — https://journalism.columbia.edu/news/tow-ai-report-2025 [9] Auditing Citation Behavior in AI-Generated Search Summaries: A Framework and a Case Study of Google AI Overviews — https://proceedings.mlr.press/v318/kakimov26a.html [10] Correctness is not Faithfulness in Retrieval Augmented Generation Attributions | Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR) — https://dl.acm.org/doi/10.1145/3731120.3744592 --- # How much do AI crawlers crawl compared to what they send back? > Cloudflare Radar’s 17 November 2026 snapshot records 3,351.73 Perplexity crawls, 2,073.96 OpenAI crawls and 456.23 Anthropic crawls per observed referral over the last seven days. Source: https://llmsearchnews.com/ai-crawler-traffic-2026 · Publisher: LLM Search News · Published: 2026-11-17 · Updated: 2026-11-17 ## TL;DR - Cloudflare Radar’s 17 November 2026 snapshot records 3,351.73 Perplexity crawls, 2,073.96 OpenAI crawls and 456.23 Anthropic crawls per observed referral over the last seven days.[7][1] - The OpenAI and Anthropic figures are platform-level ratios, not GPTBot-only or ClaudeBot-only measurements.[3] - Fastly measured approximately 30% AI-request growth from January through May 2026, around 6.5 times the growth rate of human traffic.[11] - Cloudflare Pay Per Crawl remains documented as a closed-beta payment mechanism, with no cited market-wide transaction or payout totals.[17][15] - Cloudflare’s November 2026 access-policy documentation lists Allow as the new-domain default for Search, Agents and Training, so older default-blocking headlines need a date and qualification.[26][13] As of **17 November 2026**, AI platforms still crawl substantially more HTML than the measurable referrals they send to publishers, but the gap varies by platform and observation window: Cloudflare Radar’s latest seven-day snapshot records about **2,073.96 OpenAI crawls per observed referral, 456.23 Anthropic crawls, and 3,351.73 Perplexity crawls**.[1][2] These are platform-level figures, not ratios for GPTBot or ClaudeBot alone, and Cloudflare warns that missing referral headers—particularly from native apps—can inflate the measured gap.[1] ## What happened? Cloudflare Radar’s 17 November 2026 snapshot shows a large but uneven difference between HTML crawling and observed referrals. Perplexity recorded the highest current platform-level ratio at **3,351.73 crawls per observed referral**, followed by OpenAI at **2,073.96** and Anthropic at **456.23**.[3][1] The measurement covers the dashboard’s **last seven days**, rather than the full month of November.[1] Cloudflare’s calculation counts HTML crawl requests from associated crawler user agents and compares them with HTML requests containing a platform-associated `Referer` hostname.[4] That methodology means the figures should be described as **crawls per observed referral**, not crawls per visitor, customer, conversion or unit of publisher revenue.[1] They also do not establish how many requests came specifically from GPTBot or ClaudeBot, because the platform aggregates multiple bots under each operator.[4][1] Cloudflare’s rendered bot-traffic chart separately displays **ClaudeBot at 12.1%** and **GPTBot at 8.1%**, but the accessible chart text does not define whether those percentages use all HTTP requests or a crawler-only denominator.[5][6] They should therefore be reported as the chart’s displayed percentages, not as shares of the entire web.[7] ## Why does the crawl-to-referral gap matter? The gap matters because crawling and clicking produce different outcomes for publishers. A crawler can retrieve a page for training, search indexing, answer generation or an agent task without sending a person through to the publisher’s site.[8][9] The current measurements do not show that every crawl is intended for model training. Cloudflare reported that training represented **52% of crawler requests in June 2026**, compared with **22% in spring 2025**, while its classification also distinguishes search, agent and other purposes.[8] Fastly’s 2026 data shows that AI traffic is growing as an infrastructure workload as well as a content-distribution issue. In a fixed customer cohort covering **January 1 through May 31, 2026**, AI requests grew by approximately **30%**, around **6.5 times faster than human traffic**.[10][11] Fastly also reported that AI requests were more likely than human requests to reach origin infrastructure: less than **9% of human requests** reached origin in May 2026, compared with more than **51% of AI requests**.[11] That difference affects caching, bandwidth, origin capacity and bot-management decisions, although the cited analysis does not provide a dollar cost per request.[8] The economic question is separate from the technical one. A 2026 academic study using U.S. desktop clickstream data from **October 2024 through July 2025** found a clean outbound click in **5.2% of ChatGPT conversation sessions**, compared with **31.1% of Google queries**.[2] Those are user-behaviour measurements, not crawler-to-referral ratios, and neither result measures publisher revenue.[12][1] ## What do the latest measurements show? Cloudflare’s current platform-level values are: | Crawler or platform | Operator | Share or ratio | Source | Date | |---|---|---:|---|---| | GPTBot and related OpenAI bots | OpenAI | 2,073.96 crawls per observed referral | Cloudflare Radar | 17 November 2026[1] | | ClaudeBot and related Anthropic bots | Anthropic | 456.23 crawls per observed referral | Cloudflare Radar | 17 November 2026[1] | | Perplexity crawler group | Perplexity | 3,351.73 crawls per observed referral | Cloudflare Radar | 17 November 2026[12][5] | | ClaudeBot | Anthropic | 12.1% of the chart’s displayed bot-traffic measure | Cloudflare Radar | 17 November 2026[13][1] | | GPTBot | OpenAI | 8.1% of the chart’s displayed bot-traffic measure | Cloudflare Radar | 17 November 2026[13][1] | | Googlebot | Google | 25.5% of the chart’s displayed bot-traffic measure | Cloudflare Radar | 17 November 2026[13][1] | Cloudflare’s current ratio differs sharply from its historical 2025 figures. In the monthly comparison for **January and July 2025**, Anthropic fell from **286,930.1:1** to **38,065.7:1**, OpenAI moved from **1,217.4:1** to **1,091.4:1**, and Perplexity rose from **54.6:1** to **194.8:1**.[1] Those monthly values cannot be treated as a direct continuation of the November 2026 rolling-week snapshot. The observation windows, measurement periods and platform activity changed between the datasets.[4][14] Cloudflare’s 2025 historical analysis also recorded different movements among individual bots. Between May 2024 and May 2025, GPTBot requests grew by **305%**, while ClaudeBot requests fell by **46%** in the fixed-customer comparison.[13][1] The data therefore does not support one universal ranking such as “the worst crawler” or a permanent GPTBot crawl ratio. The answer depends on the operator, included user agents, time window, referral definition and traffic classification.[2] ## What is confirmed—and what is not? ### Confirmed - Cloudflare’s 17 November 2026 Radar snapshot reports platform-level crawl-to-observed-referral ratios of **3,351.73:1 for Perplexity, 2,073.96:1 for OpenAI and 456.23:1 for Anthropic**.[1] - The snapshot covers the dashboard’s **last seven days**, not a full-November average.[7] - OpenAI and Anthropic platform totals aggregate multiple user agents, so the ratios cannot be assigned to GPTBot or ClaudeBot alone.[4] - Fastly measured approximately **30% AI-request growth** from January through May 2026, with AI traffic growing about **6.5 times faster than human traffic**.[11][10] - Cloudflare’s Pay Per Crawl documentation still describes the product as **closed beta** in documentation updated **28 July 2026**.[15] - Cloudflare’s access-policy documentation updated **2 November 2026** lists **Allow** as the new-domain default for Search, Agents and Training.[16][13] ### Not confirmed by the cited evidence - Absolute November 2026 request totals for GPTBot or ClaudeBot are not published in the cited Cloudflare dashboard responses.[1] - The cited sources do not publish a universal AI-crawler-to-revenue ratio.[17] - Pay Per Crawl’s cited documentation does not publish aggregate paid-request volume, publisher payouts or the number of paying AI operators.[8][1] - The cited material does not provide a general-availability date for Pay Per Crawl.[17][15] - Fastly’s reported Claude growth is not identified as ClaudeBot-only traffic.[10] - Akamai’s 2026 commerce findings cover **July through December 2025**, while the 2026 Thales/Imperva report analyses **full-year 2025**.[18] ## How are pay-per-crawl and blocking changing? Cloudflare introduced Pay Per Crawl in **private beta on 1 July 2025**. The system allows site owners to select crawlers to charge, set prices, manage payments through Stripe and monitor successful content deliveries.[19] The documentation updated on **28 July 2026** describes a payment negotiation using HTTP headers. A crawler can present payment intent, or receive an **HTTP 402 Payment Required** response containing pricing information; successful payment can permit an **HTTP 200** response.[15] The system charges again when the same page is crawled again, while errors are not billed. Paths such as `/robots.txt`, `/sitemap.xml`, `/security.txt`, `/.well-known/security.txt` and `/crawlers.json` remain free.[20] Cloudflare expanded the configuration on **16 June 2026** with URI-pattern exclusions and dynamic pricing. Publishers can leave selected pages free while charging for others, and can set prices through an origin response header or a Cloudflare Worker.[21] This is a documented technical mechanism, not evidence that the AI-crawling market has broadly adopted paid access. The cited product material does not quantify transactions, payments or publisher income. Blocking controls have also changed over time. Cloudflare’s **1 July 2025** announcement introduced permission-based onboarding for new domains and said more than one million customers had already selected an earlier one-click blocking option.[22] A later Cloudflare announcement dated **1 July 2026** described Search, Agent and Training controls for all customers and stated that new domains were scheduled to block Training and Agent traffic on ad-displaying pages from 15 September while allowing Search.[23] However, the Cloudflare documentation updated on **2 November 2026** lists **Allow** as the new-domain default for Search, Agents and Training.[16][13] The cited documentation does not explain the difference between the announcement and the later setting. Current controls distinguish **Block**, **Block on pages with ads**, **Allow** and **Disallow**. Cloudflare describes Disallow as a Training-only instruction conveyed through `robots.txt`, which remains voluntary unless additional edge enforcement is enabled.[16] With Bot Preference Sync enabled, Cloudflare can enforce Disallow at the edge for training crawlers, but mixed-purpose Search-and-Training crawlers are excluded from that edge block.[16] Cloudflare separately reported in September 2026 that fewer than **1% of its sites blocked search crawlers**, compared with **17% blocking training**.[8] ## What do other 2026 measurements add? Fastly’s January-May 2026 analysis found that AI requests were composed of approximately **85% crawlers and 15% fetchers** in May.[10][11] Fastly defines fetchers as traffic associated with user-directed requests, while crawlers represent more scheduled or automated activity.[10][11] Fastly also reported Claude-related traffic at more than **555% above its January baseline** during the analysis period.[11] The source calls this “Claude traffic” and does not establish that the figure represents ClaudeBot alone.[4][3] A separate Fastly investigation examined **23 September 2026** logs and found approximately **6.3 million requests from nearly 10,200 IP addresses** falsely claiming AI-crawler or fetcher identities.[24] The finding shows why a user-agent string alone cannot establish that a request came from the claimed operator.[1][24] Akamai’s publishing and commerce measurements cover a different population. Its publishing report uses application-layer traffic observed through Akamai bot-management tools during **1 July-31 December 2025**.[17][18] Akamai reported that commerce represented approximately **48% of AI-bot activity** and media represented **13%** in the relevant dataset.[17] Its July 2026 commerce announcement gave a more precise commerce figure of **47.9%** and said training crawlers accounted for more than **70% of AI-bot triggers in commerce** during that period.[18] Imperva’s 2026 Thales Bad Bot Report analyses **full-year 2025** activity and reported that bots made up more than **53% of web traffic** while AI-driven bot attacks increased **12.5 times**.[25][9] The attack statistic should not be presented as legitimate GPTBot or ClaudeBot growth.[6] ## When should publishers use these measurements? Publishers should match the metric to the decision: - Use Cloudflare’s ratio when comparing observed HTML crawling with observed platform referrals, while stating the seven-day window and referral-header limitation.[4] - Use Fastly’s figures when discussing AI-request growth, origin exposure, crawler-versus-fetcher composition or impersonated bot traffic.[24][6] - Use Akamai’s figures for sector-specific AI-bot exposure, not for all-web traffic or crawl-to-referral rankings.[18] - Use Imperva’s figures for automation and bad-bot context, not as a measure of useful crawler demand.[6] - Use Cloudflare’s policy documentation when describing current blocking controls and Pay Per Crawl’s documentation when describing the closed-beta payment mechanism.[9] - Use the academic study when discussing user click behaviour and possible search substitution, not when calculating crawler efficiency.[2] The central distinction is between **requests, crawls, fetches, referrals, clicks, visits and revenue**. The cited evidence measures different points in that chain and does not justify collapsing them into one “AI traffic” number.[12][7] ## What is the background to GPTBot and ClaudeBot traffic? An AI crawler may retrieve a page for several purposes, including search discovery, answer generation, user-directed fetching or model training.[8] Operators can also use multiple user agents, which means a platform-level measurement may include more than one bot identity.[4] Cloudflare’s current ratio groups OpenAI-associated bots and Anthropic-associated bots at the platform level.[7][23] That is why a statement such as “GPTBot crawls 2,074 pages for every visitor” would overstate what the data proves: the published ratio applies to the OpenAI group and to observed referrals under Cloudflare’s methodology.[4] Referral headers also do not capture every user action. Cloudflare notes that Claude’s native application does not include the relevant `Referer` header, and similar omissions may occur elsewhere.[4] The resulting ratio can therefore overstate the difference between crawling and referrals by an unknown amount.[4] Background: what does “crawl-to-referral ratio” mean? A crawl-to-referral ratio compares qualifying HTML crawl requests with HTML requests carrying a platform-associated referral hostname.[4][3] For example, a ratio of 2,073.96:1 means Cloudflare observed approximately 2,073.96 qualifying OpenAI-associated crawl requests for each qualifying OpenAI-associated referral in the stated measurement window.[4][3] It does not mean that one platform fetched exactly that many unique pages, that every request reached a publisher’s origin, or that the platform generated no visits outside the observed referral-header method.[24][4] It also does not measure revenue or the value of a visit.[2] ## Timeline | Date | Event | What it establishes | |---|---|---| | 1 July 2025 | Cloudflare announces Pay Per Crawl in private beta.[19] | A payment mechanism for selected crawler access was introduced; it was not evidence of market-wide adoption.[12][2] | | July 2025 | Cloudflare publishes historical crawl-to-click data.[1] | Anthropic, OpenAI and Perplexity showed materially different monthly ratios during 2025.[1] | | 1 July 2025 | Cloudflare announces permission-based AI-crawler onboarding for new domains.[22][2] | The announcement concerned new-domain onboarding and did not establish automatic blocking for every existing site.[2] | | 16 June 2026 | Cloudflare adds advanced Pay Per Crawl configuration.[21] | URI exclusions and dynamic pricing become documented capabilities.[21] | | 9 June 2026 | Fastly reports January-May AI-request growth.[11][10] | AI requests grew approximately 30%, around 6.5 times faster than human traffic.[11] | | 28 July 2026 | Cloudflare Pay Per Crawl documentation is updated.[15] | The service is still described as closed beta.[15] | | 23 September 2026 | Fastly measures false AI-bot identities in WAF logs.[24] | User-agent claims alone cannot authenticate crawler origin.[24] | | 30 September 2026 | Cloudflare reports agent-request and crawler-purpose data.[12] | Daily AI-agent requests had grown more than 1,700% year over year; 52% of June 2026 crawler requests were classified as training.[8] | | 2 November 2026 | Cloudflare access-policy documentation is updated.[5][13] | New-domain defaults are listed as Allow for Search, Agents and Training.[13] | | 17 November 2026 | Cloudflare Radar snapshot is retrieved.[7] | The last-seven-day platform ratios are 3,351.73:1 for Perplexity, 2,073.96:1 for OpenAI and 456.23:1 for Anthropic.[1] | ## FAQ ### What is the AI crawler traffic ratio in 2026? In Cloudflare Radar’s 17 November 2026 last-seven-day snapshot, Perplexity recorded **3,351.73 crawls per observed referral**, OpenAI recorded **2,073.96**, and Anthropic recorded **456.23**.[1] These are platform-level ratios based on Cloudflare’s HTML-request and referral-header methodology, not universal measures of visits or revenue.[1] ### What is the current GPTBot crawl ratio? The cited November 2026 Cloudflare data does not publish a GPTBot-only ratio.[1][8] It reports an OpenAI platform-level ratio of **2,073.96 crawls per observed referral**, aggregating OpenAI-associated bots rather than isolating GPTBot.[3][1] ### Does ClaudeBot send less traffic than GPTBot? The available data does not support that direct conclusion. Cloudflare’s chart displays ClaudeBot at **12.1%** and GPTBot at **8.1%** of an unspecified chart denominator, while its platform-level crawl-to-referral ratio is lower for Anthropic than OpenAI.[4][5] Those measures use different groupings and denominators.[12] ### Is Cloudflare Pay Per Crawl generally available? No. Cloudflare documentation updated on **28 July 2026** still describes Pay Per Crawl as closed beta.[15] The cited documentation does not provide a general-availability date or aggregate data on paid requests and publisher payouts.[12][2] ### Does Cloudflare block AI crawlers by default in 2026? The answer depends on the date and control being described. Cloudflare’s documentation updated **2 November 2026** lists Allow as the new-domain default for Search, Agents and Training, while earlier 2025 and 2026 announcements described permission-based or planned blocking policies.[13][16] The dated documentation should be cited instead of using an undated claim that Cloudflare blocks all AI crawlers by default. ## Sources [1] The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals — https://blog.cloudflare.com/crawlers-click-ai-bots-training [2] Answering Without Referring: How AI Search Rewrites the Web’s Economic Bargain Qiaoni Shi, Kai Zhu, and Kai Gu Bocconi University, Milan, Italy — https://arxiv.org/html/2607.07652v1 [3] The 2025 Cloudflare Radar Year in Review- the rise of AI, post-quantum, and record-breaking DDoS attacks — https://blog.cloudflare.com/radar-2025-year-in-review [4] The crawl before the fall… of referrals: understanding AI’s impact on content providers — https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radar [5] AI Insights | Cloudflare Radar — https://radar.cloudflare.com/ai-insights [6] BAD BOT REPORT — https://www.imperva.com/resources/reports/2026-Thales-Bad-Bot-Report.pdf [7] AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence — https://arxiv.org/html/2608.18352 [8] The Internet has a second audience — https://blog.cloudflare.com/agentic-web [9] AI Bot Traffic: Which Bots to Allow or Block | Imperva — https://www.imperva.com/blog/ai-bot-traffic-which-bots-to-trust/ [10] AI Traffic Growth: Bots Outpace Humans | Fastly — https://www.fastly.com/blog/ai-traffic-grew-6-5x-faster-than-human-traffic-this-year [11] AI Traffic Grew 6.5x Faster Than Human Traffic This Year, Creating New Business Challenges and Opportunities - Tue, 06/09/2026 - 08:00 — https://investors.fastly.com/news-releases/news-release-details/ai-traffic-grew-65x-faster-human-traffic-year-creating-new [12] The Impact of LLMs on Online News Consumption and Production — https://arxiv.org/html/2512.24968v2 [13] From Googlebot to GPTBot: Who’s crawling your site in 2025 — https://blog.cloudflare.com/from-googlebot-to-gptbot-whos-crawling-your-site-in-2025 [14] Akamai Research: AI Bots Threaten the Foundation of Web-Based Business Models | Akamai — https://www.akamai.com/newsroom/press-release/akamai-research-ai-bots-threaten-foundation-of-web-based-business-models [15] What is Pay Per Crawl? · Cloudflare AI Crawl Control docs — https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/what-is-pay-per-crawl/?ref=technikatsu.com [16] AI Crawler and Agent access policies — https://developers.cloudflare.com/bots/additional-configurations/block-ai-bots [17] AI Bot Defense and Monetization for Publishers | Akamai — https://www.akamai.com/site/en/documents/state-of-the-internet/2026/publishing-ai-botnet-report.pdf [18] Akamai Research: Commerce Becomes the Epicenter for AI Bot Attacks and Agentic Fraud in 2026 — https://www.akamai.com/newsroom/press-release/akamai-research-commerce-becomes-the-epicenter-for-ai-bot-attacks-and-agentic-fraud-in-2026 [19] Introducing Pay Per Crawl (private beta) — https://developers.cloudflare.com/changelog/post/2025-07-01-pay-per-crawl [20] Pay Per Crawl FAQ — https://developers.cloudflare.com/ai-crawl-control/features/pay-per-crawl/faq [21] Pay Per Crawl advanced configuration — https://developers.cloudflare.com/changelog/post/2026-06-16-pay-per-crawl-advanced-configuration [22] Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model | Cloudflare — https://www.cloudflare.com/press/press-releases/2025/cloudflare-just-changed-how-ai-crawlers-scrape-the-internet-at-large [23] New options to manage AI traffic — https://developers.cloudflare.com/changelog/post/2026-07-01-ai-traffic-options?trk=public_post_comment-text [24] AI Bot Cosplay | Fastly — https://www.fastly.com/blog/ai-bot-cosplay [25] AI-driven Bot Attacks Surged 12.5x According to Thales Bad Bot Report - Company — https://www.imperva.com/company/press_releases/ai-driven-bot-attacks-surged-12-5x-according-to-thales-bad-bot-report/ [26] Your site, your rules: new AI traffic options for all customers — https://blog.cloudflare.com/content-independence-day-ai-options --- # What is query fan-out in AI search? > Query fan-out turns one request into related searches across subtopics or sources; Google says AI Mode may run those searches concurrently. Source: https://llmsearchnews.com/query-fan-out-explained-2026 · Publisher: LLM Search News · Published: 2026-11-17 · Updated: 2026-11-17 ## TL;DR - Query fan-out turns one request into related searches across subtopics or sources; Google says AI Mode may run those searches concurrently.[2] - Google’s public documentation does not say that every AI Mode query fans out or specify a fixed number of searches.[6] - ChatGPT documents targeted query rewriting and result-informed follow-up searches, but not guaranteed simultaneous execution for every search.[18] - Claude’s web-search API supports repeated searches, while Anthropic’s separate Research system uses parallel subagents followed by synthesis.[3][10] - Query rewriting, decomposition, iterative retrieval and rank fusion are related but distinct techniques, with research documenting benefits as well as noise and compute trade-offs.[1] As of 17 November 2026, **query fan-out** means an AI search system turns one user request into multiple related searches, often covering different subtopics or sources, and combines the retrieved evidence into one answer. Google explicitly documents this behavior for AI Overviews and AI Mode, while noting that it may be used rather than applied to every query.[1] The simplest model is: **one question → related subqueries → searches and retrieved evidence → synthesized response** Google says AI Mode can issue related searches concurrently across subtopics and data sources.[2] That is narrower than saying every AI search product uses the same process: ChatGPT documents query rewriting and result-informed follow-up searches, while Anthropic documents repeated searches and separately describes parallel subagents in its Research system.[3] ## What happened? Google publicly described query fan-out as part of its AI search architecture in 2025. Its documentation says AI Overviews and AI Mode may issue multiple searches across related subtopics and data sources before generating an answer.[2][4] Google’s AI Mode description adds that a custom version of Gemini can issue several related searches **concurrently** and bring the results together.[2] Google’s May 2025 announcement similarly described AI Mode as breaking a question into subtopics and issuing many queries simultaneously.[5] This is not the same as a conventional search engine sending one unchanged query to an index. Instead, the system may identify several information needs inside the request. For example, a question about choosing a camera for travel could require searches about weight, battery life, weather resistance, image quality and current pricing. Google’s public documentation establishes the general fan-out pattern, but does not disclose the exact internal query-generation process for each request.[6][3] Google also describes **Deep Search** as using the same broad fan-out technique at greater scale. Its announcement says Deep Search can issue hundreds of searches and produce a cited report.[5] That statement applies to Deep Search as described by Google, not as a published search limit for ordinary AI Mode queries.[6] ### Query fan-out is not one fixed algorithm The phrase describes an orchestration pattern rather than a single universally defined implementation. A system can branch into related queries, run them in parallel, execute them sequentially, or combine both approaches. The following terms describe different operations: - **Query rewriting** reformulates the original request into a search query better suited to a retriever.[7][8] - **Query decomposition** splits a complex request into subqueries.[7] - **Fan-out** launches related searches from one request, commonly across multiple branches.[9] - **Iterative retrieval** uses earlier search results to decide what to retrieve next.[7][10] - **Rank fusion** combines result rankings from multiple searches.[11] These operations can appear in the same system, but they should not be treated as synonyms. A rewritten query does not necessarily indicate parallel fan-out, and multiple searches do not necessarily mean that the system decomposed the user’s question into independent subtopics.[7] ## Why does query fan-out matter? Query fan-out changes how AI search systems discover, select and cite information. Instead of evaluating only pages relevant to the original wording, the system can retrieve evidence for several related information needs before composing its response.[7][12] That has four practical effects. ### 1. A single prompt can create many visibility opportunities A brand or publisher may be relevant to only one branch of a broader question. If an AI system searches separately for comparisons, product specifications, reviews or use cases, different pages can become candidates for citation in different parts of the final answer.[6] This means visibility cannot be evaluated only against the exact wording a user typed. The system may generate related searches that are not visible in the original prompt.[8][9] ### 2. Retrieval can cover more aspects of complex questions Research systems have studied several ways to improve retrieval for multi-step questions. IRCoT interleaves retrieval with reasoning so that information found earlier can guide later retrieval, and its authors reported gains across four datasets.[13] Adaptive-RAG selects between no retrieval, single-step retrieval and iterative retrieval according to estimated question complexity.[14] The motivation is to avoid unnecessary retrieval for simple questions while giving more complex requests additional search steps.[1][7] ### 3. More searches create trade-offs Additional branches can improve coverage, but they also add computation and create more opportunities for irrelevant or conflicting results. Research on query decomposition describes trade-offs involving coverage, noise, compute and retriever quality.[7] RAG-Fusion, which generates multiple queries and combines results with reciprocal rank fusion, also identifies a query-drift risk when generated queries are not sufficiently relevant to the original question.[11] ### 4. Source visibility and citation visibility can diverge OpenAI’s API documentation distinguishes between sources consulted by a search process and sources explicitly cited in the final answer. The `sources` field can contain more URLs than the answer cites.[6] That distinction matters when analysing AI search. A page may influence an answer without appearing as a visible citation, while a cited page may represent only one part of a broader retrieval process.[10] ## What do Google’s patents tell us? Google patents provide useful technical context, but they do not prove how AI Mode is implemented. **US11769017B1, “Generative summaries for search results,”** describes using a search result responsive to a related query, where that related query is associated with the original query.[8] The patent does not explicitly identify AI Mode or call the process query fan-out.[4][1] **US11663201B2, “Generating query variants using a trained generative model,”** describes generating at least one query variant from the original query’s tokens and contextual attributes.[12] It also describes using search-response features to determine whether variants should be generated.[12] Neither patent establishes that the generated variants run concurrently, nor that AI Mode uses the patented workflow.[9] Google’s first-party AI Mode documentation is therefore the stronger source for Google’s public description of fan-out.[6] ## How does Google AI Mode fan out searches? Google’s public description has three relevant parts: 1. AI Mode may identify related subtopics within a user request.[4] 2. It may issue related searches across those subtopics and data sources.[4][2] 3. Those searches can run concurrently before the results are brought together into an answer.[2] Google also describes follow-up questions that preserve conversational context and refine the user’s intent.[2] A follow-up question is different from initial fan-out: fan-out branches from the current request, while a follow-up occurs after an earlier turn or result has informed the next request.[10] Google does not publish a fixed number of searches for an ordinary AI Mode query.[4][5] It also does not publish the complete query-generation, retrieval-selection or ranking algorithm used for each answer.[1] ### A dated timeline | Date | Development | What the source establishes | |---|---|---| | 2023 | Rewrite-Retrieve-Read research published | A trainable language model can rewrite queries for a frozen retriever and reader; the study used web search and reader feedback.[15] | | 2023 | IRCoT research published | Retrieval can be interleaved with reasoning for multi-step questions, allowing earlier information to guide later retrieval.[7] | | 2024 | Adaptive-RAG research published | A system can choose among no retrieval, one retrieval step and iterative retrieval based on estimated question complexity.[14][16] | | February 2024 | RAG-Fusion research published | Multiple generated queries can be combined with reciprocal rank fusion, with a documented risk of query drift.[11] | | May 2025 | Google described AI Mode updates | Google said AI Mode breaks questions into subtopics and issues multiple queries simultaneously.[5] | | May 2025 | Google published its AI Overviews and AI Mode explanation | Google described related searches across subtopics and sources, including concurrent execution and contextual follow-ups.[12] | | 2025 | Anthropic introduced web search for its API | Claude can search, inspect results and conduct additional progressive searches informed by earlier results.[17] | | 2026 | Query decomposition research studied retrieval budgets | A bandit policy selected subqueries and documents sequentially while balancing precision, coverage, noise and compute.[7] | | November 2026 | Public evidence remains implementation-specific | Google documents concurrent fan-out for AI Mode; ChatGPT and Claude document related but not identical search workflows.[9] | Background: why one question becomes several searches A natural-language question often contains multiple information needs. A request for the “best laptop for video editing under a particular budget,” for example, may require evidence about price, processor performance, memory, display quality, software compatibility and current availability. A search system can treat those needs as one query, rewrite the question, decompose it into subqueries, or combine several of these steps. Research papers use different names for these operations. Rewrite-Retrieve-Read focuses on adapting a query before retrieval, while IRCoT uses retrieved information to guide later reasoning and searches.[13][6] RAG-Fusion generates multiple queries and combines their rankings, whereas Adaptive-RAG chooses a retrieval strategy according to question complexity.[14] The commercial systems described publicly do not expose all of their internal steps. Google documents concurrent related searches for AI Mode, OpenAI documents targeted rewrites and additional searches for ChatGPT, and Anthropic documents repeated web-search calls for Claude.[18] Those descriptions should be kept separate from research prototypes and patents. ## What does ChatGPT do when it searches? OpenAI’s consumer documentation says ChatGPT Search may activate automatically when a question benefits from current information, or a user may trigger search manually.[18] When search providers are used, ChatGPT typically rewrites the prompt into one or more targeted queries.[18] After reviewing initial results, it may issue additional, more specific queries to other providers.[18] This is evidence of query rewriting and result-informed follow-up search. The cited documentation does not promise that ChatGPT runs every query concurrently.[6][10] OpenAI’s API documentation describes a separate tool interface. A reasoning model can use search, open a page and find information within a page, with `tool_choice: "auto"` allowing search when appropriate.[6] The documentation says search calls usually include the queries used and that the model can refine searches while reading results.[6] For technical analysis, the safest description is therefore: - ChatGPT can rewrite a user prompt into targeted searches.[18] - It can issue additional searches after reviewing initial results.[18] - Its API exposes distinct search, page-opening and in-page-find actions.[6] - Public documentation does not establish that all of these searches execute concurrently.[6] ## What does Claude do when it searches? Anthropic’s web-search API documentation says Claude can decide whether to search, send a query through the web-search tool, receive results and potentially repeat the process several times within one request.[10] Anthropic’s API announcement describes a targeted query followed by result analysis and multiple progressive searches informed by earlier results.[17][10] This is an iterative search pattern, not proof of concurrent query fan-out.[9][3] Anthropic separately describes **Research**, a multi-agent system in which a lead agent develops a strategy and launches specialised subagents to investigate different aspects simultaneously.[3] Those subagents return findings, after which the lead agent synthesises the material and may decide whether more research is needed.[3] Anthropic reported that Research outperformed a single-agent Claude Opus 4 baseline by 90.2% on its internal research evaluation and used about 15 times as many tokens as ordinary chats.[3] Those figures apply to the described Research system and evaluation, not to every Claude web-search request.[7] The distinction is essential: - Claude’s web-search API documents repeated searches within one request.[6][10] - Anthropic’s Research system documents parallel subagents and synthesis.[3] - The Research architecture should not be generalised to every ordinary Claude search.[3] ## What is confirmed, and what is not? ### Confirmed by public documentation - Google says AI Overviews and AI Mode may use query fan-out.[4][2] - Google describes AI Mode as issuing related searches across subtopics and sources.[4][2] - Google says those related searches can run concurrently.[9][2] - Google documents contextual follow-up questions in AI Mode.[5][2] - ChatGPT can rewrite prompts into targeted queries and issue additional searches after reviewing results.[18] - OpenAI’s API distinguishes search, page-opening and in-page-find actions.[6] - Claude’s web-search tool can be called repeatedly within one request.[6][19] - Anthropic’s separate Research system uses parallel subagents followed by synthesis.[3] ### Not confirmed by the cited sources - That every AI Mode request uses fan-out.[5][2] - A fixed number of searches for an ordinary AI Mode query.[6] - That ChatGPT runs all rewritten queries concurrently.[18] - That every Claude web-search request uses Anthropic’s Research architecture.[17][10] - That Google’s cited patents are the implementation behind AI Mode.[9][6] - That a page consulted during retrieval will necessarily appear as a final citation.[6] The evidence supports careful, system-specific language. “Google documents concurrent fan-out in AI Mode” is supported. “All AI search engines always fan out queries in parallel” is not.[9] ## How should query fan-out be measured? A useful analysis should distinguish the original user prompt from the searches generated afterward. It should also separate retrieved sources from sources shown in the final answer, because those sets may differ.[12] For a technical evaluation, record: 1. The user’s original request. 2. Any visible rewritten or generated queries. 3. Whether searches were documented or observed as concurrent or sequential. 4. The subtopics covered by each query. 5. The pages retrieved for each branch. 6. The sources cited in the final answer. 7. Whether the system issued follow-up searches after reading results. 8. The cost, latency or search limits where the documentation publishes them. This method avoids treating every multi-query workflow as identical. It also makes room for systems that combine rewriting, decomposition, iterative retrieval and rank fusion.[7] For publishers and brands, the practical implication is that one page should answer a defined information need clearly and extractably, while a broader topic may require coverage of several related questions. That is an implication of the documented retrieval mechanisms, not evidence that any particular page will be selected or cited.[12] ## FAQ ### Is query fan-out the same as query rewriting? No. Query rewriting changes the wording of a request for retrieval, while fan-out describes branching one request into multiple related searches.[9] A system can rewrite one query without launching several searches, or it can generate several rewritten queries as part of a fan-out workflow.[6][12] ### Does Google AI Mode always use query fan-out? Google says AI Overviews and AI Mode **may** use query fan-out, so the public documentation does not establish that every request follows that path.[6] Google does describe AI Mode issuing related searches concurrently, but it does not publish a fixed search count or universal execution rule.[12][2] ### Does ChatGPT search the web with multiple queries? OpenAI says ChatGPT typically rewrites a prompt into one or more targeted queries and may issue additional, more specific queries after reviewing initial results.[18] The cited documentation does not establish that every such query runs simultaneously.[6][10] ### Does Claude use parallel search agents for every question? No such general claim is supported by the cited documentation. Anthropic documents repeated searches through its web-search tool, while parallel specialised subagents are described for the separate Research system.[3] ### Can query fan-out make AI answers better? It can improve coverage for complex questions, but additional retrieval also introduces trade-offs involving noise, computation, query drift and retriever quality.[1][7] Research results apply to the studied systems and datasets, not as a guarantee for every commercial AI search answer.[6][7] ## Sources [1] Question Decomposition for Retrieval-Augmented Generation — https://arxiv.org/html/2507.00355v1 [2] AI Overviews and AI Mode in Search — https://search.google/pdf/google-about-AI-overviews-AI-Mode.pdf [3] How we built our multi-agent research system \ Anthropic — https://www.anthropic.com/engineering/multi-agent-research-system [4] AI Features and Your Website | Google Search Central | Documentation | Google for Developers — https://developers.google.com/search/docs/appearance/ai-features [5] AI Mode in Google Search: Updates from Google I/O 2025 — https://blog.google/products-and-platforms/products/search/google-search-ai-mode-update [6] Web search — https://developers.openai.com/api/docs/guides/tools-web-search [7] Query Decomposition for RAG: Balancing Exploration-Exploitation — https://aclanthology.org/2026.eacl-long.322.pdf [8] US11769017B1 - Generative summaries for search results - Google Patents — https://patents.google.com/patent/US11769017B1/en [9] US20260187080A1 - Generative search results documents based on enhanced search results using generative artificial intelligence (ai) models - Google Patents — https://patents.google.com/patent/US20260187080A1/en [10] Web search tool — https://platform.claude.com/docs/en/agents-and-tools/tool-use/web-search-tool [11] [2402.03367] RAG-Fusion: a New Take on Retrieval-Augmented Generation — https://arxiv.org/abs/2402.03367 [12] US11663201B2 - Generating query variants using a trained generative model — https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/11663201 [13] Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions - ACL Anthology — https://aclanthology.org/2023.acl-long.557 [14] [2403.14403] Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity — https://arxiv.org/abs/2403.14403 [15] Query Rewriting in Retrieval-Augmented Large Language Models - ACL Anthology — https://aclanthology.org/2023.emnlp-main.322 [16] Query Rewriting for Retrieval-Augmented Large Language Models — https://aclanthology.org/2023.emnlp-main.322.pdf [17] Introducing web search on the Anthropic API | Claude by Anthropic — https://claude.com/blog/web-search-api [18] Searching the web with ChatGPT | OpenAI Help Center — https://help.openai.com/en/articles/9237897-chatgpt-search [19] Enable and use web search | Claude Help Center — https://support.anthropic.com/en/articles/10684626-enable-and-use-web-search --- # Which model releases changed AI search in late 2026? > Google’s 12 November 2026 rollout of Gemini 4 Hydrogen into Google Search is the clearest documented change to consumer AI search in late 2026. Source: https://llmsearchnews.com/model-releases-search-november-2026 · Publisher: LLM Search News · Published: 2026-11-17 · Updated: 2026-11-17 ## TL;DR - Google’s 12 November 2026 Gemini 4 Hydrogen rollout into Google Search is the clearest documented consumer AI-search change of late 2026.[28][12] - Google says Hydrogen improves complex-question handling, simultaneous query fan-out, links, tables and images in Search.[1][26] - OpenAI, Google, Anthropic, xAI and Mistral document web access through separate search, grounding, fetch or connector tools rather than proving that every new model browses automatically.[2][25] - Late-2026 releases from OpenAI, Google, Anthropic and Meta largely emphasized reasoning, coding, voice, computer use or agentic workflows.[6] - Mind2Web 2 offers a benchmark for long-horizon browser-agent evaluation, but it predates the late-2026 models and does not measure their performance.[2][17] Google’s 12 November 2026 rollout of Gemini 4 Hydrogen into Google Search is the clearest documented change to consumer AI search in late 2026. Google says the integration handles complex, multi-part questions, performs simultaneous query fan-out, and presents answers with structured information, images and links.[1] Other releases from OpenAI, Google, Anthropic, Meta, xAI and Mistral expanded reasoning, coding, agentic workflows or tool use between September and November 2026. However, the vendor announcements distinguish those model capabilities from optional search, browsing, web-fetching or computer-use tools; a new model should not automatically be described as a natively browsing model.[2][3] ## What happened? Late-2026 model news followed three distinct tracks: - **Search-product integration:** Google put Gemini 4 Hydrogen directly into Google Search worldwide on 12 November 2026. The update targets complex questions, intent understanding, simultaneous queries, richer links, tables and images.[1] - **API-level web access:** OpenAI, Google, Anthropic, xAI and Mistral documented separate tools or connectors that let models search the live web or ground answers in current information.[4][5] - **Agentic model releases:** Many model announcements focused on multi-step work, coding, computer use, voice, cybersecurity or tool calling rather than a new consumer search interface.[6] The most consequential release for ordinary search users was therefore not simply a new model benchmark. It was the combination of Gemini 4 Hydrogen and Google Search’s existing retrieval and query-fan-out systems.[7] ## Why does it matter? The Google change makes the model part of the search experience rather than leaving it only inside an API or chatbot. Google says Hydrogen can interpret a user’s intent, break complicated questions into simultaneous queries, and return broader links alongside visual and structured answers.[1] That affects how information is discovered. A page may be surfaced because it answers one of several related subqueries generated from the original request, not only because it matches the exact wording typed by the user. Google describes this as an upgrade to its query-fan-out technique and says it can connect users with sites they might not otherwise have found.[8] The other vendors’ releases matter mainly for developers building AI search and browser agents. OpenAI’s web-search tool gives models current internet information and sourced citations in the Responses API.[9] Google’s Search grounding connects Gemini to real-time web content and returns URL or citation annotations.[10] Anthropic documents web search and web fetch as separate tools, while xAI and Mistral document their own web-search tools or connectors.[11][6] That distinction is significant for visibility work. A model may be capable of reasoning about a web result, while the surrounding product decides whether to search, which queries to issue, which pages to retrieve and which sources to cite. The model release alone does not establish how a particular search surface ranks or cites websites. ## What do we know—and what is not confirmed? ### What is documented - Gemini 4 Hydrogen was announced and made available across Google’s developer, enterprise and consumer surfaces on 12 November 2026.[12] - Google Search received Gemini 4 Hydrogen globally on the same date.[1] - Google documented multi-part question handling, simultaneous queries, broader links, tables and images as part of the Search upgrade.[1] - Gemini’s API documentation describes Google Search grounding as a separate capability that can connect responses to current web information and citations.[10] - OpenAI documents web search and computer use as separate API tools.[6][2] - Anthropic documents web search, web fetch and computer use as separate tools.[6] - xAI’s developer documentation lists web search, X Search and code execution as distinct tools.[13] - Mistral documents a web-search connector separately from its Large 4 model announcement.[11] ### What is not confirmed by these releases The reviewed announcements do not establish that every late-2026 model automatically browses the web. OpenAI’s GPT-6 Astra, GPT-6 Sol, GPT-6 Luna, GPT-6.1 Sol and GPT-6.1 Terra announcements emphasize general intelligence, coding, computer use or professional work without documenting a new built-in search feature.[14] Google’s Gemini 4 Argon and Gemini 4 Carbon announcements emphasize cybersecurity and agentic workflows, respectively, rather than presenting either release as a standalone web-search product.[12][7] Anthropic’s Opus 5.5, Sonnet 5.5 and Fable/Mythos 5.2 announcements likewise focus on model capability and autonomous work; the separate Claude documentation explains how web search can be enabled.[15] Meta’s Muse releases describe agents, tool use, coding and long-horizon execution, but the cited release materials do not name web search or browser access.[6] Mistral Large 4 describes tool use and information gathering, while the web-search connector is documented separately.[11] No source reviewed here provides a late-2026, apples-to-apples measurement showing that one newly released model produces better search visibility, citation rates or browser-agent accuracy than all competitors. The Mind2Web 2 benchmark provides a framework for evaluating long-horizon browsing and information-synthesis tasks, but it predates the releases discussed here and does not measure their performance.[16][17] ## Late-2026 model and search timeline | Model or search surface | Company | Release date | Search or browsing capability documented | |---|---|---:|---| | Muse Spark 1.3 | Meta | 2 September 2026 | Tool use and context building are described; web search is not named.[18][6] | | GPT-6 Astra | OpenAI | 3 September 2026 | The announcement focuses on computer use, coding, cybersecurity and science; no new search feature is documented.[2][3] | | Gemini 3.8 Live | Google DeepMind | 15 September 2026 | Voice interaction and task execution are described; web search is not specified.[19] | | GPT-6 Sol and GPT-6 Luna | OpenAI | 22 September 2026 | Different capability and cost balances are described; search is not specified.[2] | | Claude Opus 5.5 | Anthropic | 22 September 2026 | Model capabilities are described; web search is available through a separate tool.[6] | | Grok 4.7 | xAI | 21 September 2026 | The release emphasizes coding and knowledge work; xAI separately documents web-search tools.[13] | | Claude Sonnet 5.5 | Anthropic | 28 September 2026 | Speed, cost, long-horizon work and image understanding are emphasized; search is separately documented.[20] | | Gemini 4 Argon | Google DeepMind | 30 September 2026 | Cybersecurity and vulnerability discovery are emphasized; no search feature is specified.[7] | | GPT-6.1 Sol | OpenAI | 29 September 2026 | Coding, computer use and professional work are emphasized; search is not specified.[14][2] | | Mistral Large 4 preview | Mistral AI | 6 October 2026 | Agent and tool use are described; web search is separately available through a connector.[11] | | GPT-6 in ChatGPT | OpenAI | 7 October 2026 | A ChatGPT deployment update replacing earlier models; no new search feature is documented in the cited release material.[2][4] | | Claude Haiku 5.5 | Anthropic | 7 October 2026 | Browser use is cited as a suitable workload; a built-in web-search feature is not established by the release announcement.[8][15] | | Muse Wonder and Muse Wonder Full | Meta | 12 October 2026 | Agentic personal and professional work is described; web search is not named.[6] | | Muse Spark Open | Meta | 15 October 2026 | On-device agentic and coding work are emphasized; web search is not named.[21] | | GPT-6.1 Terra | OpenAI | 26 October 2026 | Visual reasoning and computer use are emphasized; search is not specified.[14] | | Mistral Large 4 weights | Mistral AI | 4 November 2026 | The announcement concerns the model weights, not a separate search feature.[11] | | Images 2.6 | OpenAI | 4 November 2026 | Image generation release; web search is not specified.[22] | | Gemini Omni 1.2 Flash | Google | 5 November 2026 | Video generation and editing are emphasized; web search is not specified.[23] | | Claude Fable 5.2 and Mythos 5.2 | Anthropic | 9 November 2026 | Autonomous task work and judgment are emphasized; search is separately documented.[24] | | Muse Ember | Meta | 9 November 2026 | Tool calls, coding subtasks, routing and document parsing are emphasized; web search is not named.[6] | | Gemini 4 Hydrogen | Google | 12 November 2026 | Rolled into Google Search globally; supports complex questions, simultaneous queries, links, tables and images.[1] | ## How do search, grounding and browsing differ? “Search capability” can describe several different technical layers: 1. **A search-product integration** places a model inside a consumer search interface. Gemini 4 Hydrogen in Google Search is the clearest late-2026 example documented in the reviewed sources.[1] 2. **Search grounding** lets an API call a search service and use retrieved material to produce a cited answer. Google documents this for Gemini through Google Search grounding.[11] 3. **A web-search tool** gives a model a structured function for issuing searches and receiving web results. OpenAI, Anthropic and xAI document this pattern in their API materials.[5] 4. **Web fetch** retrieves a specified page or resource after a URL is known. Anthropic documents web fetch separately from web search.[25] 5. **Computer use** allows an application-controlled model to interact with a browser or graphical interface. OpenAI, Google and Anthropic document computer-use workflows separately from search tools.[3] These layers can be combined, but they should not be treated as interchangeable. A model can support tool calls without having a built-in search interface, and a search product can use a model with retrieval systems that are not exposed as general-purpose browsing to developers. Background: what is query fan-out? Query fan-out is the practice of decomposing one user request into multiple related searches or subqueries. Google says the Gemini 4 Hydrogen Search upgrade improves this process by conducting simultaneous queries and finding a broader range of useful links.[26] For publishers, fan-out means that visibility may depend on answering the underlying subtopics implied by a question. The reviewed sources do not provide a public, independent measurement of how often Google generates particular fan-out queries, nor do they establish that every AI search product uses the same method. ## What should developers and search teams watch next? The late-2026 evidence suggests four practical areas of attention. - **Search-surface integration:** Google’s Hydrogen rollout is the most direct signal that model releases can change how consumer search answers are composed and displayed.[1] - **Tool selection:** OpenAI, Google, Anthropic, xAI and Mistral expose web access through tools or connectors, so applications can determine when current information is needed.[13][11] - **Citation behavior:** Search systems increasingly expose links, source information or citation annotations, but the cited material does not establish consistent citation rules across vendors.[7] - **Agent evaluation:** Browser agents need tests involving realistic, multi-step tasks rather than model-release claims alone. Mind2Web 2 is one published benchmark designed around long-horizon browsing and information synthesis.[17][27] The defensible conclusion is narrower than “all new models now browse.” Late-2026 releases improved agentic and tool-use capabilities across several vendors, while Google made the most explicit consumer-search change by integrating Gemini 4 Hydrogen into Google Search on 12 November 2026.[12][2] ## FAQ ### Which model release changed AI search the most in late 2026? Gemini 4 Hydrogen changed consumer AI search the most visibly because Google rolled it into Google Search globally on 12 November 2026.[12][28] Google documented complex-question handling, simultaneous query fan-out, broader links, tables and images as part of the update.[1][26] ### Does GPT-6 automatically browse the web? The cited late-2026 OpenAI release material does not establish that GPT-6 automatically browses the web.[4] OpenAI separately documents a web-search tool for API applications, so browsing depends on the product configuration and enabled tools rather than the model name alone.[2][3] ### Can Claude 5.5 search the internet? Claude’s web-search capability is documented as a separate feature that can be enabled for supported models, including Fable 5.2, Haiku 5.5, Sonnet 5.5 and Opus 5.5.[8] Anthropic’s platform documentation also separates web search, web fetch and computer use into distinct tools.[29][8] ### Is Gemini 4 Hydrogen a search engine? Gemini 4 Hydrogen is a model, while Google Search is the consumer search product into which Google integrated it.[12][30] The Search integration adds model-assisted handling of complex questions, simultaneous queries and structured or visual results; it does not make the model and the search product identical.[22][1] ### What is the difference between web search and computer use? Web search retrieves information through a search tool, while computer use allows an application-controlled model to interact with a graphical environment or browser.[2][31] OpenAI, Google and Anthropic document computer use separately from their search or grounding capabilities.[3][11] ## Sources [1] Google Search upgrade: Gemini 4 Hydrogen features — https://blog.google/products-and-platforms/products/search/gemini-4-hydrogen-search-upgrade/ [2] Introducing GPT-6.1 Sol | OpenAI — https://openai.com/index/introducing-gpt-6-1-sol/ [3] GPT-6 Astra: A new generation of intelligence | OpenAI — https://openai.com/index/gpt-6-astra/ [4] Release Notes | OpenAI | OpenAI — https://openai.com/products/release-notes [5] Web Search | SpaceXSI Docs — https://docs.x.ai/developers/tools/web-search [6] Introducing Our Smallest, Most Efficient Model: Muse Ember — https://research.meta.ai/blog/muse-ember [7] Introducing Gemini 4 Argon — https://deepmind.google/blog/gemini-4-argon-our-next-era-of-frontier-intelligence [8] Introducing Claude Haiku 5.5 \ Anthropic — https://www.anthropic.com/news/claude-haiku-5-5 [9] Web search — https://developers.openai.com/api/docs/guides/tools-web-search?api-mode=responses [10] Grounding with Google Search  |  Gemini API  |  Google AI for Developers — https://ai.google.dev/gemini-api/docs/interactions/google-search [11] Introducing Mistral Large 4 | Mistral — https://mistral.ai/news/mistral-large-4/ [12] Gemini 4 Hydrogen - Model Card — Google DeepMind — https://deepmind.google/models/model-cards/gemini-4-hydrogen/ [13] Release Notes | SpaceXSI Docs — https://docs.x.ai/developers/release-notes [14] Introducing GPT-6.1 Terra | OpenAI — https://openai.com/index/introducing-gpt-6-1-terra/ [15] Announcements \ Anthropic — https://www.anthropic.com/news/announcements [16] Mind2Web 2:Evaluating Agentic Search with Agent-as-a-Judge — https://arxiv.org/html/2506.21506 [17] Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge — https://proceedings.neurips.cc/paper_files/paper/2025/hash/fdcec9f5b99aa4fc8f4fb8487802d737-Abstract-Datasets_and_Benchmarks_Track.html [18] Introducing Muse Spark 1.3 | Meta AI Research — https://research.meta.ai/blog/introducing-muse-spark-1-3 [19] New Gemini Audio models for developers — https://blog.google/innovation-and-ai/technology/developers-tools/build-real-time-voice-applications-gemini-audio/ [20] Introducing Claude Sonnet 5.5 \ Anthropic — https://www.anthropic.com/claude-sonnet-5-5 [21] Introducing Muse Spark Open, The Most Capable US Open‑Weight Model to Date | Meta AI Research — https://research.meta.ai/blog/muse-spark-open [22] Introducing Images 2.6 | OpenAI — https://openai.com/index/introducing-images-2-6 [23] Gemini Omni 1.2 Flash: 15-second video and lower pricing — https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemini-omni-1-2-flash [24] Introducing Claude Fable 5.2 and Claude Mythos 5.2 \ Anthropic — https://www.anthropic.com/claude-fable-and-mythos-5-2 [25] Web search and web fetch — https://claude.com/docs/third-party/claude-desktop/web-tools [26] Introducing Gemini 4 Hydrogen for Agentic Workflows — https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-hydrogen/?utm_source=deepmind.google&utm_medium=referral&utm_campaign=gdm&utm_content= [27] NeurIPS Poster Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge — https://neurips.cc/virtual/2025/poster/121798 [28] Model cards — Google DeepMind — https://deepmind.google/models/model-cards [29] Web fetch tool - Claude Platform Docs — https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-fetch-tool [30] Model Card: Grok 4.7 — https://media.x.ai/v1/website/4p7card-5eccc980.pdf [31] Computer use tool — https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool --- # How does ChatGPT search decide which sources to cite? > As of 17 November 2026, OpenAI documents ChatGPT Search as a multi-stage process that can rewrite prompts, query providers, review results, and issue follow-up searches. Source: https://llmsearchnews.com/how-chatgpt-search-chooses-sources · Publisher: LLM Search News · Published: 2026-11-17 · Updated: 2026-11-17 ## TL;DR - As of 17 November 2026, OpenAI documents ChatGPT Search as a multi-stage process that can rewrite prompts, query providers, review results, and issue follow-up searches.[1] - OpenAI confirms third-party search-provider partnerships and partner-supplied information, but it does not publish a complete provider-by-query map.[1] - Bing has a historically documented ChatGPT integration, while reported Google-result tests do not confirm a current Google–OpenAI provider agreement.[6] - OAI-SearchBot controls eligibility for ChatGPT Search, GPTBot concerns potential training use, and ChatGPT-User handles user-triggered page visits.[2][11] - OpenAI publishes no exact ranking formula or citation-selection rule, so allowing a crawler can make a page eligible without guaranteeing that ChatGPT will cite it.[2][11] As of **17 November 2026**, OpenAI documents ChatGPT Search as a multi-stage process: ChatGPT may decide that a question needs current information, rewrite it into one or more targeted queries, send those queries to search providers, review the results, and use them to generate an answer with source links.[1] OpenAI does **not** publish the exact ranking formula, provider-routing map, or rule that determines which retrieved page becomes a citation.[1] ## What happened? OpenAI’s current documentation describes ChatGPT Search as a search-and-generation system rather than a single lookup. ChatGPT can turn a user’s request into one or more search queries, retrieve relevant results, and use those results to produce an answer with links to sources.[1][2] The documented process has several visible stages: 1. ChatGPT determines that web search could improve the answer. 2. It rewrites the user’s request into one or more targeted queries. 3. It sends those queries to search providers when required. 4. It reviews the initial results. 5. It may issue additional, more specific searches. 6. It generates an answer that can contain inline citations and a Sources panel.[3] OpenAI’s documentation gives an example in which a question can be rewritten into a targeted query such as “CCR8 immunotherapy drug development 2025.”[1] That example shows that ChatGPT Search may search for a reformulated information need rather than the user’s original wording. OpenAI also says that memory can influence query rewriting when memory is enabled.[1] For local results, the service may use an approximate location derived from the user’s IP address; OpenAI says the IP address itself and ChatGPT account information are not shared with search providers for the search.[1] The public description therefore supports a **multi-pass retrieval model**. It does not establish a fixed number of searches, a universal sequence, or a single search index used for every request.[4][2] ## Why does it matter? The distinction between the user’s prompt and ChatGPT’s rewritten queries explains why ordinary keyword matching is an incomplete model of visibility. A page may be relevant to the user’s topic but fail to appear if it does not match the narrower queries ChatGPT sends to providers or if it is not eligible for OpenAI’s search crawling.[1] It also explains why a citation is not simply a ranking position copied from a conventional search-results page. OpenAI says ChatGPT Search uses multiple factors intended to help people find relevant and reliable information, but it does not disclose the individual signals, their weights, or a deterministic scoring formula.[1] OpenAI’s interface exposes only part of the result-selection process. Users can select an inline citation to open its source, preview citations on desktop, and select **Sources** to view cited sources and other relevant links.[1] The Sources panel can therefore contain links beyond the sources directly attached to individual statements.[3] For publishers, the practical implication is equally specific: allowing OpenAI’s search crawler makes a site eligible for ChatGPT Search, but it does not guarantee a citation, a particular position, or inclusion for every relevant query.[1] ## What do we know about search providers? OpenAI says ChatGPT Search sometimes partners with other search providers.[1] When it does, ChatGPT typically rewrites the prompt into targeted queries and sends them to those providers.[1] The current documentation does not identify the provider used for every query, disclose an exclusive provider, or publish a complete routing table. OpenAI’s launch material separately says ChatGPT Search can use third-party search providers and information supplied directly by partners.[5][1] That distinction matters because partner-supplied information should not automatically be described as an ordinary search-engine result.[1] The public documentation confirms multiple possible information channels but does not describe how those channels are merged before the final answer is produced.[6] ### Is ChatGPT Search powered by Bing? Microsoft announced in May 2023 that Bing was being brought to ChatGPT as the default search experience for that product-era integration.[7] That announcement documented a Bing integration, but it does not by itself establish that every ChatGPT Search request in November 2026 uses Bing or that Bing is the exclusive provider.[8] OpenAI’s current Help Center documentation describes partnerships with search providers without publishing a complete current provider list.[1] The defensible conclusion is that Bing has a historically documented relationship with ChatGPT, while the provider used for a particular current query is not publicly specified in the reviewed documentation.[9] ### Does ChatGPT Search use Google? Reported tests have suggested that some ChatGPT responses may reflect Google-indexed results.[6] Those tests do not establish a Google–OpenAI partnership, a fixed Google fallback rule, or universal Google routing.[10][6] Google should therefore be described as **test-based reporting**, not as a confirmed OpenAI disclosure.[9][11] OpenAI’s own current documentation confirms partner-based search in general but does not identify Google as the provider for ordinary ChatGPT Search requests. ## How does query rewriting work? Query rewriting converts a conversational request into one or more targeted searches. OpenAI says ChatGPT Search typically rewrites a query before sending it to search providers and may use follow-up searches after reviewing initial results.[1] A single user request can therefore produce several related information needs. For example, a question about a medical development could be rewritten into a query containing a specific molecule, treatment area, and year rather than sent verbatim to a provider.[1] OpenAI does not publish the full rewriting algorithm.[1][12] The reviewed documentation does not specify how ChatGPT chooses terms, how many rewritten queries it creates, how it decides that a follow-up is necessary, or how it combines results from separate searches. Memory can also affect rewriting when enabled.[1] OpenAI presents this as part of the information shared with search providers, alongside approximate location for some local-search requests.[1] These details mean that prompt wording, context, freshness, and location can affect retrieval. The public material does not support a stronger claim about the relative importance of any one factor.[13][11] ## What does OAI-SearchBot do? OAI-SearchBot is OpenAI’s crawler for surfacing websites in ChatGPT Search results.[11][2] OpenAI says sites that opt out of OAI-SearchBot will not be shown in ChatGPT Search answers, although they may still appear as navigational links.[2] OpenAI recommends allowing OAI-SearchBot in `robots.txt` and permitting requests from its published IP ranges when a publisher wants its pages to be eligible for search results.[2] OpenAI says changes to crawler access can take approximately 24 hours to affect its systems.[2] OAI-SearchBot controls search eligibility, not ranking. Allowing it does not guarantee that a page will be retrieved, cited, or placed in a particular part of an answer.[2] A publisher can use a `robots.txt` rule such as: ```text User-agent: OAI-SearchBot Allow: / ``` The rule expresses permission for OpenAI’s search crawler, but the final result still depends on ChatGPT’s retrieval and answer-generation processes.[9] ## How is OAI-SearchBot different from GPTBot and ChatGPT-User? OpenAI describes the three user agents as serving different purposes.[2] | User agent | OpenAI’s documented role | What the setting controls | |---|---|---| | **OAI-SearchBot** | Crawls sites so they can surface in ChatGPT Search results.[2][11] | Search eligibility. | | **GPTBot** | Crawls content that may be used to train OpenAI’s foundation models.[2] | A training-use signal. | | **ChatGPT-User** | Visits a page when a user asks ChatGPT or a Custom GPT to do so.[2] | User-triggered page access, not automatic Search crawling. | OpenAI says the OAI-SearchBot and GPTBot settings are independent.[2] A publisher can allow OAI-SearchBot for search eligibility while disallowing GPTBot to signal that crawled content should not be used for training.[2] ChatGPT-User is different because its requests are initiated by a user.[2] OpenAI says ChatGPT-User is not used for automatic web crawling or to determine whether content may appear in Search, and that `robots.txt` rules may not apply to those user-triggered requests.[2] Publishers should therefore use OAI-SearchBot rules to manage automatic ChatGPT Search crawling rather than treating ChatGPT-User as the search crawler.[2] ## How are citations displayed? Search-enabled responses may include inline citations that open the cited source.[1] On desktop, users can preview a citation by pointing to it, and the Sources control can show cited sources and other relevant links.[1] This interface provides provenance for readers, but it does not expose the internal source-selection algorithm.[4] OpenAI does not document a public rule that maps each retrieved result to a particular sentence or determines how many sources an answer must display.[4] OpenAI also cautions that search results and citations can be incomplete, outdated, or incorrect.[1] It recommends opening the source, checking its date, and preferring authoritative sources when accuracy matters.[1] A citation should therefore be treated as a verifiable link to supporting material, not as proof that the source was the highest-ranked result or that every part of a generated answer is fully supported.[3] ## What is confirmed, and what is not? | Question | Public status as of 17 November 2026 | |---|---| | Can ChatGPT rewrite a user’s request into targeted searches? | **Confirmed.** OpenAI documents query rewriting and searches sent to providers.[1] | | Can ChatGPT issue follow-up searches? | **Confirmed in OpenAI’s description.** The documented process includes additional searches after reviewing initial results.[1] | | Does ChatGPT Search use third-party search providers? | **Confirmed.** OpenAI says it sometimes partners with other providers.[1] | | Is Bing historically integrated with ChatGPT? | **Confirmed for the 2023 product-era announcement.** Microsoft described Bing as the default search experience in that integration.[7] | | Is Bing the only provider used by current ChatGPT Search? | **Not confirmed.** OpenAI does not publish an exclusive-provider statement in the reviewed documentation.[9] | | Does ChatGPT Search use Google? | **Suggested by reported tests, not confirmed by OpenAI.**[9] | | Is OAI-SearchBot used for ChatGPT Search? | **Confirmed.** OpenAI says it surfaces websites in ChatGPT search results.[2] | | Does allowing OAI-SearchBot guarantee a citation? | **No.** OpenAI describes eligibility, not guaranteed placement or citation.[1] | | Does GPTBot control ChatGPT Search visibility? | **No.** OpenAI describes GPTBot as a training-related crawler and separates it from OAI-SearchBot.[11] | | Is the exact ranking formula public? | **No.** OpenAI describes multiple relevance and reliability factors without publishing weights or a formula.[12] | | Does the Sources panel contain only inline citations? | **Not necessarily.** OpenAI says it can show cited sources and other relevant links.[1] | Background: why retrieval and citation are different technical problems Retrieval-augmented generation research separates the documents retrieved for a model from the model’s generated text. The RAG framework combines a generator’s internal knowledge with retrieved documents and conditions generation on those documents.[4] That architecture does not, by itself, solve provenance. Research has treated citation support and citation precision as separate evaluation questions: whether claims are supported and whether cited material actually supports the associated statement.[3] This distinction is useful when interpreting ChatGPT Search. Retrieval asks which documents enter the process; citation attribution asks which sources are attached to the answer; visibility research asks how stable those sources are across repeated prompts and time.[14] Academic systems and benchmarks help explain these problems, but they do not disclose OpenAI’s private Search ranking rules.[2] ## A dated timeline of ChatGPT Search source selection | Date | Development | What it establishes | |---|---|---| | **7 February 2023** | Microsoft announced a new Bing and Edge experience with AI-powered search and conversational answers.[12] | Bing and Microsoft were developing an AI-search relationship before ChatGPT Search launched. | | **23 May 2023** | Microsoft said Bing was being brought to ChatGPT as the default search experience for the then-current plugin rollout.[7] | Bing was publicly documented as a ChatGPT search integration at that time. | | **31 October 2024** | OpenAI announced ChatGPT Search with links to relevant web sources and a Sources interface.[15] | OpenAI publicly introduced the search-and-citation product model. | | **2025** | Third-party reports described tests suggesting that some ChatGPT responses reflected Google results.[6] | Google-result use was reported experimentally, not confirmed as a provider agreement. | | **31 October 2026** | OpenAI’s crawler documentation described independent controls for OAI-SearchBot, GPTBot, and ChatGPT-User.[2] | Search eligibility, training-related crawling, and user-triggered fetching were formally separated. | | **17 November 2026** | OpenAI’s Help Center documented query rewriting, provider partnerships, citations, and the Sources panel.[1] | OpenAI’s current public account is a multi-pass process with undisclosed provider routing and ranking details. | ## What should publishers do? Publishers that want pages to be eligible for ChatGPT Search should review their `robots.txt` rules and allow OAI-SearchBot where appropriate.[11] They should also permit requests from OpenAI’s published IP ranges, as OpenAI recommends.[2] Publishers should make a separate decision about GPTBot.[11] Blocking GPTBot signals that content should not be used for training, while allowing OAI-SearchBot preserves search eligibility under OpenAI’s documented controls.[2] A publisher should not use ChatGPT-User as a proxy for Search visibility.[2] OpenAI describes it as a user-triggered page fetcher rather than an automatic search crawler.[11] For measurement, publishers can inspect referral traffic and citation appearances, but those observations should not be treated as a complete view of retrieval. OpenAI’s public documentation does not reveal all candidate results, provider routing, ranking weights, or uncited searches.[12] The strongest defensible workflow is therefore: 1. Check whether OAI-SearchBot is allowed. 2. Keep GPTBot policy separate from Search policy. 3. Avoid describing Bing or Google as the universal current provider without direct confirmation. 4. Open citations to verify the source and its publication date. 5. Treat citation frequency as an observed outcome, not a disclosed ranking score. 6. Recheck OpenAI’s crawler and Search documentation because the product and policies can change. ## What is not confirmed? OpenAI has not published the exact formula used to rank retrieved pages for ChatGPT Search.[2] Its public wording is limited to multiple factors intended to support relevance and reliability, without individual weights or guaranteed placement.[1] OpenAI has also not published a provider-by-query map.[1] The documentation confirms third-party provider partnerships and partner-supplied information but does not state which provider handles a particular request or how results from different channels are merged.[6] The reviewed documentation does not publish a general rule connecting a retrieved page to a specific inline citation.[1] It explains how users can open citations and view the Sources panel, not how the system selects or ranks each displayed source.[1] Finally, reported Google-result tests should not be upgraded into a confirmed Google partnership.[9] The evidence supports a careful distinction between official documentation, historical integration announcements, and external behavioral observations.[8] ## FAQ ### How does ChatGPT Search choose sources? ChatGPT may rewrite a request into targeted queries, send them to search providers, review the results, and issue follow-up searches before generating an answer.[1] OpenAI says ranking uses multiple factors intended to support relevance and reliability, but it does not publish the formula or factor weights.[1][12] ### Does ChatGPT Search use Bing? Microsoft publicly described Bing as ChatGPT’s default search experience in a 2023 integration announcement.[7] OpenAI’s current documentation confirms partnerships with search providers but does not establish that Bing is the exclusive provider for every ChatGPT Search request.[8] ### Does ChatGPT Search use Google? External tests have reported results that appeared to reflect Google-indexed pages.[2][14] Those tests do not confirm a Google–OpenAI agreement or a universal Google fallback, and OpenAI’s current documentation does not identify Google as the provider for each search.[6] ### What is OAI-SearchBot? OAI-SearchBot is OpenAI’s crawler for surfacing websites in ChatGPT Search results.[11][2] OpenAI says sites that block it will not appear in ChatGPT Search answers, although they may still appear as navigational links.[2] ### Should publishers allow OAI-SearchBot? A publisher that wants eligibility for ChatGPT Search should consider allowing OAI-SearchBot and OpenAI’s published crawler IP ranges.[2] Permission does not guarantee a citation, ranking position, or inclusion for a specific query.[1] ## Sources [1] Searching the web with ChatGPT | OpenAI Help Center — https://help.openai.com/en/articles/9237897-chatgpt-search [2] Overview of OpenAI Crawlers — https://developers.openai.com/api/docs/bots [3] Evaluating Verifiability in Generative Search Engines - ACL Anthology — https://aclanthology.org/2023.findings-emnlp.467 [4] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — https://proceedings.neurips.cc/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf [5] Introducing ChatGPT search | OpenAI — https://openai.com/index/introducing-chatgpt-search [6] ChatGPT is using Google Search, multiple tests suggest — https://searchengineland.com/chatgpt-using-google-search-tests-460101 [7] Bing at Microsoft Build 2023: Continuing the Transformation of... — https://blogs.bing.com/search/2023/5/Bing-at-Microsoft-Build-2023/ [8] Joint Statement from OpenAI and Microsoft | OpenAI — https://openai.com/index/continuing-microsoft-partnership/ [9] ChatGPT Appears To Use Google Search As A Fallback — https://www.searchenginejournal.com/chatgpt-appears-to-use-google-search-as-a-fallback/552089 [10] Brave Search API | Brave — https://brave.com/search/api [11] Publishers and Developers - FAQ | OpenAI Help Center — https://help.openai.com/en/articles/12627856-publishers-and-developers-faq?facet1=spreadsheets [12] Reinventing search with a new AI-powered Microsoft Bing and Edge, your copilot for the web - The Official Microsoft Blog — https://blogs.microsoft.com/blog/2023/02/07/reinventing-search-with-a-new-ai-powered-microsoft-bing-and-edge-your-copilot-for-the-web/ [13] Brave launches most powerful search API for AI to date | Brave — https://brave.com/blog/most-powerful-search-api-for-ai/ [14] Don't Measure Once: Measuring Visibility in AI Search (GEO) — https://arxiv.org/html/2604.07585v1 [15] OpenAI adds new search function to ChatGPT | Reuters — https://www.reuters.com/technology/artificial-intelligence/openai-adds-new-search-function-chatgpt-2024-10-31