A new eight-month study of how ChatGPT-4o searches the web reveals a troubling pattern: the AI defaults to recycling the same trusted sources and avoids authoritative academic or journalistic sources unless explicitly challenged. Researchers from the London School of Economics found that ChatGPT behaves as a "conservative mediator" of web knowledge, relying heavily on commercial domains and Reddit posts while ignoring institutional sources like Harvard library guides and the National Institutes of Health database unless users specifically demand credibility justification .

The findings raise urgent questions about how AI search engines are shaping what information reaches us, especially as AI agents become more autonomous and integrated into everyday tools like ChatGPT's Agent mode, Opera's Neon, and Perplexity's Comet. Before these systems act on our behalf, researchers argue, we need far greater transparency into how they "see" and prioritize web sources.

What Does ChatGPT Actually Cite When You Ask It Questions?

Researchers Janna Joceli Omena, Giulia Tucci, and Aanila Kishwar conducted a systematic study from December 2024 to July 2025, repeatedly prompting ChatGPT-4o with the same questions and tracking which sources it cited, how consistent its answers were, and whether its reasoning changed over time . The results were striking: ChatGPT recycled approximately 54.8% of web sources across repeated prompt rounds, meaning it kept returning to the same handful of trusted domains rather than drawing from the broader web.

The sources that dominated ChatGPT's responses clustered around a familiar commercial core: tech blogs like TechTarget and DataCamp appeared in over half of all citations. When explaining its own technical functioning, ChatGPT surprisingly relied on Reddit, a user-generated discussion platform, as a primary source of credibility. This happened precisely where OpenAI's official documentation was absent, suggesting the AI treats Reddit posts as quasi-technical confirmation for its own architecture .

What's particularly revealing is what ChatGPT didn't cite by default. Academic sources, journalistic outlets, and institutional research databases remained largely invisible unless users explicitly asked the model to justify its reasoning or cite more authoritative sources. When researchers prompted ChatGPT to explain source credibility, it suddenly cited Harvard University's library research guide, ask.library.harvard.edu. When they asked for scientific authority, PubMed, the U.S. National Institutes of Health database, appeared in responses. These authoritative sources weren't unavailable to the model; they were simply not the default .

How Can Users Get More Reliable Answers From AI Search?

• Challenge the Model's Credibility: Ask ChatGPT to justify its sources or explain why it selected particular citations. Meta-cognitive and evaluative prompts trigger the model to diversify beyond its default commercial sources and pull in more authoritative institutional sources.
• Request Scientific or Academic Authority: When seeking information on technical or scientific topics, explicitly ask the model to cite peer-reviewed research or institutional databases. This activates access to sources like PubMed and academic libraries that remain hidden in default responses.
• Demand Transparency on Search Providers: Ask which third-party search engines the AI is using and how it ranks results. The study found ChatGPT gives inconsistent answers to this question, shifting from unnamed providers to Bing-only over time, revealing the model's own uncertainty about its search infrastructure.

Why Does ChatGPT Default to "Conservative" Searching?

The researchers emphasize that ChatGPT's conservatism is not ideological but operational. The model appears to be designed with a bias toward caution: once a source is trusted, it remains trusted, and the model recycles it repeatedly. This creates a self-reinforcing loop where commercial blogs and Reddit posts become the epistemic foundation for answers, while more rigorous sources remain in the background .

The study also uncovered a second troubling pattern: inconsistency. When asked to explain which search providers it uses and how it ranks results, ChatGPT-4o returned contradictory answers over the eight-month study period. Early responses mentioned unnamed providers; later responses shifted to multi-provider or Bing-only explanations. This inconsistency suggests the model itself may not have a clear, stable understanding of its own search infrastructure .

Researchers note that source-type diversity, rather than geographic diversity, is not ChatGPT's default. The model must be pressured into drawing from different categories of sources. When users stop challenging it, ChatGPT reverts to its comfortable core of commercial domains and user-generated content. This pattern has significant implications for how people experience the web through AI intermediaries.

What Does This Mean for AI Agents Acting on Your Behalf?

The stakes of this research extend beyond search quality. AI agents are increasingly autonomous, managing calendars, writing code, shopping, and taking actions on users' behalf. If these agents inherit ChatGPT's conservative search patterns, they may make decisions based on incomplete or biased information sources. A shopping agent might favor products from domains that appear frequently in ChatGPT's training data. A research agent might miss critical academic findings because institutional sources remain deprioritized .

The researchers call for robust forms of "platform observability," meaning methods for capturing how AI chatbots scan, rank, and package web information over time. Currently, most AI search systems operate as black boxes. Users cannot see which sources the model considered and rejected, how it weighted different types of evidence, or why it made particular citation choices. This opacity becomes dangerous as AI agents gain more autonomy.

"Before AI agents could act on the web, they first had to learn how to see it. This raises an important question: how does AI search the web?" noted researchers examining the integration of web search into large language models.

Janna Joceli Omena, Giulia Tucci, and Aanila Kishwar, London School of Economics

The study's findings suggest that transparency in AI search is not a luxury but a necessity. As AI systems become more integrated into how we discover information and make decisions, understanding their search biases becomes a matter of public interest. The current model, where commercial sources and user-generated content dominate by default while academic and journalistic sources remain hidden, may be reshaping what counts as credible knowledge in ways we barely understand .

For now, the burden falls on users to actively challenge AI systems, demand justification for their sources, and push them toward more authoritative information. But researchers argue this is not a sustainable solution. The technology itself needs to be redesigned with transparency and source diversity as defaults, not afterthoughts.