Reddit AI Search Favors Formal Comments Over Personal Stories
A University of Illinois study found Reddit’s AI search prefers formal, highly upvoted comments, sidelining those rich in personal experience. The feature also strips first‑person language from responses, reshaping user testimony into generic advice.
By Felo News Desk · Published
When Reddit rolled out its AI‑powered search feature, many users expected the system to surface the most helpful, experience‑rich replies. Instead, a recent preprint from researchers at the University of Illinois Urbana‑Champaign reveals that the AI favors formal, already‑upvoted comments and strips away much of the first‑person language that gives advice a personal touch.
How the Audit Was Conducted
The research team processed 10,000 questions through Reddit’s AI search three times, capturing 30,000 answers that referenced 14.68 million comments across 20 advice and support subreddits. The questions were generated by an LLM that rewrote real posts from ten large communities—such as r/personalfinance and r/AskDocs—and ten smaller ones like r/UKJobs and r/AusLegal. The three runs were spaced about five hours apart to isolate system behavior from changes on Reddit itself.
Each answer was analyzed to trace back to the quoted comment or source list, allowing the researchers to compare the chosen comment with other comments in the same thread that were ignored. This method uncovered patterns in the language and metadata that influence selection.
Formality Wins Over Experience
Formality emerged as the strongest predictor. A comment that was one standard deviation more formal was 49% more likely to be chosen (odds ratio 1.488). Formality was measured by a text classifier that scores language on a scale from informal to formal. Comments that used modal verbs such as “should” and “must” also had higher odds of selection, though to a lesser degree (odds ratio 1.070).
Conversely, comments with markers of personal experience—identified by first‑person pronouns, past tense, and other experiential cues—were less likely to be selected (odds ratio 0.789). Supportive language also saw a slight decline in selection probability (odds ratio 0.924). Even after controlling for comment score, position, and age, formality remained a significant factor (odds ratio 1.213) while experiential voice stayed slightly negative (odds ratio 0.860).
Visibility and Timing Matter
Visibility within a thread was the most influential factor. A comment one standard deviation higher in score increased the odds of being chosen by 2.88 times. The median selected comment ranked at the 91st percentile of its thread’s score, compared to the 45th percentile for non‑selected comments. Additionally, 92% of selected comments were direct replies to the original post, and they appeared a median of 1.2 hours after the post, versus 5.9 hours for comments that were not chosen.
Other factors that boosted selection odds included comment length (odds ratio 1.79) and the presence of an external link (odds ratio 2.25). Surprisingly, comments with a score of zero or below made up 0.53% of selected comments, despite representing only 5.1% of all comments examined.
First‑Person Language Vanishes in AI Answers
In a sample of 1,000 queries, the researchers compared the first‑person usage in the quoted Reddit comments to the AI‑generated responses. The use of words like “I” and “my” dropped from 3.3% in the original comments to a mere 0.06% in the answers. When the same queries were fed to GPT‑4o‑mini and GPT‑5 with web search enabled, both models also reduced first‑person language, but Reddit’s responses showed the most pronounced shift.
Out of 4,932 citations across both models and search configurations, only 18 referenced Reddit directly, indicating that the AI largely relies on external sources rather than the community’s own content.
Why This Matters for Reddit Users
Reddit’s value lies in its crowd‑sourced, first‑hand experiences. When the AI search system filters out those voices and rephrases them in a formal tone, it risks delivering generic advice that may not reflect the nuanced realities of individual users. This can erode trust in the platform’s ability to provide authentic, community‑driven help.
What Happens Next?
The study is still a preprint and has not undergone peer review. The authors caution that their findings are observational and not causal. Future updates to the AI search feature or a peer‑reviewed version of the paper may alter these conclusions. Reddit’s CEO, Steve Huffman, has emphasized the platform’s strength in aggregating multiple perspectives, so it remains to be seen how the company will respond to these insights.
Key facts
- Reddit AI search favors formal, highly upvoted comments over personal experience
- First‑person language drops dramatically in AI‑generated answers
- Visibility and timing of comments strongly influence selection
- External links and longer comments increase odds of being chosen
- Only a handful of citations reference Reddit directly in AI responses
Why it matters
Reddit’s community‑driven advice depends on authentic personal testimony. If AI search removes that voice, users may receive less relevant or trustworthy guidance, undermining the platform’s core value.
Frequently asked questions
What is Reddit AI search?
Reddit AI search, formerly known as Reddit Answers, is a feature that uses large language models to generate concise answers to user queries by pulling information from Reddit posts and external sources.
Why does the AI strip first‑person language?
The system rephrases quoted content into a more neutral tone, which reduces first‑person pronouns to create a generalized response.
Will Reddit change its AI search algorithm?
It is unclear; the company has not announced any changes following this study, but future updates may address the bias toward formal comments.
Sources
- [1] searchenginejournal.com — originally reported as “Reddit AI Search Picks Already-Popular Comments, Audit Finds”
