Quotes beat keyword stuffing in a GEO benchmark
updated 3 min read Augustyn Głowacki
In a 10,000-query benchmark, adding quotations increased a source's share of an AI-generated answer by 40.9%, adding statistics increased it by 30.6%, and keyword stuffing reduced it by 8.3% (Aggarwal et al., KDD 2024). The study tested nine rewriting methods on sources that had already been retrieved. It did not test whether those methods help a page get retrieved.
What did the study actually measure?
The researchers gave their answer-generation system a query and five retrieved documents. They measured each document’s share of the generated answer using Position-Adjusted Word Count. They did not test whether a rewriting method helps a document appear in search results. The authors say: “owing to the black-box nature of search engine algorithms, we didn’t evaluate how GEO methods affect search rankings.”
What were the actual effect sizes?
The study tested nine methods against a baseline score of 19.3 on its primary metric (GEO, KDD 2024). Quotation Addition scored 27.2, a 40.9% improvement. Cite Sources scored 24.6, a 27.5% improvement. Statistics Addition scored 25.2, a 30.6% improvement. Keyword Stuffing scored 17.7, 8.3% below the baseline. The authors wrote: “while widely used for Search Engine Optimization, we find such methods offer little to no improvement on generative engine’s responses.”
“While widely used for Search Engine Optimization, we find such methods offer little to no improvement on generative engine’s responses.”
- Aggarwal et al., “GEO: Generative Engine Optimization,” KDD 2024
The authors are describing keyword stuffing, which scored below the unoptimized baseline.
Did the effect hold up on a real commercial engine?
The paper’s test on Perplexity measured smaller gains. Quotation Addition fell from 40.9% to 20.7% on the word-count metric. Statistics Addition fell from 30.6% to 8.7% on that metric and reached 37.2% on the LLM-judged Subjective Impression score. Keyword Stuffing scored 9.1% below the baseline. The 40.9% result came from a 2023 research system and does not predict the gain on a current engine.
Why does an answer engine reward quotes and statistics specifically?
Numbers and named quotations give an answer engine text it can repeat exactly. In MaxAEO’s analysis of 3,200 cited passages, statistic lines were quoted word for word 61% of the time, table rows 58%, and plain narrative prose 12%. The passages were classified by hand and the underlying data was not released, so I treat these figures as directional (MaxAEO).
What are the study’s limits?
The study used GPT-3.5-era models, one prompt template, five sources and data from 2023. GPT-3.5 scored some of its metrics. All sources had already been retrieved, so the results say nothing about whether quotes or statistics help a page get found. The percentages may differ on current engines.
The study tested ways to rewrite text. It did not compare page formats or test a standalone answer block. I put quotations and statistics near the start of a page because models handled information at the start and end of a context better than information in the middle in a separate study (Liu et al., “Lost in the Middle”, TACL 2024). That placement is my inference.
What does an answer block that follows this evidence actually look like?
Same claim, two phrasings. Before: “Adding quotations to a page tends to help quite a lot with how AI engines treat it.” After: “Quotation Addition improved a source’s share of a generated answer by 40.9% over the unoptimized baseline, across a 10,000-query benchmark (GEO, KDD 2024).” Both sentences make the same claim. The second one carries the number, the subject, the source, and the sample inside the sentence, which is what survives being lifted out of the page and quoted with the link stripped. It is the rule I apply to every page on this site.