"Kyle Roof: Keyword Math Still Beats Unguided AI Content (SEO Testing)"

Kyle Roof argues that SEO fundamentals (keyword density math, contextual terms, and links) still drive ranking, and that LLMs produce poor SEO content out of the box.

Kyle Roof, the technical SEO tester introduced as "the legend of SEO," closed this Day 1 segment by arguing that the underlying mechanics of ranking have not changed: Google still ranks on mathematical keyword and contextual-term factors, and links still work. His central evidence is a series of tests showing that large language models score poorly on those terms out of the box, and that asking a model to "SEO optimize" often strips needed terms out while only improving formatting. His framing throughout: AI is a productivity tool that only performs when a human directs it.

Main takeaways

1. Keyword density math still beats unguided AI content. Across repeated tests, LLMs produce low keyword and contextual-term scores out of the box, and "if you're bad at SEO, it's only gonna help you do bad SEO faster."
2. Asking an LLM to "SEO optimize" often makes the content worse on the metrics that matter. Most models improved their formatting score but dropped word count and stripped out terms, and Google's NLP could understand the optimized content less.
3. The base of SEO has not changed. From Roof's vantage point Google "turns the knobs," but the core ranking factors (keyword math, contextual terms, links) are stable.
4. AI is the Mechanical Turk. Like the chess "automaton" with a human hidden inside, AI only works because a human is directing it. "We still have jobs."
5. Schema "just for the sake of schema" was never and is still not a ranking factor. Roof tested it; schema is "a containing keyword." You can rank with schema, but schema alone is not a factor.
6. Links still work. A test firing "the spammiest of grossest links" lifted non-ranking sites into positions and moved page-4 sites to page 2.

Key points

Framing and the Mechanical Turk metaphor

• Introduced by the host as "the legend of SEO himself." Talk titled around "the unhelpful concept of Just for the Robots."
• Opens with the 1790s Mechanical Turk story: an inventor built a chess "automaton" that beat famous opponents (Napoleon and Benjamin Franklin are named); a final North American tour score is cited as "like 84 to 5." The secret was a man hidden inside with a magnetic board and a sliding seat. Metaphor: "the world's first AI," which only works because a human directs it.
• The inventor's name is transcribed as "Wolfgang von Kleppen"; the historical figure is Wolfgang von Kempelen, so the transcript spelling is likely an error (verify before quoting).
• References Mechanical Turk (the Amazon micro-worker platform) as a tool he used to split-test H1s and ad copy for click-through rate (a few hundred dollars to avoid losing thousands later).

Testing methodology ("where the sidewalk ends")

• First test ever: "is an H1 stronger than an H2?" Googling it returned 3 yeses, 3 nos, 3 maybes, so he tested.
• "Where the sidewalk ends": search for a term that returns no results to get a clean, private testing environment. Create 5 identical pages, change one element on one page; rises = positive factor, falls = negative factor, no move = non-factor.
• Speed-up technique: test the inverse. Put the element on all OTHER pages; if the page WITHOUT it drops, the element is a factor.
• Holds a patent on this testing method. Says the first conference he spoke at was "Rockstars," where he realized "nobody's testing" and capitalized with the patent.

Rhinoplasty Plano

• A public 30-day ranking competition for "rhinoplasty Plano" (Plano is outside Dallas). Only rule: a fresh domain. 27 professionals entered. Roof's team entered rhinoplastyplano.co.
• He built the site in lorem ipsum, using Page Optimizer Pro and Cora (transcribed "Coral"; the known tool is Cora, verify) to calculate how many times to place the exact keyword, variations, and contextual terms, then pasted those into the lorem ipsum.
• Results: about 2 weeks after the 30 days, #7 organic and #1 maps; about 2 weeks later #1 organic, wiped out maps, held the knowledge panel.
• Search Engine Journal published "Google ranks site written in Latin"; about 6 hours later Google de-indexed the site as "pure spam." That night, between 1:15 and 1:27 a.m., Google de-indexed 20 of his test sites ("major spam").
• Claims Google subsequently added a rule (verified via the Wayback Machine: absent before the contest, present after) in the auto-generated-content guidelines about text that "makes no sense to the reader, but which may contain search keywords." Roof says he may be the only person with both an SEO patent and a Google SEO rule.

Persistence and current proof

• Shows lorem ipsum pages still on page one after 7 years: "Rhinoplasty Garden Grove" (#1) and "Rhinoplasty Denver South" (#1; 3 months prior he held the knowledge panel #1 and #3). Says he also ranked pages live during a talk in Garden Grove, Bali, Chiang Mai, and Milan.
• New proof: a page for "best SaaS tool for SEO," put up 4 days before the talk, clocked in at #10, written in lorem ipsum and indexed (likely ranking for "best SaaS tool for SEO companies").

Schema and links

• Schema "just for the sake of schema" was never and is still not a ranking factor; schema is "a containing keyword." You CAN rank with schema; he credits Corey, Simon, and Terry doing "awesome things" with it.
• Links still work: cites a test from his friend Martin (who lives near him in Chiang Mai) where firing "the spammiest of grossest links" lifted non-ranking sites to positions around 30, 57, and 39, and moved page-4 sites to page 2.
• "From my vantage point, nothing has changed... the base of what we do really hasn't changed." Google "turns the knobs" but core ranking factors are stable.

LLM content tests (the central evidence)

• Goal: find the best LLMs for SEO by testing unprompted baseline output. Two prompts: (1) "a thousand-word article on what to see in Rome"; (2) the same "that is SEO optimized." Scored in Page Optimizer Pro (POP).
• Out-of-the-box scores were poor: a human-optimized article scores around 80, but the best LLM came in at 64 (transcribed "Gwen," likely Qwen). Google's own models (Gemma, Gemini) repeatedly landed near the bottom.
• Word count: of the models, only three delivered the requested 1,000 words. Going above the target is fine for SEO; going below is "often very problematic." Worst was Llama at 565 words.
• Used Flesch-Kincaid readability (not a ranking factor), aiming around 7th-grade level. Out-of-box readability did not change much.
• The key metric: important terms (variations, LSI, target-keyword usage) in the main content. For a ~1,000-word page the target range cited is 167 to 278 terms, and "the midpoint is the sweet spot." The most any LLM produced out of the box was 53 ("that's garbage"). "It blew my mind that a language model is not good at language."
• Cross-checked against Google's NLP API (the concepts Google understands about a page); LLM output fell well short of the needed midpoint.
• When asked to "SEO optimize," almost all models improved their POP score but DROPPED in word count and STRIPPED OUT terms (only one increased). Google's NLP could understand the optimized content LESS. The score rose only because formatting and subheadings improved; "they tricked POP in their formatting."
• Six months later and one year later (the data shown is "last March"): still poor. Word count improved over time; contextual-term scores and reading level remained low. Perplexity Pro was the best on terms but still "not a win."
• Bottom line: "we still have jobs." AI is a productivity tool, like the Mechanical Turk; it only performs when a human directs it. "If you're bad at SEO, it's only gonna help you do bad SEO faster."

To verify

• The talk transcript cuts off mid-sentence ("I've done about twelve consultations"); the remainder may exist in a Part 2 file.
• Inventor name ("Wolfgang von Kleppen" vs Wolfgang von Kempelen) and tool name ("Coral" vs Cora) are likely transcription artifacts; verify before quoting.

Source

Synthesized from the Day 1 transcript (SEO ST DAY 1 PART 1) covering Kyle Roof's portion of a three-speaker block. No slide deck was provided for Kyle Roof's talk (the only deck in this session was Marty Marion's). The source knowledge folder is internal and is not published on this site.