Jim Liu
2 years ago in General SEO
How Branding and Rankings Work in LLMs: An In-Depth Study seovendor.co
Jim Liu
2 years ago in General SEO
For those that are technically curious, this detailed study explores rankings in a whole new LLM (Large Language Model) world, which may have a profound impact on the future of SEO. As AI and LLMs enter the search space, there’s still a knowledge gap between how AI really works and how businesses appear in them. For one, results from AI can vary greatly from one prompt to another, almost at random. But just how much can results differ? Is there a pattern that emerges? This study goes into extensive testing to observe the impact of AI as it exists today, including prompt variations, web-browsing/RAG capabilities (ChatGPT), and citations (SearchGPT). Using local dentists as a reference, this study demonstrates the results from sampling 10X to 100X in ChatGPT to explore the impact of Brand Visibility in AI on everyday businesses.
Never miss any SEO updates, news, tools, jobs, events! Subscribe to the weekly SEOFOMO newsletter
Login with Google
3 Comments
can anyone summarise this in layman’s terms?
There’s several high-level insights from this study, I’ll list them here:
1. ChatGPT’s knowledge contains significant training bias that affect the appearance of a brand or website when prompted for a recommendation.
2. Results from ChatGPT are volatile, even when given the same prompt over and over, requiring multi-sampling to measure the true visibility of a brand.
3. Web-capable LLMs like ChatGPT Search offer greater consistency in results, however, they still exhibit bias and variations in results.
4. Branding is going to be a significant signal to have a presence in LLMs as it’s about information repetition and similarity, rather than say, a link to a page a crawler discovers.
5. LLMs like ChatGPT are trained with a wealth of small business brand and website knowledge. Business discovery is not limited to Fortune 500 companies.
6. Business knowledge in ChatGPT is nowhere near perfect. It can hallucinate names or create incorrect variations of names and websites.
7. Prompt variations won’t change results if the meaning stays the same even if words are different. Another way to look at it, if changing a word in a prompt, no matter how small, changes the meaning, then the outcome will be different.
thanks