B rief recap In an earlier post, “Same prompt, four AIs: why answers aren’t always the same” , I looked at what happened when the same prompt was given to four different LLMs. Unsurprisingly, perhaps, the answers were not identical. That raised an interesting follow-on question: what was the prompt actually trying to do? The answer is that it was trying to support a form of thematic analysis. In that case, the object of analysis was an academic profile, including my Google Scholar profile. The version I discuss here is a modified prompt, used with Claude.ai, where I uploaded a CV and asked the system to find other appropriate public resources connected with a named person and institution. The aim was not simply to summarise the CV, but to identify visible and less visible themes across a whole career. Why use thematic analysis? Thematic analysis is widely used by academics, especially in qualitative research. Braun and Clarke’s well-known paper, Using thematic analysis in psy...
Same Prompt, Four AIs — Why the Answers Aren’t the Same The differences aren’t just in the answers—they’re in the thinking Generative AI tools are often discussed as if they were interchangeable—different interfaces delivering broadly similar outputs. However, when applied to complex intellectual tasks, meaningful differences begin to emerge. To explore this, I ran the same academically rigorous prompt through four leading systems—Claude, ChatGPT, Google Gemini, and Copilot. The task required a full thematic analysis of a researcher’s career using the framework developed by Virginia Braun and Victoria Clarke . What followed was not simply variation in output, but variation in how each system approached the act of analysis itself. Same Input, Different Interpretations At a high level, the experiment is simple: One prompt → Four models → Four distinct approaches What changes is not the instruction, but how each system: Interprets the task Handles uncertainty Applies methodology Defines ...