could AI help surface patterns in 300+ pieces of teaching scholarship that aren't visible just by browsing it?
 |
| co-author map |
The National Teaching Repository (NTR) has just won a prestigious national award for teaching and learning — the Collaborative Award for Teaching Excellence 2026. As a member of the NTR board (I am hiding in the bottom left corner of the team image :- ), and someone who spends a lot of time on this blog testing what gen AI is actually good for, I wanted to use the moment to try something on the repository itself: could AI help surface patterns in 300+ pieces of teaching scholarship that aren't visible just by browsing it?
Two activities came out of this — mapping who has co-authored with whom, and then a thematic analysis of what the repository is actually about. One needed code. The other didn't. But the thing I actually want to talk about is a prompting habit that ran through both, because it's the part any educator can pick up and reuse, whether or not you can write a line of Python. All the prompting was done with Claude.ai.
The pattern: ask before you answer
Both prompts I used had the same instruction buried in them: don't just answer — ask me questions first.
For the co-author mapping:
I would like to go through the https://figshare.edgehill.ac.uk/The_National_Teaching_Repository articles to show links between authors. Produce code to do this before generating the final code ask questions for clarity
For the thematic analysis:
From the uploaded file using shared titles, carry out a thematic analysis. Before doing the analysis, please ask any questions you need, then do the analysis. Please use the steps in Braun and Clarke (V. Braun, V. Clarke, Using thematic analysis in psychology, Qualitative Research in Psychology, 3(2), 2006, pp. 77–101, DOI: 10.1191/1478088706qp063oa) to do this. Constraints Scope: All titles Depth of data: All available data Analytical lens: What are the themes in the repoistory this is data from https://figshare.edgehill.ac.uk/The_National_Teaching_Repository Target themes: Look for semantic themes and then provide a separate analysis of latent themes Please ask any clarification questions before starting the analysis.
This is the transferable bit. Neither prompt tells the AI what a "theme" should look like, how many there should be, or what counts as a meaningful author link. Instead, the prompt hands over a well-established method — in this case Braun and Clarke's thematic analysis — and asks the model to interrogate the task before attempting it. That single move does a few things at once: it forces scope decisions into the open (all titles? a subset?), it surfaces assumptions before they get baked into an answer, and it gives you, the human, a checkpoint to correct course before any real work happens. I've written about this pattern before in Iterative Prompting: How Better Questions Produce Better AI Answers, and it holds up here on a genuinely different kind of task.
None of that requires code. You could apply exactly this pattern to a reading list, a set of module descriptors, a batch of student feedback comments, or — as I did previously — an academic CV.
Activity 1: mapping who's connected to whom (the part that needed code)
The first activity did need some vibe-coding to go through the NTR's FigShare listing and build a map of co-author links — essentially, nodes for authors and lines connecting anyone who's published together. The output is a visual map showing clusters of frequent collaborators and the occasional bridge between clusters.
I'm flagging this one honestly: it needed a developer-adjacent step, and even then there was a small setup snag (a missing library import, since fixed in the repo's README). If you're not coding yourself, the practical takeaway isn't "go and do this" — it's that the same asking-questions-first prompt is what got a workable first version of the code, rather than a guess dressed up as a finished tool. The code is on GitHub if you have someone who can run it: scottturnercanterbury/NTRAI.
|
| co-author map |
Activity 2: what's the NTR actually about? (the part anyone can do)
The second activity is the one every reader of this blog could run themselves, in a chat window, with no coding at all: taking the list of titles produced in Activity 1 and asking for a thematic analysis.
Using titles only — not abstracts, not full text — the analysis came back with eight themes covering all 324 titles. In condensed form:
- Designing, Assessing and Structuring the Curriculum (≈53 titles)
- Digital, Technological and AI-Mediated Teaching (≈57 titles) — the largest cluster, with generative AI itself now a fast-growing sub-theme
- Equity, Belonging and Student Wellbeing (≈52 titles)
- Active, Playful and Creative Pedagogy (≈19 titles)
- Discipline-Specific and Practice-Based Teaching (≈34 titles)
- Scholarly Identity, Recognition and the SoTL Community (≈45 titles)
- Internationalisation and Cross-Border Partnership (≈27 titles)
- The Repository as Infrastructure (≈29 titles) — a reflexive cluster about the NTR itself: how to submit, how it's governed, its own history
That last theme is worth sitting with for a second: a chunk of the repository's own content is about being a repository — guidance for contributors, conference presentations, governance documents. It's a reminder that a resource like this doesn't sustain itself; there's ongoing, visible labour keeping it alive, produced by the same community whose collaborations show up in the Activity 1 map.
Worth trying, but...
This is the part I want to spend the most time on, because a tidy list of eight named themes is exactly the kind of AI output that invites more trust than it's earned.
It only saw titles. A title is a compressed, sometimes playful gesture at what a piece is about — not the piece itself. "The e(ducation) Pizza Game" tells you almost nothing about its pedagogical argument. Run the same exercise on abstracts instead of titles and I'd expect the theme boundaries to shift, possibly a lot. Treat the eight themes as a first hypothesis, not a settled account of the repository's content.
The output looks more definitive than the process was. Eight clean themes with tidy prevalence percentages reads like a rigorous finding. But a title-only, single-pass analysis is a long way from what Braun and Clarke's method looks like in a proper qualitative study, where a researcher immerses themselves in the full dataset, iteratively codes, and revisits earlier decisions in light of later ones. Citing the method doesn't automatically confer its rigour. If you use this pattern yourself, be honest in how you report it — "AI-assisted, titles-only, semantic-level pass" is a fair description; "thematic analysis" on its own oversells it.
The same-author problem compounds silently. In the co-author map, one real person can appear as two or three separate nodes because their name was entered inconsistently (initials one time, full name another, a maiden or married name change). That's a data-quality issue any AI system will happily paper over rather than flag, unless you specifically ask it to check for near-duplicate names. The thematic analysis has a quieter version of the same risk: near-identical titles, or one prolific author's stylistic habits, could distort prevalence counts without anyone noticing.
Ask what wasn't asked. Both prompts included an instruction to question the task first — and it's worth checking what questions the AI actually asked back, and whether they were the right ones. If it didn't ask about how you wanted duplicate authors handled, or what should happen with untitled or very short titles, that's a sign the AI accepted convenient defaults you might not have chosen yourself.
None of this is an argument against trying this. It's an argument for treating the output as a fast, genuinely useful starting point for human judgement — good for spotting a pattern you'd want to investigate properly, bad as a citable finding on its own.
A template you can reuse
If you want to try this on your own repository, reading list, or dataset of titles — no coding required — here's a generalised version of the prompt structure that worked above:
I would like to carry out a thematic analysis of [describe your dataset — e.g. "the titles in our institutional teaching repository" / "the descriptions of modules in this programme" / "these conference abstracts"].
Before doing the analysis, please ask me any clarifying questions you need — for example about scope, how to handle duplicate or near-identical entries, and what counts as a meaningful theme — then carry out the analysis.
Please use the steps in Braun and Clarke's thematic analysis method (V. Braun, V. Clarke, Using thematic analysis in psychology, Qualitative Research in Psychology, 3(2), 2006, pp. 77–101, DOI: 10.1191/1478088706qp063oa).
Constraints: Scope: [all entries / a defined subset] Depth of data: [titles only / abstracts / full text] Analytical lens: [what question are you bringing to this data?] Target themes: identify semantic themes, then provide a separate analysis of latent themes.
When you present the results, also tell me: which of your assumptions I should sanity-check, and what this analysis cannot tell me given the depth of data provided.
This prompt was produce by Claude.ai and the last line in it is one I wish had come up — it wasn't in my original prompt--- and I think it adds a bit more rigor to the process Asking the AI to name its own blind spots doesn't eliminate them, but it's a good way to get the caveats onto the page rather than leaving you to discover them later.
The National Teaching Repository earning national recognition is a good vehicle to keep testing what gen AI can do with the scholarship already sitting inside it — as a way of asking better questions of a body of work.
Comments
Post a Comment