This site presents the analysis of ten in-class focus groups on how HKS students are navigating AI, and uses that analysis to demonstrate what different qualitative coding methods can and cannot do. (An eleventh section ran the protocol but was not recorded.)
Two-sided risk split and pair-share dot totals, hand-entered from notes and flip-chart photos.
Every method the site uses has a specific shortcoming.
| method | what it's good at | how it breaks |
|---|---|---|
| human deductive | fairly fast | blind to what you didn't ask |
| human inductive / in-vivo | preserves voice, surfaces surprise | doesn't scale, not reproducible across coders |
| LLM deductive | fast and consistent when given a real codebook | homogenizes, loses affect, invents plausible codes |
| LLM inductive | generates themes from scratch in seconds | generic themes, no codebook trail, no in-vivo preservation |
| embeddings / topic models | "objective"-looking | (probably) recovers structure you already knew — often a function of how the protocol was designed |
Triangulating across methods is expensive but is sometimes the only defensible approach for a public artifact, peer-reviewed research, or high-stakes real-world analysis.
One six-turn exchange from FG-10 Sanchez (§4, two-sided risk vote), read three ways: by an LLM applying the a-priori codebook, by the same LLM generating themes from scratch, and by a human reading closely for what the other two missed.
Three different things you see in the same six turns, depending on what method you use to code.
A note-taker in each focus group wrote down what the microphone couldn't hear: who laughed, who went quiet, who crossed their arms, who leaned in. Below are four moments (this is just a selection), paired with the exchange in the transcript.
make stage01e.Grounded in 67 votes on the two-sided risk question, 90 dot-vote items aggregated across ten focus groups, and 1,565 coded utterances (out of 3,674 transcribed segments).
The dominant emotional register is anxiety, not enthusiasm
(but the protocol primes for this). When asked which risk feels more
present, 57% of students named over-use (their own learning, writing,
and judgment) over the 43% who named under-use (falling behind peers
and employers). Across the corpus, anxiety codes — about lost skill,
loss of authorship, dependency — appear roughly 250 times;
anxiety.learning_loss alone is the fourth-most-frequent
code. Students' main concern seems to be something like
"I am not yet calibrated."
However: Section 4 of the protocol asked participants to pick between an over-use concern and an under-use concern, which structurally invited concern-talk; a different opening prompt might surface a different register.
The integrity worry is less salient than the learning-atrophy worry.
Anxiety about getting caught or plagiarism
(anxiety.integrity, 77 utterances) lands well below
anxiety about losing skill (anxiety.learning_loss, 130).
The dominant fear is not "what will my professor think,"
it's "what is this doing to me?"
[see also: The Little Mermaid code]
The credential feels fragile. Many students seem to be experiencing the degree itself as vulnerable to being hollowed out by AI use while they are earning it, or to being devalued by a generation-level stigma they did not choose. The learning-atrophy worry and the credential-devaluation worry are fused in ways the codebook does not capture separately.
What students want is judgment, not vague honor-code policy or a new course on how to use ChatGPT. They want a framework for making their own decisions, consistent guidance across classes, and real-world case studies of how people in their intended fields are actually using these tools. They want to be treated as people making serious bets about their own formation.
React with an emoji or leave a comment on any finding.
Every substantive quote from the ten focus groups, plus the ten multi-voice exchanges, grouped by the section of the protocol they came from. Lives on its own page so it doesn't swallow this one.
What do you think?
Reactions, pushbacks, questions? Anonymous unless you say otherwise.