Reflection · Jul 2019 · 6 min

I called it breadth-first. Five of ten students called it too fast

In a teaching statement written for an award nomination, I described my approach as deliberately breadth-first — whet the appetite, give them enough to explore further. The anonymous course evaluations from the same course named the same property as the main problem. Both documents are honest. Only one of them was written by someone who knew what it cost.

Two documents about one course

In July 2019 I wrote a teaching statement for CG 501, three months after the course ended, as part of an institute teaching award nomination. Here is the paragraph that matters:

"I tried to incorporate a variety of topics and tried to follow a 'breadth first' approach; where the students' appetite is sufficiently whetted and they possess enough information to explore more, wherever they feel the desire to do so."

Here is the free-text response from five of the ten students who filled in the anonymous course reaction survey, to the question "list the topics which were taught too fast":

"Neural networks." · "neural networks" · "Some of the neural network topics were covered too fast." · "Perceptron and Hopefield network" · "Linear Regression kNN"

Nobody listed anything as too slow. Two students explicitly wrote that nothing was.

I did not write the teaching statement in ignorance of this. The same statement contains the sentence: "managing my research work and teaching responsibilities proved too much at times and I ended up teaching some topics too fast, while others not fast enough." So I knew. What I had not done was connect the admission to the philosophy — they sit four paragraphs apart in a two-page document, one framed as a considered pedagogical choice and the other as a scheduling apology.

They are the same fact.

What breadth-first actually cost

The arithmetic is not subtle. I had eleven lectures. I spent roughly six on machine learning — linear algebra and the perceptron, multilayer perceptrons and backpropagation, model evaluation and Hopfield networks, linear regression and K-means, k-nearest neighbours, NumPy and a from-scratch kNN implementation — and roughly five on game theory, from the prisoner's dilemma through iterated dominance, mixed strategies, extensive-form games and backwards induction.

That is a graduate machine learning course and a graduate game theory course, compressed into eleven sessions, for students whose backgrounds were cognitive science rather than computer science. Written out like that, "too fast" is not a student complaint. It is a description of the timetable.

The evaluation data supports reading it as a delivery problem rather than a content problem. Pace of instruction scored 2.20 out of 4; six of ten respondents put it at exactly "average" and none at the top box. But the item asking whether the amount of material was just right scored 2.80, and whether topics were covered in a logical sequence scored 3.20 — the second-highest of the twenty teaching-related items. The students did not think I had covered the wrong things, or the wrong number of things, or in the wrong order. They thought I had gone through them too quickly. Which is what breadth-first means when you have a fixed number of hours: the same map, drawn faster.

One respondent made the tradeoff explicit from the other direction, asking for "more time on neural networks than machine learning topics because that is what the course outline mentioned." That student wanted depth in the advertised topic and read the breadth as a substitution. On the terms of the published outline, they were right.

The seduction

I have written elsewhere about running into breadth-first failure at full speed in a research context — eight months of literature synthesis across four disciplines that produced a genuinely good map and no completed study. What I did not realise when I wrote that piece is that I had run the same experiment on twenty graduate students five years earlier, with the results written down, in a document I filed and did not re-read.

The structure is identical in both cases and worth naming precisely, because the failure mode is not laziness and is not ignorance. It is that breadth-first feels generous.

Every additional topic I put into CG 501 was defensible on its own. Hopfield networks are the right way to connect neural computation to human memory for a cognitive science audience. Model evaluation must be taught or the students will believe accuracy scores. kNN is the cheapest possible demonstration that a model can be a dataset. Game theory was not in the original syllabus at all and I added it, with my supervisor's permission, because collective decision making is genuinely central to cognition and genuinely absent from the curriculum. I still think every one of those calls was correct in isolation.

And that is exactly the trap. The failure is not visible at any individual decision point. It is only visible in aggregate — in the fact that eleven lectures had to absorb every one of those defensible additions — and aggregate is the view you do not have while you are inside the semester writing next week's slides on Sunday night.

There is one specific thing that makes the classroom version worse than the research version. When I over-read for eight months, I paid the cost. When I taught breadth-first, twenty students paid it, and they paid it in a required course they could not opt out of.

The opinion

"Breadth-first" and "too fast" are the same policy described from the two ends of it, and the instructor is structurally the person least able to tell them apart.

The reason is a difference in what the two parties are counting. From the front of the room I was counting topics covered, and eleven lectures covering two fields is an impressive count. From the seats they were counting topics understood, and that count runs on a different clock entirely — one governed by how long it takes to go from seeing backpropagation derived to being able to implement it, which is not a quantity I could observe and was not measuring.

I had also mistaken my own experience of the material for theirs. I was a PhD student who had implemented neural networks from scratch; the perceptron lecture was, to me, a warm-up. The evaluation contains the correction. Fundamental principles well emphasised: 2.30, the joint-lowest teaching item on the form, with one respondent giving it a 0. That item is precisely the difference between covering a fundamental and landing it, and it is the item I scored worst on.

I will defend the breadth itself. A cognitive science student who has seen game theory exists in a wider intellectual space than one who has not, and several of the students said as much — "exposure to new topics", "there was applicability", "application of concepts in projects" were what they named as the good part. The game theory module, the part I invented, drew the only unprompted enthusiasm in the free text.

What I will not defend is having chosen the breadth without pricing it. Adding game theory to a course does not add lecture hours to the semester. It takes them from somewhere, and where it took them from was the neural network block, and five of ten students independently located the wound.

What I would do differently

Cut one field. Teach either machine learning or game theory in eleven lectures, not both, and teach it slowly enough that the fundamentals item scores above 3.

I am aware this is easy to say seven years later, and that the version of me writing slides on Sunday nights while running headphone experiments would have found the argument for including everything just as compelling as he did at the time. The honest summary is not that I have solved this. It is that I have now seen the same failure twice in two unrelated domains, which is the minimum evidence required to call something a personal failure mode rather than a situational mistake.

The forcing function I would want is the same one I now use in research: state the single thing a student should be able to do at the end, in one sentence, before writing the first lecture. Not the topics they should have seen. The capability they should have. "Seen the perceptron, the MLP, Hopfield networks, regression, clustering, kNN, and the whole of introductory game theory" is a syllabus. It is not a capability, and it does not terminate.

Caveats

Ten respondents, one course, one instructor, self-reported reflection on my own teaching. The free-text convergence on neural networks is the strongest thing in here — five independent anonymous responses naming the same block, with zero naming anything as too slow, is not a pattern I can talk myself out of. Everything else is interpretation, including my reading of why the game theory module fared better, which has an obvious alternative explanation I cannot rule out: it came second, when I had more preparation time and had already made my worst mistakes on somebody else.