There was an eye-catching article in the WSJ this weekend: “Online Education: My Teacher Is an App.”

The author was looking at online public schools as an alternative to the traditional classroom model. The article spends a good bit space comparing test scores between online and classroom students and making note of the consistently lower test scores of the online students. Interestingly, the discussion of test scores follows a number of comments from student who discuss how much they are getting out of online learning. So either the students are dead wrong about how much they are learning (possible), or testing isn’t necessarily the best measure of learning (also possible) Both may be true; they are not mutually exclusive, but….

I found the article a bit cringe-inducing for a simple reason: it’s missing the point.

Test Scores are not Learning.

Even when I was in grade school I could see through standardized testing; the weaknesses were quite visible. I usually had extremely high marks on my tests, so my beef wasn’t against testing per se – it made me look good. But I also wasn’t stupid. I knew full well who was cleverer than I was in ways that mattered, often clever in ways that would involve creating things (what we would now call “innovation” ,but at 12, I didn’t really toss that word around a lot, I just knew what I saw). But some of the innovators didn’t necessarily score well on tests.

Test Scores are not Ability.

In college, I would study in the labs with a fellow student working through samples and discussing processes. His depth of understanding would blow me away, but come test time, he’d get a C, and I’d get an A.

Test Scores are not Mastery.

It may be that the online students had lower standardized test scores because they haven’t learned as much as classroom students. But it might also be that that they scored lower because they learned more. It’s possible that they were busy learning, discovering… while their classroom counterparts were being taught how to succeed on tests.

Ultimately, though, the whole conversation in this article points to something deeper than test scores. The reliance on standardized tests as the basis of measuring learning (and structuring the educational system) is a symptom of a bigger issue: the overlying assumption that learning is something which can generally be standardized, and which is scalable.

What can we validly test at a massive scale? Standardized tests can do a decent job of measuring basic skills: things like the ability to do arithmetic, and perhaps the ability to read with comprehension, to write a coherent sentence and paragraph? Those are foundational tools for any job or just living life (balance the accounts, read an email from your boss, write a thank you letter…).

Skills are good, even necessary. But there’s a difference between using standardized tests to verify skills, and using them to define education.

Learning is not inherently scaleable** nor stadardize-able. Evolving technology can either be used to build even more extensive cookie-cutter approaches to education, or it could be a golden ticket to something better – a chance to allow for a more human scale of learning; supporting the creation of something quite remarkable – allowing something more than standardization. I’ve been playing with this a lot lately. So much in fact that it’s becoming a major focus of my work.

I don’t see myself as taking on education reform as a whole – that’s far too complex and extensive a problem for my abilities. Instead I’m playing with ideas like narratives, and games (meaning world-building, not gamification); looking at how digital and tactile environments affect learning and how they can work together; and I’m using lots of very bad, and highly labored, analogies. You’re welcome to come join in. (Website going live very soon.)

**Looking at learning in its uber-scaled, standardized test format, compared to what learning can be, is rather like the difference between a strawberry milkshake at the local fast food emporium as opposed to fresh strawberries and cream at the local cafe. The same base ingredients, but what a difference in the delivery. Uniform, prefrabricated goo versus the lovely individual dish, with it’s mix of flavors, colours and textures. I think it’s pretty clear that I’m in the “strawberries and cream” camp on this one.