What about qualitative research in the “new data science of learning?” #digped

An email I once received from a graduate student at Stanford:

“There was also some talk last week about starting a Qualitative Methods Special Interest Group (haha almost wrote support group)…” [emphasis mine]

Support group indeed. I will send a shout-out on Twitter to the person who comes up with the best name for such a group. Use #qualsupportgroup 3…2…1….GO!

Okay, in all seriousness, for many of us trained in qualitative and critical traditions in social sciences, the quantitative moment we are seeing in higher education is unsettling.

I recently gave a talk with Jesse Stommel at the Digital Pedagogy Lab Institute. We talked about emergence and how making space for emergence and complexity in education is exciting and needed more than ever. The pressures of accountability, combined with continued policy moves and budget cuts, are creating an increasingly untenable environment for the pleasure of learning, the openness and exploration that comes with viewing “education as a practice of freedom,” (bell hooks) rather than a practice of calculating “value.”

I spoke for a bit about how the accountability movement and emphasis on “evidence-based teaching” has had the effect of 1) narrowing our views of what learning is, thus ignoring more humanistic, holistic, critical, and speculative approaches to learning; and 2) narrowing our views of what learning looks like and what counts as valid research on learning, thus ignoring more humanistic, holistic, critical, and speculative approaches (stay tuned for a paper from Jen Ross on speculative research methods and not-yetness!). Maggie Maclure calls the push for evidence-based education “animated by the desire for certainty, willing to sacrifice complexity and diversity for ‘harder’ evidence and the global tournament of standards.” The push for “harder evidence” often pushes out the kinds of learning and evidence that come from post-structural, phenomenological, and critical approaches. For example, the Evidence-Based Teachers Network proclaims that studies on teaching and learning involving few students, no comparison groups, or only one teacher constitute poor evidence. After all, randomized controlled experiments are the “gold standard”—why would we accept anything else?

We lose a lot with the “gold standard” perspective. What about learning that is mildly informed by course topic but is essential to a human being’s view of how to be a contributor in a public sphere? What about learning that is more tied to a person’s identity that to curriculum standards? What about learning that is the result of a mentoring relationship that helps a learner to find the drive to pursue an educated life rather than an education? What about the learning that happens when the chemistry of a particular class leads to places of learning a teacher could never predict? Does this learning not matter? Does it not matter because it does not “count” (in a gold-standard-only kind of way) as learning?

Ken Morrison says the evidence focus pushes researchers and teachers “to atomize phenomena into measurable variables and then to focus only on certain of these [which] is to miss synergies and the significance of the whole.” What gets missed in learning an education when we refuse to acknowledge learning that is not imminently measurable? What happens to learning when all we acknowledge is that which is imminently measurable? As Gardner Campbell (2014) argues, the push toward evidence does not necessarily result in responsible action or marked improvements in learning, instead it rewards narrow conceptions of what it means to be a good student, ‘marching toward compliance and away from more elusive and disruptive concepts like curiosity or wonder’.

Why does this matter? Because we have people like Kevin Carey telling people whose decisions (and funds) impact the future of education that our schools are failing because we have not been able to produce true “evidence” of what he thinks education should look like. And Carey is just a mouthpiece for a much broader movement questioning the value of education and calling for more “evidence” to prove its value (see A Nation at Risk, for example). As Jan Morse says, “Evidence is not just something that is out there. Evidence has to be produced, constructed, represented. Furthermore, the politics of evidence cannot be separated from the ethics of evidence” (cited in Norman Denzin’s The Qualitative Manifesto). Jan McArthur notes that the politics of evidence are tied to the politics of complexity reduction, which is when we try to remove ambiguities, complexities, and indeterminants from our understanding of a social phenomenon for the purposes of what she calls “wicked clarity.” In educational research, approaches that rely only on tidy conceptions of what learning is and looks like because they are rooted in views that negate anything that is not captured by experimental research trials, are potentially sweeping under the rug messy but essential social issues like systematic inequities and the role of diverse lived experiences in learning.

Here is where qualitative and critical lenses can help. Not because the answer is “no standards—it’s a free for all” or “replace all empirical research with qualitative.” C’mon people. Upon hearing my views on learning and research, a man sitting next to me on a plane last week (a retired & now consulting K12 administrator) said, “I spent my whole career trying to get teachers to write specific outcomes and now you’re telling me I should have let it be a free for all?” It’s easy to take a polarized approach—the hard work is figuring out how to bring these perspectives and approaches to the same table and value equally what each brings.

It does not have to be an either/or. It doesn’t have to be all qualitative or all quantitative. It does not have to be qualitative only in service of adding layers of description to empirical research. It can be qualitative leading the way to our asking better questions, developing empathy, and deeply understanding what matters in education.

Qualitative and critical lenses remind us that humans are involved in education and their lived experiences matter. Qualitative and critical lenses remind us that there is no such thing as value-free learning or research. All inquiry is value-laden, and these lenses acknowledge how power, identity, performance, bodies, culture, and biases play into every part of the process, into every research question, every sample, every algorithm, every interpretation. And qualitative and critical lenses recognize that everything is situated in contexts and bodies, and context and bodies matter (Audrey Watters and Tressie McMillan Cottom).

The week Jesse and I gave the talk to #digped, and in a funny convergence of events, EDUCAUSE released a video about measuring learning in which I make a statement about the “magic” of learning. I’ll admit that I somewhat question my phrasing of learning as “magic;” perhaps with more consideration I would have said “magical.” I keep asking myself, “what was I trying to say? What would I want people to take away aside from a catch phrase about learning?” I think it’s that we are losing the wonder of learning. Maybe it’s not magic, but can be unexpected, unpredictable, wild, transformative, liberatory, life-changing, and freeing. Qualitative approaches can help us to understand and care for that kind of learning.

That matters.