Reclaiming impact: Not-yetness as a lens for resisting de-complexification [#InnovateOSU]

Impact. Defined as: A force exerted by something physical, or by a new concept, idea, or technology. A striking of one thing against another. An impinging. Influence. Effect.

In education, the term impact has been co-opted to represent a problematic paradigm for education. It’s educational positivism—looking for strong causal relationships between input and output, and clearly expressed metrics for those outputs, also called outcomes.

When I saw that this conference was about Impact, I looked at the description for this conference and was relieved to see a broad definition used for impact. Unfortunately, too often we see impact negotiated in terms of what can be observed “objectively” (quantitatively), analyzed, and reported in tidy ways. I want to spend a bit of time talking about what the impact as evidence mentality represents. At its roots are deeper political and social forces that we don’t have time to get into today. Instead I’ll allude to some of those forces, but talk specifically about how they play out problematically in education. The goal here is to push back on the ways “impact” gets narrowly constructed and to introduce a concept—not-yetness—that can help to broaden and complexify what impact means to us.

I’ll start with a confession: I was once the director of a Center for Excellence in Teaching and Learning. I know, I’m sorry.

In fairness, it was a truly wonderful job. I had a great team who loved to play practical jokes on me [like this, and this, and my reaction to the latter here]…and took a creative approach to faculty development. The faculty with whom we worked were committed to their students. But I always chaffed on the Excellence part of the center’s title. Excellence is a word that means very little but is used to reward or punish teachers, students, and institutions. It’s a word that gets applied as if it means a single thing and therefore forecloses conversations about diverse approaches to education, diverse students, and diverse purposes and impacts of education. We fetishize excellence (see Moore et al., 2016), even when we’re not sure what excellence means. And worse, we use it as a way to motivate, but we rarely question who gets to decide what is excellence, and for what purpose is it excellent?

Here are some other words that do the same and, not coincidentally, these are words you often see or hear when people talk about impact. “What works.” Best practices. Performance. Efficiency. Measurement. Evidence-based. Scale.

I’m not trying to paint these words as bad, or even the goal of finding impact as bad. It’s not that we don’t want impact. We want it, especially when the definition of impact is broad, open, inclusive. We want lasting and meaningful impact. But the ways we are talking about impact in higher education are founded on some underlying values that I think many of us would resist, or at the very least, question.

These words represent education’s Age of Accountability. The Age of Accountability began in the 1980s and has continued as the cost of education soars, and as education has become commoditized, expectations about the impact of education have becomes dictated by “what the market wants” and what can be accounted for or measured. Gert Biesta talked about the impact this has on the process of education (what happens in the classroom, for example) and on educational research (what we understand about what happens in the classroom).

In his article “Why What Works Won’t Work” and in the follow up, “Why What Works Still Won’t Work,” Gert Biesta argues that the consequence of the accountability movement in the US, which have been more pronounced than in other countries, “radically changed the landscape of educational research. Although the idea that research in education should be able to tell us ‘what works’ had already been articulated in the 1980s, it was not until the late 1990s that this way of thinking began to shape legislation regulating federal research funding. Since the reauthorization in 2001 of the Elementary and Secondary Education Act (‘No Child Left Behind’), the ‘gold standard’ of randomized controlled field trials seems to have become the preferred — if not prescribed — methodology for educational research” (Biesta, 2007).

Biesta also critiqued the impact of the accountability on the educational process. He calls this the “learnification” of education, represented by a shift of focus from teaching to learning. This shift is largely seen as a good thing—it’s hard not to. Of course we want our students to have experiences of learning. With learnification, Biesta says, education is being instrumented to make clear connections between inputs and outputs, and to strip content, context, purpose, and relationships from education.

Learnification requires decomplexification—so that you can see outcomes that are based on inputs, and then do the ‘gold standard’ research and/or assessment to tell you how to improve and adjust inputs accordingly. Complexity is not welcome in this equation. Education becomes about simplicity and tidiness, risk-free-ness, and the notion that instruction can be made better by assembling and deploying content, techniques, and approaches endorsed by unproblematic evidence.

We’re creating a perfectly controlled system of education. We say: “Write learning outcomes that can be measured. Then design the learning activities to reach to those outcomes. Then assess based on those outcomes. Then certify those assessments.” And we research the entire process to look for ways the specific thing we are doing can be improved. Rinse and repeat. Rinse and repeat. We create a recursive loop that implements, looks for, and validates the things that we implement, look for, and validate. Funding then, gets dedicated primarily to things within that loop; technology focuses on efficiencies and improvements of that loop. It’s controlled. It’s risk free.

Noel Gough (2010) says that our focus on assessment shows the recursive nature of the input-output relationship: “Complexity reduction in education not only happens prospectively (through the reduction of initial variables) but also retrospectively (through backwards selection of particular trajectories). One of the most explicit examples of retrospective complexity reduction in education is assessment, because assessment validates some learning trajectories and invalidates others but always does so ‘after the event.’ Because education is a recursive system, the anticipation of assessment also reduces complexity. In this way assessment also functions prospectively in the reduction of complexity” (pp. 9-10).

This condenses instruction to a series of decisions, devoid of professional judgement and empathy—the determining of what students will learn, think, and do, and the measurement of outcomes thought to be based on the effects of these determinations. As Rajiv said in his presentation earlier, “Don’t you think it’s strange that we craft learning outcomes before we ever meet a student in that class?” It forecloses conversations about the purpose of education, for whom and for what is education, and who gets to make those decisions on behalf of students.

A focus on certification of performance means assessment. And assessment emphasizes measurable learning outcomes. Jen and I wrote, “Gardner Campbell (2014) argued that the push toward evidence does not necessarily result in responsible action or marked improvements in learning, instead it rewards narrow conceptions of what it means to be a good student, ‘marching toward compliance and away from more elusive and disruptive concepts like curiosity or wonder’.” This is my concern with learning outcomes, with assessment, as we currently conceive them. This is what we mean by learnification.

In other words, the impact/accountability movement has had the effect of “1) narrowing our views of what education is, thus ignoring more humanistic, holistic, critical, and speculative approaches to education; and 2) narrowing our views of how we understand education and what counts as valid research on learning, thus ignoring more humanistic, holistic, critical, and speculative approaches to education.”

So, my concerns are that the current accountability movement in education, often reflected in words like impact and outcomes are that:

1) It creates a limited view of education, and discourages risk-taking and open-endedness

2) It discourages things that cannot be measured

My colleague Kristen Eshleman, in a recent episode of the podcast, Teaching in Higher Ed, wondered “are we only going to end up designing learning for the things that can be measured? This is a legitimate concern. And does the focus on only what is measurable make Campbell’s Law more likely—that on which we focus becomes what we manipulate to our benefit? What gets missed in learning an education when we refuse to acknowledge learning that is not imminently measurable? What happens to learning when all we acknowledge is that which is imminently measurable?

3) Forecloses conversations about who gets to decide what is measured, what is excellence/appropriate impact.

In foreclosuring the conversation about impact and outcomes, we are not addressing the way dominant values (white, cis, male values) are pressed upon our understanding of what’s desirable. Jim Scheurich and Michelle Young (1997) write that “all of the epistemologies currently legitimated in [Euro-American] education arise exclusively out of the social history of the dominant White race” (p. 8). Noel Gough (2010), citing their paper adds, “This restricts the range of possible epistemologies and methodologies available to us, and makes non-dominant constructions of knowledge suspect, pathological, sensational, or simply illegitimate. Thus, the extension of consensus techniques to broader and more culturally inclusive publics might be one way to generate multiple alternative futures in educational inquiry.”

A narrow focus on impact de-complexifies education and moves it away from the liberatory power of “education as a practice of freedom,” (bell hooks). hooks says, “education as the practice of freedom affirms healthy self esteem in students as it promotes their capacity to be aware and live consciously. It teaches them to reflect and act in ways that further self-actualization, rather than conformity to the status quo.” This is complex education, but the kind of education we really need today.

We lose a lot with the current impact perspective. As Kristen Eshleman asks, “If outcomes-based assessment doesn’t account for other domains we value, is there a method that can?” 

Now wait…did you notice that I haven’t really talked about technology yet? I think it’s appropriate to remember that technology often adopts, affirms, and amplifies our values. The role technology plays in much of this is to affirm and amplify the de-complexification of education: the clear relationship between the inputs and outputs, leading students down so-called personalized/individualized paths to the same place.

Not only are the technologies we are using in the classroom largely about de-complexification, the data and computational methods we use on that data further implicate a limited understanding of “impact.” I want to say that again: I am not saying we should discard entirely the quantitative notions of impact, the linear and mechanistic ones, but we should situate them in a greater understanding of impact, not a narrower one.

But there are ways technology COULD help us to push back on the de-complexification and to embrace complexity in education.

A few years ago, I began thinking about “the digital” and complexity when my colleague Jen Ross and I were preparing a talk for the Emerging Technologies for Online Learning conference, now (hilariously) also called Innovate. We were talking about embracing the complexity and mess that emerging technologies can bring and we mentioned how emerging technologies have not-yetness. The word not-yetness draws from George Veletsianos’ description of emerging technologies as “not yet fully understood” and “not yet fully researched.” Rather than thinking of education and of impact in narrow terms, not-yetness can help us to embrace the fact that there are parts of education and impacts of education that are not yet fully understood or researched. Those things shouldn’t be devalued. They should be sought, cared for, and valued. Digital tools, environments, and practices can bring more complexity and open up conversations that we may not have had before. “We must therefore choose to dwell as teachers in a state of radical and enduring uncertainty.”

State of radical and enduring uncertainty. Surprise. Emergence. Speculation. These are the word and ideas that accompany not-yetness. With these ideas, the notion of impact starts to take on broader understandings.

My colleague Jen Ross wrote last year about speculative methods in research on digital education, all of which she ties to this idea of not-yetness. Speculative methods are chiefly found in design, social sciences, and art. Ross (2016) writes that they are “are aimed at envisioning or crafting futures or conditions which may not yet currently exist, to provoke new ways of thinking and to bring particular ideas or issues into focus…Speculative approaches can bring new practices and ideas into being while maintaining space for curiosity, critique, doubt, unintended consequences and emergent properties of technologies in use. They have a quality of participant-observation, interweaving insider and outside perspectives, and they are by necessity ‘baroque’ methods, as described by MacLure: ‘resist[ing] clarity, mastery and the single point of view [and] be[ing] radically uncertain about scale, boundaries, and coherence.”

This notion of crafting futures intrigues me. I went back to some of the literature Jen cited and found a Gough (2010) paper about “possible futures,” which he defines as “anything that we can imagine is possible (including that which seems ‘impossible’ in the light of knowledge that we presently hold to be ‘certain’), although some possibilities will appear to be more plausible than others.” He distinguishes possible futures from probable futures, which are things that are likely to happen, and preferred futures. “Since one purpose of exploring futures is to improve upon past and present policies and practices, we should seek to elucidate preferred futures by imagining and exploring the implications of possible alternatives rather than by choosing among those alternatives which might now seem most probable.”

He goes on:

“But futures exist in human minds and, thus, in an objective sense they are never ‘out there’ but, rather, are always here, now. Recognising that futures are components of present action and existence liberates the critical and creative imagination. It allows us to explore longer time frames than those usually dared by empiricists and, unlike those who are concerned with prediction and control, to explore possible futures without colonising them. Thus, the types of futures study which can expand the horizons of education are, perhaps paradoxically, located firmly in our present consciousness, in critical reflection on the concepts, values, meanings, images and metaphors that we use to navigate our ongoing journey through time.”

Not-yetness, as part of a broader view of impact is speculative. It considers possible futures, maybe preferred futures, and it explores borderlands. Often when I talk about not-yetness, I’m reminded of Gloria Anzaldua’s poignant piece on the Borderlands/La Frontera. She writes about the physical border between the US and Mexico, but also the emotional and psychological border of her mestiza heritage. For Anzaldua, it is the space around the border, the borderlands, the undetermined spaces caused by the boundaries that are interesting. For her, they were places of both fear and uncertainty, but also great cultural richness. It is the not-yetness of these borders, and the traversing across borderlands that we find fruitful.

At the core, I wonder, can not-yetness help us to find love in education? Could love be an impact we seek?

I often reference Gardner Campbell’s Taxonomy of Student Engagement. Here, I am taking a couple of liberties and changing it to Taxonomy of Student Impact. My apologies, sir Campbell.

At the very bottom of the impact taxonomy, compliance. You know that so much of our impact metrics and technological solutions to encourage those metrics is about some version of compliance. That’s what the accountability movement is all about.

But then there’s interest. I wonder, can we begin to think of educational experiences as creating a foundation for students’ interest, that will drive their further learning outside of the classroom?

What about connection? We heard today from our keynote, Kathryn Finney, that connection to mentors was an essential and often missing part of the educational experience of her program participants. Connection is a kind of life-long impact that we hope students will have right? So how do we think about impact in those terms, looking for connections students make?

And finally, love. Can we start to embrace the complexities that will come when the impact we seek in education is love? What are possible futures when love is the ultimate impact? Love for topics, love for learning, love for others. My god, we need more love for others, empathy, listening, all of that, these days. And yes, I know I lose people when I start talking about love. But it’s maybe more important than we give any credit to. Love means advocacy, and sponsorship, and support, and caring, and mutuality, and continuity, longevity.

So as Biesta, I wonder, how can our seeking of impact help us to find an “understanding of possibilities,” not outcomes? And based on our and our students’ broader understanding of impact, can we seek possible futures that are reflective of who our students are, what they need and want in life, and what they already bring to the educational experience?

How do you design for this, and how to you reflect this complexity within an architecture that is hostile to things outside the narrow configurations of understanding?

These are the challenges we need to take on as educators.

I want to end with witness marks. In the podcast episode I referenced earlier, Bonnie Stachowiak referenced the section on witness marks in another incredible podcast, S-Town. Witness marks are marks on an object, in this case on antique clocks, that show the history of that clock’s inner workings and its repairs/alterations. They provide clues/information about the character of the clock and of the clock maker or restorer. Bonnie notes that witness marks may connect to how we think about the impact of education on students and Kristen notes the importance of thinking of students’ lived experiences (including their educational experiences) as witness marks. I think Bonnie and Kristen are spot-on and maybe witness marks is an analogy that can help us break free from our outcomes orientation. I hope so.

In the meantime, go listen to those podcasts!