A Case Against Data-Based Decision-Making
“Data alone cannot decide which path one follows. And if one chooses to follow the wrong path, they can be sure that data-based decision making will expedite their journey down that path.”
Data, unfortunately, is not our savior. It is not the oracle we think it is.
Data’s veneration in today’s society is thoroughly, tragically misplaced. Vast edifices of data, monuments to the exalted quantifiable, are erected again and again in the data centers of feudally-powerful tech conglomerates, scrappy AI upstarts with multibillion-dollar valuations, and state intelligence agencies. When data of this quantity–and intimacy–is put to use, its outputs mimic reality so closely as to be stomach-churning. One feels exposed; are we so easily predicted?
Algorithms equipped with staggering amounts of data and their very own pedagogical processes construct facsimiles of reality from thin air (and billions of tons of liquid coolant) that, in their resemblance to real life, are impressive if not exactly inspiring. The constructions of these algorithms–high school and college essays, Seldon-esque psychohistorical election predictions, 3D animation and porn–mimic the forms of their references faithfully enough to leapfrog the uncanny valley and provide passability for one’s purposes, even if they are not always entirely convincing as the real thing. In this fashion, data creates useful tools, which can occasionally be insightful, and are good in a pinch.
When data–quantified representations of reality in informational form–is overlaid atop reality, its clean, discrete matrices can help simplify real-world problems. In the micro, data provides scaffolding–artificial guardrails whose parametric handholds constrain human thought in a manner conducive to the making of decisions. However, when the discarnate stencil of the quantified is applied in systematic fashion to reality, the understanding of an interconnected whole–and often the acknowledgement of nuance–is sacrificed at the altar of outcome optimization.
In an example of this flavor of sacrifice, data-driven agricultural methods based on maximizing output are used to discover and employ the optimal spacing grids and nutrient schedules for monoculture crops like soybeans, corn, or sugar beets. With these methods, farmers and Big Ag corporations can maximize the yield–measured in pounds, tons, or dollars–of their crops, at the catastrophic expense of long-term soil health, which is considered only as a variable to be kept to minimum functional levels. This is data-based decision-making in action.
In schools, my primary area of interest, educational specialists use Pavlovian behaviorist techniques like classical reward-and-punishment conditioning and objectivist perspectives which allow the intricacies of students’ lives to be reduced to variables in an equation. Such variables are manipulated when possible to optimize for specific outcomes without the need of the student’s equal participation, and at the expense of their autonomy. The counterargument to these norms is deceivingly hard to make. When autonomy means throwing chairs and running out of classrooms, maybe it’s not a bad idea to manipulate their environment to minimize these occurrences. I would even tend to agree. Besides, who wouldn’t want to optimize educational outcomes? In these scenarios, the presentation of downward-sloping line graphs of chair-throwing incidents can be especially convincing.
Unfortunately, these methods are ultimately limited in that they answer no question of direction. In a broad sense they are responsive to a perceived immediate need, but their logic contains no philosophy, no direction, beyond “more of this” or “less of that”. Data can say very little regarding a student’s inner attitude toward learning, nor their capacity for critical thinking. Every time we educators pull out our protractors and our steel compasses in an attempt to measure a student’s progress toward some goal, we set aside consideration for the ultimate aims of education and constrain learning by holding down the writhing young mind to take its measurements.
Though it was not always this way, modern educators often experience existential crises when faced with the abolition of measures of success–namely grades. Emerging in the 18th century as a means of publicly ranking and broadcasting student “achievement” and perfected in Stanford University think tanks as a way of ensuring that the brightest, whitest young minds would be sat in officers’ chairs instead of lost to the front lines of battle1, the letter-grading system has entrenched itself so deeply in our conceptions of education as to feel inextricable from learning itself. It is ironic that a concept designed with the express purpose of creating, perpetuating, and providing a rational basis for inequity remains an untouchable object of dispute for educators who spend their careers fighting for equity in schools.
The concept of data-based decision-making (DBDM) in schools arose as a response to the inequity that grew so blatantly sinister in the wake of the Civil Rights movement. Before this framework, educators and school administrators were commonly marrying strict regimes of student evaluation (still present today) and unilaterally-derived codes of school conduct with unchecked racism, ableism, and bias, which essentially weaponized school discipline and special education programs against the most marginalized populations. DBDM improved accountability, illuminating the grotesque statistical disparities resulting from this style of school administration. Shocking incarceration rates for students for students who have been suspended or expelled, profound over-identification of Black and Brown students in special education programs (many of which sanction segregation from majority-white general education populations), are just two examples from an array of injustices. The previous regime featured mostly-white school administrators acting without accountability, leveraging structures in schools like exclusionary (and previously, corporeal) punishment and special education to enact their biases. Data turned a mirror to the practice, and played a vindicating role in the educational zeitgeist for those who witnessed this injustice and those who were subject to it. Data ultimately echoed their pleas, and it was data that was listened to when their voices were ignored.
You may be tempted, as many in mainstream education are, to profess data as a savior, a looking-glass through which all inequity can be identified and systematically minimized. Maybe you see it as the way forward; many people do. I do not. I do not, for the critical reason that DBDM, at its best, is algorithmic. Identify some phenomenon, flatten it to make it quantifiable, track progress, fiddle with variable knobs to optimize quantitative outcomes. Document every detail for posterity, or to cover your posterior. There is no room, no consideration, for learning in this framework, precisely because the unquantifiable cannot be quantified. The transcendent magic of discovery is taken to no account. When educators and specialists like school psychologists think of “enthusiasm for learning”, it is telling that our minds go immediately to evidence-based self-report surveys of the variable of the same name. DBDM promotes the phenomenon articulated so succinctly by the artist billy woods:
“Over time, symbols eclipse the things they symbolize”
Education’s path into the unstable, uncertain future will not, must not, be charted by data. The path may be paved with it, but it will be human ideas, human values, understandings of what is important, and human decisions that will chart its course. Conceiving data-based thinking as a guiding light, as anything more than a useful tool for simplifying what are ultimately small-scale problems, is to zombify society and recruit it in service of the ever-manipulable, often-unreliable indifferent narration of data. In the example of schools perpetuating inequity I described earlier, a data-based lens identifies accountability and non-transparency as the issues of focus. In my opinion, the real driver of said inequity, what made it possible in the first place, is the power structure baked into schools that allows administrators to suspend, expel, and make educational decisions for children–indeed to control their movements for seven hours a day and evaluate their adherence to measurable standards of success–without the equal participation of the communities that they are ostensibly a part of, not just the ones they “serve”.
Immediate-community-based (i.e. not state- or nationally-based) curriculum; education based on the cultivation of students’ individual interests and abilities; equal participation of students in education (compulsory education is not voluntary, and therefore not equal): these are just some of the educational principles that our time in the world calls for. How we get there will be the deeply human and profoundly unquantifiable work of conversation, challenging ideas, and decisions. Data will not show us the way.
For more on this, read Palo Alto: A History of California, Capitalism, and the World by Malcolm Harris.