There’s an old joke about two young fish who are swimming along, and come across an older fish. The older fish asks, “Hey boys, how’s the water?” to which the younger fish both reply that it's great. As they swim along, one of the younger fish asks the other, “What’s water?”
It can be difficult for us to see and recognize our environment for what it is, because whatever it is seems normal to us. In fact, it is often normal to the point of invisibility. This is true for our built environment (like the fish), the way our relationships work, or the ways that we look at, interpret, and understand the world around us– our mental models. This is not to say that what is invisible is wrong or bad (though sometimes it is, think Copernicus and the sun), but rather that things influence us in ways we do not even realize we are being influenced.
Quality Metrics
Walk into a doctor’s office (not mine, but most), and there are going to be a huge number of things that are tracked during the visit. Was blood pressure or weight up? If so, was there a response to that? Was the response documented appropriately? Were medications reviewed and reconciled? Was there a mechanism in place to screen for depression? Was it elevated? Was there a response to that elevation? Was smoking status inquired about, and if positive, was there counseling done to quit? These metrics are tracked, compiled, organized, and then used to determine the quality of the care that was provided. The metrics do not care if you come in for a broken bone, a broken heart, or a broken mind.
Recognizing that perhaps this can miss the point of, ahem, the reason for going to the doctor, another level of measurement has been added. You’ll likely receive a “Patient satisfaction survey” following the visit, which will have another long series of questions about the visit, whether your issue was addressed, and whether you would recommend the person you just saw.
This idea of measuring, tracking, and improving extends far beyond healthcare. In schools, test scores are gathered, tracked, and used to determine quality and drive educational policy direction, evaluate teacher effectiveness, and monitor student progress. We monitor our heart rate, our sleep score, our weight, and our macronutrient intake in an effort to improve our health. We track our spending, our income, our investments, our returns, and our savings to maximize our wealth. This may get operationalized in dashboards and key performance indicators that are used as guideposts in decision-making. We do this in agriculture, in business, in law, in public policy, in government.
Tracking Everything!
This type of measurement and tracking is ubiquitous because it is helpful and effective. This approach reveals patterns that were not otherwise apparent, helps us view change over time, and understand what is working and what is not working. In large organizations, this allows diverse parts of the organization to communicate efficiently, and allows leaders to see what is happening and where problems arise. As the adage goes, what is measured, matters. When we focus on measuring performance, we improve performance.
Because of the effectiveness of this model, we sometimes forget that it is a model at all. Like all ways of viewing the world, it highlights some things and obscures others. It is easy to lose sight of the fact that this approach is a tool, with use cases, advantages, and disadvantages, rather than the thing itself. Because of how common it is, it can become the water we swim in.
Any model or system that we use regularly shapes our thinking. My view of the world is shaped by my medical training. If I worked as a banker, that too would shape my thinking. The quantification of the world shapes our thinking in a similar way. We learn to see in terms of what we can measure, track, and improve. Doing so seems not just possible, but intuitive, and self-evidently helpful. To the degree that it is not, it is a problem with not constructing the right metrics, not getting the incentives right, not looking at the right data.
Tracking Everything?
As useful as this approach is though, many things are not amenable to it. Because this perspective has deeply influenced our view of the world, the way we make sense of things, and how we approach solving problems, we often fail to recognize when this does not work. We do not consciously think about the limitations of this model, do not remind ourselves of what assumptions must be made in order for this approach to work. We have adopted a way of understanding the world that is so familiar and so successful in many contexts that we no longer even recognize it as a particular way of understanding the world, as it presents itself as common sense.
Imagine telling someone that you want to spend four years studying philosophy or devote your time to making art. What will you do with the degree? What will your art accomplish? What value will either create? These are reasonable questions in some contexts, but notice what they assume. They assume that a pursuit should be judged by what it produces, preferably in ways that can be measured. We can estimate the financial return on an engineering degree more easily than we can measure the value of learning to think more carefully, creating something beautiful, or seeing the world differently. We can point to the economic or therapeutic benefits of art, or the practical benefits of studying philosophy, but those explanations miss something essential: art can be worth making and experiencing for its own sake, and philosophy can be worth pursuing even when it leads to no clear practical outcome. When measurable outcomes become the standard by which we judge what makes sense, pursuits whose value cannot be expressed in those terms become harder to justify.
Many things remain difficult to measure– and perhaps that is because they do not belong in the realm of measurement. We can quantify our love for each other on a 5-point scale, but saying I love this person 5 out of 5 does not really capture what love means. We can measure how much it costs to raise a family, but we cannot easily measure the joy and love and satisfaction that comes from raising a child. Measures are a representation, not the thing itself. A test score represents some aspects of learning, a satisfaction score represents some aspects of a patient's experience. When this perspective gets deeply embedded into our worldview, we easily lose sight of the fact that the measurement is not the thing itself, and forget that many things cannot really be measured. And when we recognize these limitations, we tend to see them as problems with our ability to measure, rather than with the larger framework in which measurement operates. Teachers go into teaching to help children learn and to inspire curiosity, not to raise test scores.
As this worldview gets more and more embedded in how we think, what was once a measure or a proxy becomes a target, influencing the behavior or system it was designed to evaluate. People and institutions adapt to the metric. That can improve the underlying outcome, but it can also produce better scores without corresponding improvements in what the scores were meant to represent.
The Cost
There is a human cost to this. When we interact with institutions, we want more than a process completed or a standard met. We want to feel that someone has listened, understood our circumstances, and cared about what happens to us. But metrics do more than fail to capture these things. They shape the people and institutions that rely on them. When meeting targets becomes the definition of doing a good job, people can come to prioritize the targets over the needs they were intended to represent. Institutions organize human activity, and what they measure, reward, and attend to changes how the people within them work and what they value. People working within these systems can lose touch with the purpose of their work, feeling frustrated, demoralized, or disengaged. Those on the receiving end can feel unseen and uncared for, and lose trust in the institutions meant to serve them. Over time, people on both sides of these interactions can become reduced to their roles within the system: the person processing the case and the case being processed, the provider meeting the metric and the patient being measured. The institution may continue to meet its targets and follow its procedures even as the people within it become less able to see the people they serve as people, rather than as cases to process, scores to improve, or targets to meet.
These issues reinforce each other. We measure what we can, give those measures increasing authority, and reorganize our behavior around them. Over time, the measures influence not only what we do but what we notice and value. And because this approach is powerful in so many situations, it comes to seem like common sense. We internalize the approach, and in the process lose sight of many of the things that matter most.
Love,
Doc
Forwarded this email? Sign-up here
See past posts here.