Living with uncertainty and the measurement trap
24/08/18 11:53
Perhaps it's because we're uncomfortable living with uncertainty that we've been lured into the measurement trap.
In any redefinition of school quality and purpose, the measurements around quality and purpose, as well as the forms of measurement used, need to be taken carefully so as to reflect that change. ‘Don’t value what you measure, measure what you value’ may be an old adage, but is one that is often neglected in the world of education today. Measurement is important, and data is really valuable, but the purpose needs to be clear. The role of data, properly understood, is not to provide definitive answers but to support the asking of powerful questions.
A measurement is an observation that quantitatively reduces uncertainty. Measurements cannot yield precise, certain results, though they can help to reduce uncertainty.
The object of measurement must be described clearly, in terms of what we are seeking to observe. Even if you cannot measure exactly what you want, you can learn about your area of interest with related data. A business may not be able to measure the exact benefit of a happy customer, for example, but it could get measures which give evidence of the value and magnitude of its work. It could also get measures of the cost of dissatisfied customers.
But with all measures, you must use judgement. The danger is that we often mistake the measure for the thing itself. Measures are a proxy, and we need to understand the limitations of the data we use.
We should not just pretend that the data we have tells us everything we need to know. We need to think and ask powerful questions. We need to understand. And we need to exercise our judgement.
Perhaps that is what we're really uncomfortable with.
In any redefinition of school quality and purpose, the measurements around quality and purpose, as well as the forms of measurement used, need to be taken carefully so as to reflect that change. ‘Don’t value what you measure, measure what you value’ may be an old adage, but is one that is often neglected in the world of education today. Measurement is important, and data is really valuable, but the purpose needs to be clear. The role of data, properly understood, is not to provide definitive answers but to support the asking of powerful questions.
A measurement is an observation that quantitatively reduces uncertainty. Measurements cannot yield precise, certain results, though they can help to reduce uncertainty.
The object of measurement must be described clearly, in terms of what we are seeking to observe. Even if you cannot measure exactly what you want, you can learn about your area of interest with related data. A business may not be able to measure the exact benefit of a happy customer, for example, but it could get measures which give evidence of the value and magnitude of its work. It could also get measures of the cost of dissatisfied customers.
But with all measures, you must use judgement. The danger is that we often mistake the measure for the thing itself. Measures are a proxy, and we need to understand the limitations of the data we use.
We should not just pretend that the data we have tells us everything we need to know. We need to think and ask powerful questions. We need to understand. And we need to exercise our judgement.
Perhaps that is what we're really uncomfortable with.