The act of measuring certain sensitive variables in a system can alter them, and confound the accuracy of the measurement.
This principle is based on Heisenberg’s uncertainty principle in physics. Heisenberg’s uncertainty principle states that both the position and momentum of an atomic particle cannot be known because the simple act of measuring either one of them affects the other.
Similarly, the general uncertainty principle states that the act of measuring sensitive variables in any system can alter them, and confound the accuracy of the measurement.
For example, a common method of measuring computer performance is event logging: each event that is performed by the computer is recorded. Event logging increases the visibility of what the computer is doing and how it is performing, but it also consumes computing resources, which interferes with the performance being measured.
The uncertainty introduced by a measure is a function of the sensitivity of variables in a system, and the invasiveness of the measure.
- Sensitivity refers to the ease with which a variable in a system is altered by the measure.
- Invasiveness refers to the amount of interference introduced by the measure.
Generally, the invasiveness of the measure should be inversely related to the sensitivity of the variable measured; the more sensitive the variable, the less invasive the measure.
For example,
asking people what they think about a set of new product features is a highly invasive measure that can yield inaccurate results. By contrast, inconspicuously observing the way people interact with the features is a minimally invasive measure, and will yield more reliable results.
In cases where highly invasive measures are used over long periods, it is common for systems to become permanently altered to adapt to the disruption of the measure. For example, the goal of standardized testing is to measure student knowledge and predict achievement. However, the high stakes associated with these tests change the system being measured: high-stress levels cause many students to perform poorly; schools focus on teaching the test to give their students an advantage; students seek training on how to become test wise and answer questions correctly without really knowing the answers; and so on. The validity of the testing is thus compromised, and the invasiveness of the measure fundamentally changes the focus of the system from learning to test-preparation.
Use low-invasive measures whenever possible. Avoid high-invasive measures; they yield questionable results, reduce system efficiency, and can result in the system adapting to the measures. Consider using natural system indicators of performance when possible (e.g., number of widgets produced), rather than measures that will consume resources and introduce interference (e.g., employee log of hours worked).
See also Cost-Benefit, Expectation Effects, Feedback Loop, Framing, and Signal-to-Noise Ratio.
There is an inverse relationship between the invasiveness and the accuracy of system measures — the more invasive the techniques to measure a phenomenon, the less accurate the measurements. In extreme cases, invasive measures can so severely disrupt a system that it will alter its goal to serve the measure, making measurement meaningless. System efficiency also suffers from invasive measurement techniques, since system resources must be applied increasingly to accommodate the measurement.