Most organizations have some projects that succeed, and some that fail. • When a project fails • the people on it are dispersed to other projects • the metrics that have been collected for the failed project often vanish • a retrospective (a.k.a. post-mortem) doesn’t happen for the failed project • in summary, it doesn’t get counted and therefore it doesn’t count • yet this might be the most instructive project
The Metrics Minefield
The intention is not to suggest that metrics are useless, but to identify some examples of gross dysfunction, and to try to avoid it
Measurement is the empirical, objective assignment of numbers, according to a rule derived from a model or theory, to attributes of objects or events with the intent of describing them.
How Do We Measure?
Metric models are based on some comparison, which may be explicit or implicit
We can • count things
compare individual things with each other • compare individual things with a reference
count things over time (rates)
Why Do We Measure? Implicit in our motive for measurement is some model of assessment, based on some comparison, made in accordance with some observation
facilitating private self-assessment and improvement • evaluating project status (to facilitate management of the project or related projects) • evaluating staff performance • informing others (e.g. potential customers) about the characteristics (such as development status or behavior) of the product • informing external authorities (e.g. regulators or litigators) about the characteristics of the product
Quality measurement depends upon our skill at observation, what we’re comparing, and the validity of the models that we’re using for assessment
Mine #2 Comparisons and assessments aren’t necessarily numerical
Lines of Code Metrics
Mine #3 Numbers aren’t as descriptive as words and stories. Words can be vague or ambiguous, but numbers without clarifying words are just as bad or worse
Mine #4 Many people in our field make sweeping, pseudo-statistical generalizations without proposing a measurement model.
Mine #5 Most people in the testing and quality field haven’t studied measurement theory or statistics, but some of us are asked to implement measurement programs anyway.
Glasp is a social web highlighter that people can highlight and organize quotes and thoughts from the web, and access other like-minded people’s learning.