5 years, 7 months ago

Base rate fallacy

Link: http://www.etc-architect.com/?p=187

Most architects that I have met have a great belief in numbers as they claim that there is something stable on numbers and on logic. The really funny thing however is that most people that heavily rely on numbers are also subject to the base rate fallacy. The classical example of this is DNA testing where the error rate for a standard test is that of one in a hundred thousand and as such you will argue that this kind of proof is secure. However if a really bad crime has happened and DNA testing is used for screening the odds change as if you screen a million people as you suddenly will find that 9 innocents will be caught for 1 real perpetrator (this is also why DNA screening is used very cautious and why a DNA central register makes little sense).

You will see a similar pattern when business architects are talking on the savings that employee self service will bring, as it indeed saves money of HR clerical staff, but at the same time it decreases the productivity of the personal in other jobs. And even if people go so far in actually remembering this they will then usually forget about numeric factors such as the retraining of users or users without direct access to a browser connected in the right network. Another classical example is the way that most of us calculate the TCO on technology such as it happened with virtualisation or SOA where architects only concentrated on the base rate instead of the connected cost scenarios such as with SOA governance (you can easily still see that argumentation once you read the more popular books on one of the subjects.

One of the really persistent base rate fallacies is that of average activity based costs (ABC) especially in business process improvement. Measuring ABC in a given company makes a great deal of sense to control costs on specific high frequency activities, but when comparing it across industry you get very fast in the base rate fallacy as some factors such as the frequency rates, the access to information and temperatures are making a comparision nearly impossible. The temperature is often also one of the key factors forgotten in many areas outside of ABC, such as with customer satisfaction where an extreme temperature usually is the biggest influence when asking customers.

The problem that we all face is that we need to be careful in trusting to much on numbers and especially in relying on a rule of thumb as a rule thumb maybe great in cost estimations and project planning, but once the serious business starts we should always start to think about different viewpoints.  

Related Post