Motivated Reasoning

Motivated reasoning is one of the easiest traps for an engineer to fall into.  This pleasantly oxymoronic term can be considered as an extreme case of confirmation bias.

So what is it?  I see motivated reasoning as the practice of allowing emotion to creep into the engineering process — usually through the emotion-based evaluation of data or calculations leading to an emotion-based decision.  It is a matter of putting more credence into your feelings than in the data at hand.

Now this doesn’t mean you always have to be Spock rather than Kirk, but one needs to understand that all decisions are, to a certain extent, emotional, but that one can’t dismiss engineering facts just because they don’t fit your emotional needs.

A case came up recently that illustrated this tendency perfectly.  A report generated by the best of the field offices completely disregarded a previous report from another office.  Why?  Because if that older report was true, it would mean more work and expense.  By disregarding the old data, they could avoid a lot of work and cost.  Trouble was that there was nothing really wrong with the old data.  Certainly nothing that would support tossing out the whole report.  To rationalize their position, they harped on some small errors and inconsistencies.  Their position became an emotion-driven one based on external pressures to reduce cost and schedule impact.  The correct, data-driven position was overridden.

We corrected this situation but it illustrates how easy it is, even for first-rate engineers, to fall into this trap.  Experienced engineers can sometimes go with their gut (emotions) and succeed, but when data and calculations are available, go with the math every time.  Sort of like flying in the clouds: trust your instruments and not the seat of your pants.


Tenets of Engineering Skepticism

A couple of my old posts talked about engineering skepticism (here).  I’ve been thinking more about it and came up with the following four tenets of ES.  These would be used first to test a claim, process, or machine before it could be considered useful to the engineering world.

Four tenets of Engineering Skepticism

1. The device, process, or method must be effective. It must be clear that there is a real effect without relying on advanced statistics. If the results are less than 25% better than chance (guessing), it doesn’t pass the ES test and can be dismissed as being useful to the engineering world.

2.  It must be reliable. This means that it should work the way it was intended in real life conditions and with any trained operators. If it relies on the weather, the aura of the user, or how the stars are aligned, then it fails the ES test and can be dismissed.

3.  It must be repeatable. If it is used in the same way in the same environment multiple times it should provide the same or very similar results. This should hold true even when done with different operators. For instance, if five different dowsers go through an area and give five different results, the method does not pass this ES test and can be dismissed.

4.  And it must be teachable. If it depends on some peculiar talent or right of birth then it is of no use to engineering. Knowledge must be able to be recorded and passed on to new generations of engineers.

Failure to pass all of these tenets does not necessarily mean that the device or method is fake or false, but that it is not dependable or of enough rigor to be useful to engineering. Until it meets those tenets, any engineer worth his salt will dismiss these devices or claims.

Post a comment if you think any of this makes sense, or doesn’t.