Tenets of Engineering Skepticism

A couple of my old posts talked about engineering skepticism (here).  I’ve been thinking more about it and came up with the following four tenets of ES.  These would be used first to test a claim, process, or machine before it could be considered useful to the engineering world.

Four tenets of Engineering Skepticism

1. The device, process, or method must be effective. It must be clear that there is a real effect without relying on advanced statistics. If the results are less than 25% better than chance (guessing), it doesn’t pass the ES test and can be dismissed as being useful to the engineering world.

2.  It must be reliable. This means that it should work the way it was intended in real life conditions and with any trained operators. If it relies on the weather, the aura of the user, or how the stars are aligned, then it fails the ES test and can be dismissed.

3.  It must be repeatable. If it is used in the same way in the same environment multiple times it should provide the same or very similar results. This should hold true even when done with different operators. For instance, if five different dowsers go through an area and give five different results, the method does not pass this ES test and can be dismissed.

4.  And it must be teachable. If it depends on some peculiar talent or right of birth then it is of no use to engineering. Knowledge must be able to be recorded and passed on to new generations of engineers.

Failure to pass all of these tenets does not necessarily mean that the device or method is fake or false, but that it is not dependable or of enough rigor to be useful to engineering. Until it meets those tenets, any engineer worth his salt will dismiss these devices or claims.

Post a comment if you think any of this makes sense, or doesn’t.

Published in: on May 8, 2011 at 4:49 pm  Comments (1)  

Storm

Tim Minchin is simply brilliant.

Published in: on April 9, 2011 at 1:07 pm  Leave a Comment  

Admitting you’re wrong

A good engineer, like a good scientist, must at some point admit that he is wrong.  This is very difficult to do — especially for mid- and late-career engineers who are a bit set in their ways.  It is, I guess,  a pride thing.  And may older engineers have spent their whole careers building up that reputation and pride.  But what they, and their management, don’t understand is that it is important, and right, to admit when you’re wrong and that act should bolster an engineer’s reputation, not mar it.

It is those who won’t, or can’t, admit errors who are the bad engineers.  These people ignore or spin data and calculations so as to fit their view rather than accepting that the data don’t support their belief.  This is not engineering but rather wishful thinking.  Or pride gone amok.   One problem with this is that it can take years before the truth comes out.  And sometimes it never does.  This only contributes to the cognitive dissonance in the mind of such an engineer.

Granted, this path is difficult to avoid and each step down it makes it more difficult to make that jump to the right path.  But it needs to be done and I think it is a sign of character and good engineering when people correct their path that way.  Of course in our current climate this step is just an invitation for blame; probably the main reason for people staying on that wrong path.

Bad enough for the lone engineer, but real trouble comes when you have a whole organization that fosters this stance.  We have such an organization where I work and the very idea that they are not ever wrong, and cannot ever be wrong, is ingrained in the history and management of this group.  The members are smart, no doubt of that, and are highly educated engineers, but they can’t escape the corporate climate (even if they wanted to).  This, in my opinion, makes them poor engineers in a poor organization.

And if you give such an organization power, what you get are arrogant and defensive engineers who use intimidation rather than rationality when dealing with people outside their organization.  Because, you see, they now have to convince both you and themselves that they are always right.  And that is a lot harder to do without the bluster.  No matter what, this only leads to Bad Engineering.

Don’t fall into that same trap.

Suggested reading: Mistakes Were Made (but not by me)

Published in: on March 8, 2011 at 9:36 pm  Leave a Comment  

Engineering Skepticism

Skeptic, as a word, has had a lot of different definitions and connotations in the past years and most of them become perjorative in the eyes of non-skeptics who seem to equate skeptic with cynic.  The debate over the definition of the word, and who is more deserving to use the word for self-identification, is still going on today.  From my perspective, there are two major camps: the scientific skeptics and the deniers who call themselves skeptics.

The former relies on scientifically received data upon which to base their conclusions.  The latter make conclusions and go in search of data or anomalies  to fit their beliefs.  As you can imagine, I hold with the scientific skeptics and the proven scientific method.

But I think there is another category, or at least I want to create one: engineering skepticism.  And no, this doesn’t lie halfway between the other two.  It is close to SciSkep, but I don’t think it is quite an offshoot of it.  At least not in the approach.  I think of it as the engineering approach to acceptance and rejection.

For example.  As I wrote about earlier on the ADE-651 bomb detector, SciSkep might develop double blind tests, collect data, test the hypotheses behind the machine, etc.  EngSkep would point out the lack of a power source, the cited interference of the operator’s mood on the results, and the implausible operating ranges and would reject the machine immediately.  Not because of testing data, but because of its clear unreliable nature.

Science and Engineering bring home the goods, as Carl Sagan said. Woo doesn’t. I’ll go a little farther and say that science and engineering have to be right, they HAVE to bring home the goods.  Science needs to be right eventually, through a slow process that self-corrects.  Engineering has to be right, right now.  It evolves, but it has to bring home the goods now, not later, and do so without mistake.  Engineering products have to be reliable or they don’t sell or they aren’t safe. And by reliable I mean that it does what it is supposed to do 99.99 percent of the time (or better).  So no decent engineer on a source selection committee would approve the ADE-651.  He wouldn’t have to wait for Science to catch up to know it isn’t reliable enough to consider.

So am I saying that if it works well that is enough for the EngSkeptic?  Well, not really.  Unfortunately, much of the Woo is in the realm of the body and mind and is very prone to subjective thinking.  A person may think that a homeopathic pill cured him, Science can show that it could not, and did not.  EngSkep can’t say anything about it.  On the other hand, I guess we could reject things such as prayer for an amputee – a leg has never grown back so prayers asking for it to do so are demonstratedly unreliable and are not worth doing (mental comfort aside).

So is EngSkep a shortcut to normal SciSkep?  Maybe.  If the first question we ask a suspected Woo topic is, is it reliable? we might be able to save a whole lot of effort of scientific investigation.

Does Astrology produce reliable data? No.  Toss it out.  Do psychics produce reliable predictions?  No.  Don’t listen to them.  Do Ufologists produce reliable evidence of UFOs? No.  Ignore them.  Do ghost hunters and their gadgets reliably produce evidence of ghosts? No.  Reject their methods.

So okay, maybe a shortcut.  And certainly not as rigorous as SciSkep, but do we really have the time to waste doing real Science on real Woo?  Let’s use EngSkep as the first hurdle.  Then, if the Woo passes, let Science take a crack at it.

I think I’ll have more to say about this soon.  What do you think?

Published in: on February 12, 2011 at 6:24 pm  Comments (6)  

Being Wrong

As an engineer, and a wanna-be scientist, I’m a fan of both disciplines and I’m cognizant of the differences between them. I’m not talking about the educational levels or the organizational differences or the societal pecking order. Rather I think the real difference between them is a matter of perceived and actual risk.

What happens if we are wrong? That question is always in the back of an engineer’s mind. Or if it isn’t, it should be. And I’m not talking about saving your job for the sake of the mortgage bill or getting credit to help out you promotion prospects. No, I’m talking about what happens to people and property if we are wrong. What happens if the designer of that airplane wing was’t diligent in his calculations? What happens if that elevator engineer misses a decimal place? Even what happens if that structural engineer steps a bit too far into unexplored territory and uses new fasteners in that new building?

Some of our leaders here say that we engineers shouldn’t sleep well at night because of these kind of questions. And many of us don’t.

But I wonder what kind of questions keep scientists up at night. If an astronomer or theoretical physicist is wrong, what are the consequences? Is life or limb at risk? I doubt it. Reputation, maybe, and perhaps there is a risk of grant funding, but how many working scientists (outside of the health field) directly affect the general population in terms of risk to their lives?

I’m not trying to take sides here, but rather trying to ponder on how risk affects the thinking of engineers and scientists. I think the risk questions (including economic risk) constrain an engineer’s actions while the lack of the same allows the scientist to venture into the unknown knowing that the consequences of being wrong will not be severe or widespread.

I’m thinking that it must be nice to be in that position. What do you think?

Published in: on January 10, 2011 at 10:08 am  Leave a Comment  

Don’t be a Dick, 18th Century style

I talked a bit about the whole “Don’t be a Dick” issue that started (this time around) with Phil Plait’s welcome address at TAM 8 (http://badengineering.org/2010/07/) and I concluded that, for me, I preferred to think that it should really be something more like don’t be a dick unless you’re talking to a dick.  Phil’s talk spawned numerous responses on the web that were for and against his message, and every stance in between.  If there is anything that the talk showed, it was that the skeptical movement has a similar cross-section of personality types as does the general population.  In short, we have some people who don’t mind being argumentative and strident, and some who don’t like the confrontational style, and a whole lot of people in between.  Which, I guess, means that nobody is right and nobody is wrong.

What is wrong, in my opinion, is in picking the targets incorrectly.  Sure, send PZ or Richard Dawkins after the Ken Hams of the world.  But send the Phil Plaits and Chris Mooneys of the movement to talk to moderates.  Match the weapon to the target and thereby make all the different weapons valid.

This has been percolating in my brain since TAM 8 and I was delighted recently to see this topic addressed by a very wise man.  As I am not a great confrontationalist (I know, not a word), I am going to try to follow his example.  I don’t think it is the easiest way to counter non-critical thinking, but I do think it can be very effective.

From his autobiography, Benjamin Franklin’s words on not being a dick:

While I was intent on improving my language, I met with an English grammar (I think it was Greenwood’s), at the end of which there were two little sketches of the arts of rhetoric and logic, the latter finishing with a specimen of a dispute in the Socratic method; and soon after I procur’d Xenophon’s Memorable Things of Socrates, wherein there are many instances of the same method. I was charm’d with it, adopted it, dropt my abrupt contradiction and positive argumentation, and put on the humble inquirer and doubter. And being then, from reading Shaftesbury and Collins, become a real doubter in many points of our religious doctrine, I found this method safest for myself and very embarrassing to those against whom I used it; therefore I took a delight in it, practis’d it continually, and grew very artful and expert in drawing people, even of superior knowledge, into concessions, the consequences of which they did not foresee, entangling them in difficulties out of which they could not extricate themselves, and so obtaining victories that neither myself nor my cause always deserved.

I continu’d this method some few years, but gradually left it, retaining only the habit of expressing myself in terms of modest diffidence; never using, when I advanced any thing that may possibly be disputed, the words certainly, undoubtedly, or any others that give the air of positiveness to an opinion; but rather say, I conceive or apprehend a thing to be so and so; it appears to me, or I should think it so or so, for such and such reasons; or, I imagine it to be so; or it is so, if I am not mistaken. This habit, I believe, has been of great advantage to me when I have had occasion to inculcate my opinions, and persuade men into measures that I have been from time to time engag’d in promoting; and, as the chief ends of conversation are to inform or to be informed, to please or to persuade, I wish well-meaning, sensible men would not lessen their power of doing good by a positive, assuming manner, that seldom fails to disgust, tends to create opposition, and to defeat every one of those purposes for which speech was given to us, to wit, giving or receiving information or pleasure. For, if you would inform, a positive and dogmatical manner in advancing your sentiments may provoke contradiction and prevent a candid attention. If you wish information and improvement from the knowledge of others, and yet at the same time express yourself as firmly fix’d in your present opinions, modest, sensible men, who do not love disputation, will probably leave you undisturbed in the possession of your error. And by such a manner, you can seldom hope to recommend yourself in pleasing your hearers, or to persuade those whose concurrence you desire.

Worked for him, didn’t it?

Published in: on October 27, 2010 at 9:44 pm  Leave a Comment  
Follow

Get every new post delivered to your Inbox.