November 2009 – Bad Engineering

I made some comments earlier about anecdotes of engineers not accepting the theory of evolution and natural selection (link) and that got me thinking about the engineering process that is, or closely resembles, evolution.

There is a reason we don’t see a half-crocodile, half-duck (the famous crocoduck) in nature – because those two species had a common ancestor millions of years ago and at that point they parted ways and evolved independently into the current two species. This is analogous to why we don’t see the tail of a Sopwith Camel on an F-16. They both have tails, but the design of aircraft has evolved over the years so the F-16 has a tail that is appropriate for itself, not some older airplane.

And engineering does the same kind of testing before accepting a new feature or process, just like nature. A random change in nature is tested by the environment and if beneficial to the individuals, it is more likely to be retained (a greater percentage of the population survive and breed if they have the change) for future generations and slowly the change becomes part of the species. In engineering, many designs start from a previous, successful design and add such changes that are deemed necessary for the mission, but also that are affordable and have an acceptable risk of failure. If those changes are successful, then they will be the starting point for the next design. Hence moving from the Sopwith to the F-16.

Continue reading “Evolution and Engineering” →

Despite my preference for R over Excel, I still use a spreadsheet for many applications. But I find spreadsheets are often misused, or poorly constructed, and can create work later that can exceed the time and effort they might have saved you in the beginning. One way to prevent this is to think database when you are spreadsheeting.

An Excel spreadsheet is not a database and shouldn’t be thought of one, even if the two words are used interchangeably by people like, say, upper management (if your organization is anything like mine). But if you are doing anything with a lot of rows or columns of data, it is always a good idea to construct the spreadsheet as if it were a database. This at least gives you a chance that you will be able to add or extract data from the spreadsheet.

For example, we routinely receive large spreadsheets of thousands of rows long that are, outside of the bounds of the spreadsheet format, almost useless. The addition of fancy titles, merged cells, subtotals, empty rows, and other spreadsheet junk that disrupts the data makes extracting the data tedious at best and nearly impossible at worst. It would have been so much more useful had the author made one sheet hold the data—one header row with unique field names and just the raw data below, one record per row—and one sheet holding the spreadsheet math to provide the summary and calculated information. This is actually easier to construct and doesn’t lock the data in the spreadsheet. It is easy enough to export the raw data as a dbf or comma delimited file to work on it in a proper database program or just use the data without relying on the Excel structure.

I know there is a temptation there to add the spreadsheet junk. Pretty colors, lines, comment blocks, font changes, etc. are fine for that sheet that does the calcs (even if all that is usually a waste of time), just don’t let the junk get in the way of the data. That’s the important stuff.

What do you think? Drop us a comment.

Month: November 2009

Evolution and Engineering

Think Database even when Spreadsheeting