Real Software Engineering.

Thanks to Willis, I stumbled on this talk on "real software engineering". The main thesis is that software engineering isn't really an engineering, since it doesn't "work" like other engineering. That is, the processes we call software engineering do not ensure success like in other engineerings. I'm not sure if I would call this the major distinction between software engineering and other engineering.

He also talks about how software system are often a lot more complex than the products of other engineering. This makes ensuring quality much harder. Using mathematical models is impractical, so testing is the only real way to ensuring quality. Further, this testing has to be done very often and early, in order to minimize testing costs.

He makes an interesting distinction between the engineering processes between software engineering and something like structural engineering. In structural engineering, the engineer designs the final product and encodes that in some design document. This document then goes to laborers who actually create the final product.

Software development follows a different process. It seems unlikely for this workflow to apply to software engineering, since we haven't found a good way to express a design completely and unambiguously. Instead, the source code acts as the design document. It is, after all, a very specific document encoding the exact project requirements. With this, the language compilers become the laborers who build the final product given a design.

I think he missed an important difference between software engineering and other engineering - changing requirements. I don't think any other engineering has to design systems that must be flexible enough to withstand customers adding, removing, and changing requirements at any time during the project.

I think the origin of the Waterfall Method is hilarious. In 1970, Winston Royce published a paper about the Waterfall Method. His main point was that projects using Waterfall were "doomed" to failure. However, the paper was poorly written, and it is easy to misunderstand him and think that he recommends Waterfall. This is exactly what happened. Too many managers simply looked at the diagrams that seemed to make a lot of sense. They completely disregarded warnings. In fact, on the second page, Royce presents the first ever Waterfall Diagram. The very next line reads: "I believe in this concept, but the implementation described above is risky and invites failure.". I guessed they all missed that part. *facepalm*
My favorite quote (on page 1!): "An implementation plan to manufacture larger software systems, and keyed only to theses steps, however, is doomed to failure."

So lesson: If you are going to implement some new idea from a paper, read the full paper!