Monday, August 8, 2011

Software Engineering All-Star Topics: Redundancy

There are a few very prominent topics in all fields. I think redundancy is one of the biggest ones in the field of software engineering. Redundancy in the context of software means having duplicated services or data.Why would you want to do this? Well there are many reasons.

First, redundancy is a very powerful way of creating fault tolerant applications. If there are two identical copies of some services, it's okay if one temporarily goes down. While this may seem like a rather inelegant (and expensive) solution, it works extraordinarily well. Scared about your web service going down? Make two of them. Or N of them. Worried about data corruption? Replicate it on separate hard drives (on potentially separate machines).

Got scalability problems? The solution might be to use redundancy to implement load balancing. This is commonly done to implement horizontal scaling.

Redundancy can also be used to solve a huge subset of performance problems through caching. Caching is just a form of data redundancy. In practice, caching is one of the biggest reasons computers are so fast today. The internet has many great examples of this. Your browser caches web pages to achieve huge performance boosts. Want to see the difference? Check out StumbleUpon. Start stumbling and notice how slow it is compared to refreshing your Facebook page. That's because the data you are accessing needs to be accessed from a web server, instead of (mostly) from your browser's local cache. DNS records are also cached by many machines on their way to you. Without this caching, loading every single page on the internet would take ~200ms longer to load, simply because DNS would have to redo all name resolution queries every time. File caching on your OS is another good example of this. Without system file caching, your OS would also run noticeably (and painfully) slower. Caching is responsible for some of the biggest performance leaps we've seen in computers. Interestingly enough, caching is usually implemented through hash tables, which is a computer science all-star topic.

This practice is not unique to software engineering. Redundancy has been used in most other engineering disciplines to establish fault tolerance for years. For example, Boeing 747s are equipped with 4 engines, but are designed to run with just 3.

Guess something useful came out of that Distributed Systems class after all. Want exam to be over though. :(

No comments:

Post a Comment