Over-Engineering

Over-engineering has started to become a pet peeve of mine. Sometime I feel like my thoughts on architecture follow a pendulum. It goes from the extreme of abstracting like crazy and applying design patterns out the ass, to the other extreme of not thinking about generalizing at all. Eventually, I hope to settle down at a reasonable point in the middle. Give me a few years.

I think our current DB schema is a little over-engineered. Granted, we wanted to challenge ourselves to do something new with the project, but it ended up costing us a good deal in development time. Not so cool with OS in the way. In "real life", we would have saved time in the long run, since the project would probably to stay around for 5 years. The time used to set up our architecture would have made up by speeding up all those years of future development. Unfortunately, our application has the lifetime of a month.

I think that a good rule of thumb is to write the simplest code possible, without coding yourself into a corner.

Do you need to unit test that class? No? Then don't make it an interface. Changing this later isn't hard, but working with a system where every single class is an implementation of an interface is harder. Adding unneeded complexity to your code will just make your system more error prone.

Do you really need to every function to be less than 10 lines? Well is it hard to understand your code? No? Then don't bother. Reading through a file with hundreds of three line functions is a nightmare. It's really cool when you can read the high level function as if it were pseudo code, but understanding how everything works becomes much more complex. And again, complexity will lead to more error prone code.

Do you really need to make that Singleton thread safe? No? Then why would you? It's trivial to make that change later, when you actually need it.

With that said, it's important that you don't code yourself into a corner. You shouldn't make a system that's hard to extend. It should be easy to extend when the need arises. Until then, leave it alone.

Making Carts Using Genetic Algorithms

This genetic algorithm tries to generate an optimal cart with two loads to traverse some static terrain. Complete with a nice little animation. :) I want to look through the source, but I don't have enough time now. Hopefully I'll remember to check it out after this week is over.

In other news, the deadline for DB was pushed back to Monday. It's appreciated, though we are close to done anyway. Maybe the OS deadline will be pushed back as well. :P

Anyway, time to prove things are NP-complete. lalala

Final Push

This week is going to suck...

The two projects are due this week. DB is due Friday, OS is Sunday. (and there's that pesky algorithms on Thursday :/). DB is very close to being finished. We are in the testing phase right now. Hopefully we will write our last lines of code today. OS is much, much further from being completed. In fact, we have 0 feature implemented, and we are still unsure about how everything works. We have some framework code working, but that really isn't impressive. Hopefully with DB out of the way, we can focus all our time on this and figure it out ASAP.

At least after this week is over, the crazy course load is done. I'm not expecting exams to be that bad. At the very least, it won't be a super busy time for me. This means I'll have time to sleep (and not have dreams about OS and DB)! That'll be the life!

How is your last week looking?

Architecture Lessons at 3AM

I learned a nice architecture lesson last night: Think about error handling early. The wrong time to think about it is at 3AM after most of your functionality is built in.

Originally, our architecture relied on returning null on failure. The levels above would check for this and handle it appropriately. There are a few problems with this. The problem that impeded us the most is that we were unable to differentiate between various kinds of failures for some method. In hindsight, this isn't a very clean way to handle error handling in general.

The first solution we tried involved an external error handling class. Our model (and data access code) would set flags in this static error handling class to indicate what went wrong. This meant that every time someone wanted to call on the model to do something, they would have to remember to check the error handling class to see if anything went wrong. This created a coupling between this error handling class and our models. If you forget to check that error flag, your code would fail silently or cause unexpected behavior. We eventually scrapped this idea in favor of exceptions.

Exception were better because any failure would force the programmers to deal with it immediately. If they forgot to catch the exception, the program would crash right away, and with a very clear reason (like SQLException). You could not simply ignore the error and continue running in ignorance like with the other static error handling technique.

After we decided on exceptions, we started talking about whether we wanted checked or unchecked exceptions. More fun debating there. All this happened after 2AM. I am always hesitant to make important design decisions at that hour.

We eventually decided to go with checked exceptions. This means changing almost all of our classes (including all our tests). Granted, the changes are fairly simple and straightforward, but it's still going to cost us some time to re-factor everything. Had we thought about error handling straight away, we could have saved some time.

Moral of the story: Think about error handling during your planning phase!

On an unrelated note, Willis and I got 97 on OS A2. Hell-freaking-Yeah!

The 7cubed Project

Check this out: http://7cubedproject.com/

7 UW programmers making 7 apps for 7 days. Very cool. Today is Day 5.

Day 1 was Quick Cite. With this mobile app, you can scan any book and the formatted bibliography will be sent to your email.

Day 2 was UWChat. This lets you check into UW rooms and have chat sessions with people in the same room as you.

Day 3 was Minehub, a dynamic Minecraft server.

Day 4 was supposed to be something called "AwesomeTracker", but unfortunately they were unable to ship it on time. They said that they'll get back to it after the week is over.

This seems like an interesting test of the limits of agile programming. The group says that a highly skilled team of developers can successfully follow an agile methodology where they measure releases on the order of hours. So far, it seems to be working quite well. Of course, there have been some hiccups. The failure to ship on day 4 is perhaps the biggest. The other big one is that during the rush to get Quick Cite out, they accidentally shipped code with the email functionality commented out. I think that these sort of problems are almost unavoidable when you are developing this fast.

In any case, I'm really interested in what else they can create. This seems like a really cool experiment, and a lot of fun for the coders involved.

Sanity Progress

I was feeling quite overwhelmed yesterday with the amount of work I have to do. Third year is hard. :/ In first year, they were all like lalala "If you can adjust to first year, the rest is easy". In second year, people said lalala "If you can get through second year, you can do the rest". Bullshit. Third year has been orders of magnitude harder and more work. First and second year courses don't have projects requiring more than 100 man-hours of low level C programming. Perhaps it's my fault for taking 2 project courses and algorithms at once... Maybe the next couple semesters will be easier (especially if I don't take Compilers and Real-time).

I am feeling better today. Our OS prof was nice and went through a lot of the assignment requirements in lecture today. I feel like I actually know where to go now, instead of poking through the OS/161 kernel aimlessly. :P DB also seems to be progressing fairly well. Hopefully we'll be feature complete for tomorrow or Thursday. If this was Amnesia, I would be surrounded by white light and my sanity would go up a level.

However thanks to this crazy workload, I feel like I'm coming out a stronger programmer. Writing code for an OS is quite humbling.

How much voltage does your program need?

Apparently Intel thinks they can create CPUs with up to 1000 cores. It'll be interesting to see how much of a change this will spur in Computer Science and Software Engineering.

I know some schools are already teaching multi threaded algorithms and concurrency. I can see this becoming more and more important in the future. I am getting really excited for the concurrency course at Waterloo.

Also, the Intel cores contain hooks that let you change the voltage to the cores. This sounds like yet another crazy thing for programmer's to consider in the future. I used to think that these issues were stupid. How much could they possibly reduce it by? Who cares how much voltage your CPUs are using? Google does. Saving 1% of CPU voltage will save the company thousands and thousands of dollars. It's pretty crazy.

Anyway, 4 assignments left in 2 weeks. Woo! OS an DB are both due in less than two weeks. Why does the end of the school year have to be so busy?

Ukrainian Folk Short

My dad sent me this video this morning. It's nice. I totally remember visiting family in Ukraine, living in a village like the one in the video. It must look weird to other cultures. Why are there random flies in the house? Why are people singing at the dinner table? Are they using a scythe? What's with the clothing? It seems weird how I've seen all these things in person, and now I live in a very different place. It's a cool feeling.

Software Developer Interviews

There seems to be a problem hiring good developers nowadays. It is apparently hard to hire people with even basic coding knowledge. The first time I really encountered this was in Jeff Atwood's blog post "Fizz Buzz", however it seems that this problem is still around.

That second article talks about basic interview screening questions that most("19/20") job candidates couldn't answer. The specific question was "Write a C function to reverse a singly-linked list". This seems unusual to me, since I've encountered harder questions during my co-op interviews.

It seems weird that some companies seem to be grilling co-op students harder than full time developers. Of course there are companies (like Google) who definitely put the full-timers through a lot to get a job. I've heard stories that it takes something like 6+ technical interviews to get a full time job at Google. This is certainly more than the hour and a half they spent with me when I interviewed with them this term.

I keep track of all the technical questions I've been asked in interviews. These questions have been collected over the course of two terms worth of interviews. Could you answer some of these? If so, then apparently you can have a comfy job as a developer.

1) What is a static function?

2) Can you have a static constructor?

3) Why would you use a static constructor?

4) What is a virtual function?

5) What's the difference between a thread and a process?

6) Fill an array of size 100 with the first 100 terms of the Fibonacci's Sequence.

7) What is the difference between a stack and a queue?

8) What is the difference between an exe and a dll?

9) What is object oriented programming?

10) Explain encapsulation, inheritance, polymorphism.

11) What is abstraction?

12) What is the difference between an abstract class and an interface?

13) What is the difference between pass by value and pass by reference?

14) What is a pointer?

15) Does Java/C# have pointers?

16) What is the difference between a pointer and a reference?

17) Implement modulus.

18) How would you overflow a stack? Heap?

19) Count the number of set bits in an n long array with 16 bit numbers? Can you improve the constant factor?

20) Worst case for a hash table?

21) Implement multiplication.

22) Element Distinctness problem.

23) Write Fizz Buzz.

24) What does final mean in Java? Static?

25) How would you unit test <this>?

26) Basic question involving Distance = time * velocity.

27) Design a program to find the largest word in a paragraph

28) Design a program to extract unique values in a sorted list

29) What is normalization?

30) Get the second largest salary in a DB using SQL.

31) What's the difference between GET and POST?

32) Whats the probability of flipping 3 coins and getting heads all 3 times.

33) What is synchronization in the context of concurrency?

34) What is dependency injection? How would you implement it? How would you use it?

35) Name some differences between Java and C#.

36) How to bindings work for polymorphism?

37) How do you read a file in Java? (Stupid memorization question is stupid)

38) Given 7 "letter tiles" and a dictionary of valid words, return the set of words that can be generated using those tiles.

39) Given a file with expansion data like this:

“foo” expands to “foo” “bar” “biz”; 

“buzz” expands to "blarge" “*foo”

where *foo means expand foo, so expand(buzz) = "blarge foo bar biz"

Write a function that expands strings based on such a file.

40) Write code that will always deadlock.

41) What is a buffer overflow? How is it exploitable?

42) What is DEP?  What is ASLR? How do they help prevent buffer overflows?

43) What is BotNet? What is some of the malicious things it can do?

44) What is XSS?

45) What is a SQL injection?

46) What would you do if you found a security exploit?

47) When (if ever) is it okay to release the security exploit to the public?

48) What is an exploit? What is a vulnerability?

49) If you could enforce one programming rule to improve security, what would it be?

50) What is the difference between verification and validation?

51) What makes a good requirement? “The DB needs to be fast”. Is that a good requirement?

52) What goes into a good bug report?

53) Tracing DFAs.

54) What would you consider when designing a DB?

55) Find the memory leak in the some code.

56) What is MVC?

57) What are the 4 pillers of OOP?

58) Implement inheritance in a database.

59) Use joins (then don’t use joins) to get the data from above tables.

60) What do you do when you can’t beat a deadline? (impossible deadline)

61) String builder vs String in Java. When to use which?

62) What are JS prototypes?

63) What is AJAX?

64) What is JSON?

Hopefully this is helpful to people doing interviews next semester.

Wolfram And Natural Language Programming

I just read Stephen Wolframs's most recent blog post on programming with natural languages. He talks about some really cool features of Mathematica 8 and Wolfram|Alpha. The examples he shows are quite impressive.

Wolfram talks about a future where programmers would be able to communicate with a computer using natural languages like English. The programmer would specify the requirements in English, and the computer would synthesis a program to actually satisfy these requirements. Although this sounds like a really interesting idea, I can see a lot of problems for them to overcome before this becomes practical.

Perhaps the largest of these problems is that natural languages are very imprecise. The amount of text required to specify a requirement accurately in pure English is huge. Massive programs with really complicated requirements would create a huge wall of text when encoded in English. Maintaining and expanding programs that are encoded like this sounds like a nightmare.

Further, English is an inconsistent language. That is, you can derive many contradictions and silly results using the English language. For example,

Consider transitivity. If A > B and B > C, then we can conclude that A > C.
Now,
let A = "A Cheeseburger"
let B = "Nothing"
let C = "True Love"

So now,
A Cheeseburger is better than nothing (A > B)
Nothing is better than True Love (B > C)
so it should follow that
A Cheeseburger is better than True Love.

Silly result is silly, but it just shows that English sucks at being precise. This silly proof was originally shown to me by Professor Shai Ben-David in Logic class. He was explaining why we can't use English as a language for Logic. The reasons are essentially the same as the reasons why we can't use English for programming.

We could solve some of these issues by creating restricted English, but figuring our a working subset of English would be a huge task in itself.

It is interesting to see how software engineering techniques would change if this were to be come popular. MVC would translate to separate paragraphs talking about views, models, and their interactions. Most design patterns could be specified with an additional paragraph. Would these design patterns even be useful anymore?

In any case, the progress in free-form linguistic parsing is really interesting, but we will probably see it applied to other fields effectively before we see it as a substitute for traditional programming languages.

Java's Demise?

This article talks about all the problems facing Java in the upcoming years. It'll be interesting to see how Oracle deals with all this.

I see this as a great opportunity for .NET to gain a lot of market share. According to TIOBE, Java is still ranked first, and C# is ranked 5th. I can definitely see them swapping in the next 5 years. I would be happy if C# at least swapped with C++ (currently 3rd). I wonder how C++0x and C1X will fit into the rankings whenever they actually get released.

I've enjoyed using C# a lot more than Java in the past. Java seems to be really lagging behind .NET in terms of implementing new language features (like Lambda). .NET and Mono are both progressing quite fast. Hopefully I'll get a chance to work .NET more in the future.

In other news, OS A3 has been released and work with DB is raging on. Busy busy.

Technology in Healthcare

You're doing it wrong...

Planning

I think a large part of our problem for OS came from a lack of planning. We were making design and algorithm decisions during our slip day. Not good. :/

We should have sat down and planned out the project in its entirety before we started programming. We were trying to do things sections at a time, as if it were an agile project or something. Unfortunately, pretty much everything in the system was intricately intertwined so looking at some of those modules in isolation didn't really make sense a lot of the time.

I think we'll make it a point to do much for A3. That is, I would like to have the project fully planned out and designed before we write a single line of code. Hopefully this will make the project go much smoother than A2.

There's three weeks of classes left, and A3 hasn't been assigned yet. This is going to be fun.

On a related note, our next DB project has been fairly well planned out. I think that we are making great progress, and our architecture is solid. I am actually a little proud of our dinky design, even though it's just a simple MVP architecture. With three weeks until the deadline, I think we won't be pulling any late nights with this one.

Language Expertise.

Sometimes I read articles about languages and I realize how little I actually know about these languages. I feel like I should read a nice technical book from front to back. Something like Head First C#, rather than Code Complete. I eventually want to read both, but I feel like I should know my languages in more detail before I learn more about engineering techniques.

Not to mention I know very little about some fields of development, like web development. Hopefully I will get a nice crash course in web development during my work term.

My goal is to get through a lot of books this Winter. I should be able to do it since I won't be worrying about OS or DB all the time. :P Hopefully I won't be too distracted by Roslin house/video games.

On my to read list (in order):
- Head First Java
- Re-read Clean Code
- Code Complete 2

I will be happy if I can get through those in 4 months, but more would be nice.

Anyway, I should probably do something productive. Maybe I'll fail at Algorithms for a bit. :/