Wednesday, November 17, 2010

Wolfram And Natural Language Programming

I just read Stephen Wolframs's most recent blog post on programming with natural languages. He talks about some really cool features of Mathematica 8 and Wolfram|Alpha. The examples he shows are quite impressive.

Wolfram talks about a future where programmers would be able to communicate with a computer using natural languages like English. The programmer would specify the requirements in English, and the computer would synthesis a program to actually satisfy these requirements. Although this sounds like a really interesting idea, I can see a lot of problems for them to overcome before this becomes practical.

Perhaps the largest of these problems is that natural languages are very imprecise. The amount of text required to specify a requirement accurately in pure English is huge. Massive programs with really complicated requirements would create a huge wall of text when encoded in English. Maintaining and expanding programs that are encoded like this sounds like a nightmare.

Further, English is an inconsistent language. That is, you can derive many contradictions and silly results using the English language. For example,

Consider transitivity. If A > B and B > C, then we can conclude that A > C.
let A = "A Cheeseburger"
let B = "Nothing"
let C = "True Love"

So now,
A Cheeseburger is better than nothing (A > B)
Nothing is better than True Love (B > C)
so it should follow that
A Cheeseburger is better than True Love.

Silly result is silly, but it just shows that English sucks at being precise. This silly proof was originally shown to me by Professor Shai Ben-David in Logic class. He was explaining why we can't use English as a language for Logic. The reasons are essentially the same as the reasons why we can't use English for programming.

We could solve some of these issues by creating restricted English, but figuring our a working subset of English would be a huge task in itself.

It is interesting to see how software engineering techniques would change if this were to be come popular. MVC would translate to separate paragraphs talking about views, models, and their interactions. Most design patterns could be specified with an additional paragraph. Would these design patterns even be useful anymore?

In any case, the progress in free-form linguistic parsing is really interesting, but we will probably see it applied to other fields effectively before we see it as a substitute for traditional programming languages.


  1. I don't entirely disagree with you, but I'm also calling you out on your particular argument being stupid.

    "Banana" + "Ball"
    #=> BananaBall
    1 + 1
    #=> 2

    What?! It doesn't return 11?! What a load of bollocks!

    Okay, so that seems like a trivial problem, but the cases quickly become non-trivial when you get into dynamic typing. So this isn't some sort of problem unique to English. It just so happens that English is hella more overloaded than C#.

    "In any case, the progress in free-form linguistic parsing is really interesting, but we will probably see it applied to other fields effectively before we see it as a substitute for traditional programming languages."

    Obviously, that's the entire point. It's Not For You. The impreciseness of English actually gives you flexibility in stating complex problems concisely.

    E.g. "How do I defend Poland from the Russians?"

    Maybe that's too vague, so what? Let your HAL 9000 muse about it overnight, and then you can start qualifying things it does wrong in the battle plans:

    "Actually, we'd like to avoid global thermonuclear holocaust if possible"

    "I guess it's alright to let them take the countryside if we can save Warsaw"

    "You know what, aviation fuel is kind of expensive, we'd like to do this with just 3 tanks and a gumball machine"

    Is it difficult? Sure. Is it worth pursuing? Absolutely!

  2. The problem a lot more complex than simple overloading issues. Overloading in programming languages only applies to types. It is context-sensitive, but at a shallow level. All you need to know is the type to determine what something should do.

    English certainly has to deal with same context sensitive issues, but it has a lot more problems too. For instance, English is also ambiguous. E.g. "Keyar is making a meal with kids" has two very different meanings. A computer needs to know a lot of context to determine the correct interpretation of this sentence. We have to ensure that the computer is creating solutions to the right problem.

    I agree this is definitely worth pursuing, and judgement on how effective it would be might need to wait until they have a lot more figured out.

  3. I still think that example is still fundamentally an issue of overloading. Like, take a look here:

    Whether you're trying to call method 1 or method 4 is dependent on whether you're passing in a person or something from the food library.

    Of course there is the possibility that Keyar is a cannibal and is in fact making a meal _with_ kids and then you got quite a duck typing clusterfuck, but it might be easier to correct these edge cases as they come up rather than trying to foolproof the system.

    I mean, if you mean the latter when you're talking to a _person_ it will cause some degree of confusion, so it's not altogether unintuitive for a person to say, "oops, that's not right" and specify which With method they mean to use manually.

    I mean, maybe there is going to be an issue with puns, but let's assume our aforementioned NATO General is not using his HAL 9000 to Screw Around.

    And not to make the task sound trivial, because I know it isn't, but the sentence is not as context dependent as you make it seem. I.e.:

    "Where's Keyar?"
    "Keyar is making a meal with kids"

    How many people would misinterpret that statement? You see, because natural language recursion gives us all the information we need;

    What is Keyar? Keyar is a person.
    What is Keyar doing? Keyar is making a meal.
    What is a meal? Something to eat.

    Then when we tape "with kids" onto our independent clause:

    Do we eat kids? Not usually.

    So we can deduce that he is merely playing with children.

    "Michael Jackson is playing with children"

    Oh shi-

    Addendum: I will retract everything I said about how easy it is to solve English if it turns out that I've misunderstood and Keyar is in fact eating a bowl of children right now.

  4. "Do we eat kids? Not usually."

    How would a computer know this? This is a cultural thing. It is certainly conceivable for some cultures to consider this normal. Again, this is just one specific case. We cannot "hardcode" every ambiguity in a language.

    A lot of times, we don't even recognize ambiguity. That is, most people would not even consider the case of someone eating children. The problem is getting a computer to make the sensible choice without cultural context. This is where I claim the difficulty lives.

    There are probably nice solutions to this problem, but for now, it is definitely a difficult problem to overcome.

  5. "We cannot "hardcode" every ambiguity in a language."

    We would have a hard time just putting the entire English dictionary into a computer in any meaningful way. I'd like to say learning algorithms (how do we know Keyar is a candivore?) but I'm aware it sounds like a cop-out in the sense of "we'll just synthesize adamantium out of mithril!" kind of way; but I don't think anyone has an easy solution to this one.

    "The problem is getting a computer to make the sensible choice without cultural context."

    I don't think a computer needs to always make the sensible choice. If the time it takes for the operator to recognize the mistake, correct for it and then for the computer to perform the calculations is still shorter than performing the task otherwise, I would consider that Mission Accomplished.