As you might know from its introduction post, September is a young, experimental language that is very much obsessed with function calls. Everything in it is a call - including control flow structures that any sane language would handle explicitly in the parser. Getting rid of all the syntax by replacing it with function calls is fun, but not all that easy. Today, I’d like to guide you through my attempts at doing so while comparing what September can do to Javascript and Ruby at the same time.

Since the topic is pretty deep, I’m going to split it into a series of posts. This time, we’re going to focus on getting if and while working and leave the juicier constructs for later.

Make myself an if

Let’s start with the most basic building block when it comes to control flow - the venerable if statement. In its simplest form, it needs just two things: a body to execute, and a condition guarding the execution of the body.

Since our goal is to make if into a function, we have to somehow get the condition and body inside the function. The obvious way to do so is to pass them as parameters.

Passing the condition in is straight-forward. The expression we use (e.g. 2<5) will be evaluated before the call. When the function receives its parameter, it will be a simple boolean value - True or False. But how to pass the body in?

If you are familiar with any language that treats functions as first-class citizens, you probably have the answer already - we pass it in as a function! If we have a way of declaring anonymous functions inline, we should have everything we need to get our if off the ground.

Up until this point, there is nothing special to what we are describing. Many languages can already do all of that. In fact, I can easily write down our if work-alike in Javascript:

var provided = function(condition, body) {
  if (condition)
    body();
};

This indeed works. The only issue with it is the usage.

provided(2 < 3, function() {
  console.log("Indeed.");
});

Ugly, isn’t it? Since we’re free to design our language however we like, we can surely do better.

Deuglification

We’ll try to make this prettier step-by-step. The first thing we notice is the cumbersome function(){...} notation. If September is going to lean so heavily on anonymous functions that every if requires using one, we’d better come up with something nicer. Here is a thought:

Wrapping statements in {} creates an anonymous, parameterless function out of them.

provided(2 < 3, { console.log("Indeed."); });

That’s better, but we still have to specify the body inside the argument list, which looks all kinds of weird and forces us to properly close multiple levels of nesting at the end. What we would really like to do is move the body out of the argument list altogether. Time to make another rule!

An expression of the form <function call> <anonymous function> means that the anonymous function should be added as the last argument to the function call.

provided(2 < 3) { console.log("Indeed."); } 

Mission accomplished! Other than the name provided, this syntax is conventional enough that most curly-brace languages would happily compile it as a conditional statement. But for September, this is just a fancily-written two-argument call.

A quick detour to Rubyland

If you’re familiar with Ruby, you might still say we haven’t done anything new (if you’re not, bear with me - this will only take a minute). Sure, Javascript might be hampered by its sub-par syntax, but Ruby can already do the same thing! In fact, it’s trivial to write provided in it:

def provided(condition, &body)
  if condition
    yield  # in non-Rubyesque, execute the provided 'body'
  end
end
provided(2 < 3) { puts "Indeed." }

Ruby can do many impressive things with this concept. Most looping constructs in that language are actually implemented as functions or methods, and Ruby’s users love the power and freedom this feature provides. Since we’re doing something similar with September, it seems we’re on the right track!

There is however one fancy loop which Ruby cannot implement this way - while.

To evaluate or not to evaluate

The while loop seems so superficially similar to if that we might think it should just work using the same principles. Unfortunately, it doesn’t - due to something called eager evaluation.

Ruby, and indeed most existing languages, evaluates any expressions specified as arguments before calling the function they are passed into (eagerly).

while(x < 10) {...}

When we make this call in Ruby, the x < 10 part only gets evaluated once. On the inside of the while, we’re just going to get True or False (whichever it was before the loop), with no way to get the original expression back.

Other languages (like Haskell) use lazy evaluation. When you pass an expression as an argument, the expression itself is passed and nothing is evaluated. The expression is only evaluated when the function first asks for the argument, and the value is stored - later references to the argument will not evaluate the expression again, but return the cached value instead.

Unfortunately, this evaluation scheme won’t work for while either. We will be able to pass x < 10 inside, but it will behave like a collapsing wave-function in quantum mechanics. Once we check its value, the real thing is lost - replaced with whatever value we observed the first time. Lazy evaluation is a good source of inspiration, but we need a more controlled solution.

Inverting control

Let’s take a step back and think about how we might try to bypass eager evaluation. When we look back to how we implemented if, we actually have the answer already. Instead of using the condition expression directly, we can pass in a function that evaluates that expression instead! Note the curly braces:

while{x < 10} { x=x+1; };    # almost!

Note: We have to extend our earlier parsing rule to make <id> <anonymous function> parse as a one-argument function call.

This should work. Whenever we need to evaluate the condition, we call the condition function. Since it’s a closure, it can get the correct value of x, and we will get an up to date boolean for our comparison.

The only problem is that this syntax is not what we wanted. What’s worse, we broke the pleasant symmetry between if and while, which means we have to not only remember function names, but also whether to use parens or braces. Should you forget and use the wrong one, you won’t even get an error - just the eager evaluation behavior described before, which will usually get you in an infinite loop at runtime and ruin your day.

So we can’t do that. It seems like requiring the caller to specify how the expression is treated is a bad idea. But what if we flip it around and let the function itself decide how it wants a parameter passed?

If the name of the parameter starts with ?, the parameter is lazy. Instead of the value of the argument, the function will receive the closure for calculating this argument instead. At any time, it can call resolve() on the closure to evaluate it once and get the current value.

This one simple rule allows us to make an intuitive while. The loop condition is made into a lazy parameter, so we get a closure even when using normal parentheses. Whenever we go through with another iteration, we call condition.resolve() to get the current value, and finish the loop if it’s false. Simple as that.

The butterfly effect

The lazy argument rule might look simple on paper, but it has far reaching consequences in the design of September‘s virtual machine (September is compiled down to bytecode before execution, like Java or Python). Let’s look at a sample call:

random_function(x < 10) { x = x + 1; }

Imagine you’re the compiler and you have to decide how to pass me the x < 10 - as a value or as a closure. What you quickly realize is that you don’t have enough information to do that. For all you know, random_function might really by a devious alias for while that I put there specifically to mess with you. In order to make the right decision, you would have to know the signature of random_function during compilation. But September is dynamically-typed, so that will only be known at runtime.

This in turn means that every call has to be compiled using lazy arguments. Every expression you use as an argument in your source code becomes a function in the bytecode. At runtime, if it turns that the parameter is eager after all, the closure will be very short-lived - the VM will evaluate it immediately before passing control to the target function.

Such a roundabout fashion of making calls might seem strange and make us worry about performance issues, but it empowers us to do a very interesting thing. We have moved the control over when and how a parameter is evaluated away from the caller and into the function being called. Thanks to this, our humble functions can tread in places where normally only macros dare to go.

Next week on “Getting rid of keywords”

Today, we’ve covered how to do if and while as functions without special-casing them in any way, in effect duplicating and surpassing what Ruby can do with its block passing. In the next post, we’ll see how to extend our solution to more complex statements, like if..else and other multi-parters that go beyond having a single “body”.