Joshua Bloch, Chief Java Architect at Google, gave a talk entitled “The Closures Controversy” at Javapolis in December 2007. I found it online through reddit, and it intrigued me, because I think it illustrates a disconnect between what we as an industry are doing and the goals we have. Bloch also makes what I think is a reasonable argument for limiting Java.
Those of you who’ve read what I’ve written for a while might be surprised by that statement. I have been critical of the IT industry for not thinking in more powerful terms about programming/computing. I am not endorsing Java. After watching his talk I was prepared to lay into Bloch for not having higher ambitions for the language, but as I thought about it I realized that it’s unreasonable to expect a computing environment to be more than what it was designed for. It helps me see that Java has a shelf life. To expect more would mean trying to change it into something it’s not.
The appeal of Java
Bloch began his talk using Gosling’s ideas from his article The Feel of Java, published in IEEE Computer in 1997, as a touchstone for what Java ought to be like in the future:
It is a language for a job. Doing language research was an anti-goal [in the development of Java] . . . It is practical, not theoretical. It’s driven by what people need. Theory provides rigour, cleanliness, and cohesiveness. [Java had] no new ideas. It took ideas from existing languages and put them together. It maybe added a few new things, but that wasn’t the goal. And this is perhaps the most important slide in the whole talk. He said, ‘Don’t fix it until it chafes’, to keep it simple. Don’t put anything in the language just because it’s nice. Require several real instances before including any feature. In other words, ‘Just Say No, until threatened with bodily harm.’ He said Java feels hyped, playful, flexible, deterministic, non-threatening, and rich, and like I can just code.
Bloch continued from the article:
Java is a blue collar language. It’s not a PhD thesis. It’s a language for a job. Java feels familiar to many different programmers, because we preferred tried and tested things.
When Gosling said Java is “not theoretical”, he was comparing it to languages/VMs that are based on computer science theory, like Lisp, which is based on lambda calculus. By “practical” he meant the instrumentalist approach to design, which is that the feature set and the architecture the language/VM supports should handle current problems. Nothing more, nothing less. In my view this is short-sighted. It means that after acquiring and using the language to solve current problems you will run into problems the language wasn’t designed to handle, because new challenges arise. You will have to improvize within a structure that is ill-suited to the task, until whoever is in charge of the language adds new structures that handle the problem effectively. Wash, rinse, repeat. I have a feeling that this approach is a defense against languages whose designers add features willy-nilly without consideration for how it fits into the overall goal of the platform. Bloch points out that “theoretical” languages don’t have this problem, but he passes by it quickly.
Java was a swing in the other direction from languages like C/C++ that were said to “give you more than enough rope to hang yourself with”. I guess programmers “hung themselves” more often than not. C and C++ are powerful in the sense of allowing you to access the hardware directly inside of a language that gives you high-level structures. If you’re interested in doing anything more than building kernels, device drivers, or VMs/compilers, they’re pretty weak. Java rode a wave that was happening at the time it was introduced. C and C++ were the dominant languages. All sorts of projects were done in them, including applications. Java provided a productivity boost to project groups with this orientation by getting rid of many of the pitfalls of using C/C++ (buffer overruns, memory leaks, null or uninitialized pointer references, etc.), but in terms of building more abstract software like applications/environments with the power the language and the VM gave you, it wasn’t that different from C/C++. So in essence it played to the tastes of programmers of the time. It went for familiarity, but without having a particularly good technical reason for it. Java doesn’t allow direct access to the hardware. So why did the language designers feel it necessary to play “follow the leader” with C/C++ in terms of conventions and power? It was an easier sell to programmers. They like familiarity.
Java meets the programming culture where it is. As long as the culture is ignorant of the fact that the architecture it’s using is ill-suited to the task at hand, then technologists can convince themselves and others that problems which arise from using a limited architecture for large projects just go with the territory, and there’s nothing anyone can do about it. It’s about time we became more enlightened, but it’s going to take us admitting that despite our masterful technical skills we don’t know very much about why it is we do the things we do with computers, beyond the joy of solving puzzles. According to Alan Kay, and I think he’s right about this, it will take learning about and exposing ourselves to other things besides computers.
Quoting Kay, from The Early History of Smalltalk:
Should we even try to teach programming? I have met hundreds of programmers in the last 30 years and can see no discernible influence of programming on their general ability to think well or to take an enlightened stance on human knowledge. If anything, the opposite is true. Expert knowledge often remains rooted in the environments in which it was first learned–and most metaphorical extensions result in misleading analogies. A remarkable number of artists, scientists, philosophers are quite dull outside of their specialty (and one suspects within it as well). The first siren’s song we need to be wary of is the one that promises a connection between an interesting pursuit and interesting thoughts. The music is not in the piano, and it is possible to graduate Julliard without finding or feeling it.
I have also met a few people for whom computing provides an important new mataphor for thinking about human knowledge and reach. But something else was needed besides computing for enlightenment to happen.
Tools provide a path, a context, and almost an excuse for developing enlightenment, but no tool ever contained it or can dispense it.
The state of Java today
Bloch’s introduction felt like cheerleading for Gosling’s vision for Java, but then he got into what’s painful about it now. He talked about the Java community’s involvement in the design of Java, and the wisdom of its actions. “How are we doing today?”, Bloch asked the audience. He showed several examples of messy Java generics. It didn’t look that different from C++. The audience laughed. He said, “Not so well, unfortunately. This is how Java looks now.”
The way he described things made it sound like Sun “jumped the shark” with generics. He was more polite than to say this. He said generics per se were not a bad addition to the language, but their implementation allowed so much freedom that Java code is becoming incomprehensible even to people who have been working with the language for years.
Generics: An escape hatch
In my view generics are just an attempt to get out from under the limitations of Java’s strict type system. The same goes for generics in .Net.
Generics are the “managed code” world’s version of C++ templates. They are also called “parameterized types”. They give you some of the programming flexibility of a dynamic language. Statically typed versions of generic code get generated depending on what types are called for in statically typed code. What they don’t give you is the “get to the point” feel of programming in a powerful dynamic language, because you have to tell the compiler about how many types the generic code will accept, and in what form it will accept them, where type information needs to go in the generic code, and what form type information will take when data is returned from the routines you write. In other words, you have to give some metadata for your code so that the static type system can deal with it. I call it “overhead”, but that’s just me. What static type systems force you to do is talk about the data flow of your code. This isn’t so that humans can understand it (though with some discipline it’s possible to create readable code this way), but so that the compiler and runtime don’t have to figure out what the data flow will be when your code is executed. It’s really a form of optimization.
Bloch explored where the complexity that now bedevils Java comes from. He got into talking about feature creep. It’s not the number of features, it’s the permutations of feature interactions. “When you add a feature to an already rich language, you vastly increase its complexity,” he said.
The “quagmire” of closures
First, let me get a definition out of the way. A closure can be thought of as an anonymous function. It is not officially a method in any class. You can spontaneously create closures inline in your code and use them, in dynamic languages. You can pass them around from method to method, and they never lose their integrity. In the context of OOP it’s more aptly defined as an anonymous class that contains a single anonymous function, because in OOP languages closures are objects with methods of their own.
Bloch gave a brief explanation, which I think is correct: closures “enclose their environment”. They create references to whatever variables are used within them that are in scope range, at the time they are created. This is called “lexical scoping”. They also automatically return the value of whatever is the last expression evaluated within them. Bloch gives two reasons why closures are being considered for Java:
-
They facilitate fine-grained concurrency in fork-join frameworks. You can use anonymous classes with these frameworks, but they’re a pain right now. He said it’s important that programmers use fork-join frameworks because, “we have these multi-core machines, and in order to actually make use of these cores, we’re going to have to start using these parallel programming frameworks.”
-
Resource management - Always using try-finally blocks for resource management is painful and causes resource leaks, because programmers misapply the construct. Closures would help with this.
Even though he conceded these points, he clearly has disdain for closures. He used BGGA closures as his example. With the way they’ve been implemented, I’d have disdain for them, too. The hoops you have to jump through to make them work makes me glad I’m not using this implementation.
He said closures encourage an “exotic form of programming”, which is the use of higher-order functions: functions that take functions as parameters, and/or return functions. He jabs, “If you give someone a new toy, then they’ll play with it.” Ah, so higher-order functions are a “toy”. Mm-hmm… I’m tempted to say something impolite, but I won’t.
Bloch talked about closure semantics next, and how they’re different from normal Java semantics. He expressed concern throughout the rest of his talk that this would confuse Java programmers, and thereby create bugs. He has a point. Still, I don’t mind the idea of mixing computing models in a single environment (VM). I think there are advantages to this approach.
Some history: Where modern computing comes from
Code blocks (closures) in Squeak/Smalltalk work differently than normal Smalltalk code as well.
I’d venture to say that the reason closures have been put in Ruby, and are now being considered for Java, is that Smalltalk was the first OO language to use them. Even though I’m sure it’s largely forgotten now, there are some significant artifacts of computing that have come into common use because of the Smalltalk-80 system. The GUI you’re using now, “hibernation” on your PC, bit-blitting, and the MVC design pattern are a few other examples. Language and system designers have been mining Smalltalk features for years, and incorporating them into modern, though less powerful technologies.
In retrospect people within the Smalltalk community have pointed to some design flaws in it. They didn’t ruin it by any means, but it’s recognized that there’s room for improvement. In a conversation I was in on between Alan Kay and Bill Kerr, a blogger I’ve mentioned previously, Kay expressed muted regret about the inclusion of closures in Smalltalk, saying they broke the semantic simplicity of the language. I can see where he’s coming from. Closures introduce an element of functional programming into OOP. He desired a system that was purely OO, mainly for consistency so the language would be easier for beginners to learn. I haven’t gotten far enough along in my research to see what Kay would’ve suggested instead of closures. My understanding is he was heavily involved with the development of Smalltalk up until Smalltalk-76. Due to a schism that developed in the Smalltalk team, he gave up most of his involvement in it by the time Smalltalk-80 was in development. This is the version that ultimately was released to the public. “The professionals”, as Kay put it, took over and so some of the vision was lost. They turned the Smalltalk language into something that was friendly to them, not beginners.
Closures are something that anyone new to a dynamic OO language has had to learn about. They’re pretty painless to use in Smalltalk, in my opinion. They’re easy to create. Contrast this with closures in Java. Bloch referenced a post written by blogger Mark Mahieu, where he got some of the examples. Here are a couple of them, with equivalents in Smalltalk for clarity.
Java example #1:
static <A1, A2, R> {A1 => {A2 => R}} curry({A1, A2 => R} fn) {
return {A1 a1 => {A2 a2 => fn.invoke(a1, a2)}};
}
Smalltalk equivalent:
curry := [:fn | [:a1 | [:a2 | fn value: a1 value: a2]]]
Java example #2:
static <A1, A2, A3, R> {A2, A3 => R}
partial({A1, A2, A3 => R} fn, A1 a1) {
return {A2 a2, A3 a3 => fn.invoke(a1, a2, a3)};
}
Smalltalk equivalent:
partial := [:fn :a1 |
[:a2 :a3 |
fn value: a1 value: a2 value: a3]]
This is what I meant about “hoops”. The type information you have to supply for the Java closures detracts from the clarity of the code. It’s clunky. That’s for sure.
I’m not going to say anything about these structures (”curry” and “partial”) right now. If readers would like, I can write a post just on them, explaining how they work, and how they can be useful.
Bloch’s proposal
Bloch put forward a proposal, that rather than adding closures to the language, at least in the near future, that Java designers instead focus on creating concise structures (ie. add features) that implicitly create anonymous classes, to eliminate the verbosity (I guess this was his answer to the challenge of concurrency), handle resource allocation, and critical sections for threading. Basically it sounded like he favored the stylistic approaches that Microsoft has taken with .Net, with some differences.
He rejected out of hand the idea of adding library-defined control structures to the language, something made possible by closures, because it will encourage dialects. Squeak has a lot of library-defined control structures. I think they work quite well. They don’t change the fundamental rules of the language. Instead they expand its “vocabulary”. That is, they expand upon what you can do using the language’s existing structures. I understand this is hard to imagine in the context of Java, because most programmers are used to the idea that control structures are invariants in the language. Bloch would probably feel uncomfortable with this concept, but ask yourself which language is getting more cumbersome and complicated as time passes?
I think there is a fundamental contradiction in goals between what Bloch says he prefers about Java, and the growing problems of computing. He says that adding features to a language increases its complexity, and he makes a good case for that. Yet, how is Java going to solve new problems in the future? His answer is to add more features to the language. Granted they’re focused, and not open-ended to allow further expansion, but they’re still feature additions all the same. As is illustrated by Squeak, library-defined structures would actually help reduce the number of features in the language. Even though Squeak can do more things concisely than Java can, the feature set of the language is very small. As a point of comparison, Lisp’s language feature set is even smaller, yet it’s more capable of handling complex problems than Java. They accomplish this by taking what Java does as a built-in feature, and use message-passing (Squeak), closures (Squeak, Lisp, others), an extensive meta system (Squeak, Lisp, others), named functions (Lisp, Scheme), and macros (Lisp) instead.
Where Java sits
I agree with Bloch that if the intention of Java was to be a language that is “practical” (ie. is designed from the standpoint of instrumental reasoning), easy enough to use so “you can just code”, and doesn’t strain the ability of Java programmers to understand their code, then generics should’ve been scaled back, closures shouldn’t be added, and his suggestions for expansion should be considered. Unfortunately this also means that it’s going to get harder and harder to work with Java as time passes, because the demands placed on it will exceed its ability to deal with them. Maybe at that point people will pursue another “escape hatch”.
Struggling to transcend Java
I compliment the Java community on its efforts to build something better out of a limited platform. What some are trying out is the idea of a VM of VMs. That is, building a VM on top of the JVM. This creates a common computing environment that enables some interoperability between different computing models, while at the same time transcending some of the limitations of the JVM. From what I hear, it sounds like it’s been a struggle. Personally I think it would be wiser for the industry to start with something better–a “theoretical” language/VM–and then try to build on top of that. Our computing culture has a lot of resistance to this idea. Even the efforts of groups to build better language/VM architectures on top of the JVM seem to be regarded as curiosities by most practitioners. They get talked about, and get used by a relative few who recognize their power for real projects, but they’re ultimately dismissed by most developers as “impractical” for one reason or another.
It’s funny. Several years ago I talked with a friend about how slow Java was. His solution was to “throw more hardware at it” then. Some months back I told him about Groovy. He read up on it and dismissed it as too slow. Why not just “throw more hardware at it”? I didn’t hear that recommendation from him. I didn’t press the case for it, because I didn’t have a dog in the fight. I’m not part of the Java community. I figured if he didn’t want to pursue it, he either had a good reason for it, or it was just his loss. My inclination is to believe the latter. I guess by and large there’s not enough pain yet to motivate people to reach for something better.
—Mark Miller, http://tekkie.wordpress.com
The “corporal cuddling” part was precious. Awww, so cute! Makes me feel all warm and fuzzy inside.
