For some months I’ve taken time to study dynamic languages, particularly Lisp and Smalltalk. I’ve kind of wondered why though. There is a certain elegance that attracts me to them. I also like the philosophy that Dr. Edsgar Dijkstra espoused, which is that the computer should enable the programmer to express his/her idea to it, and let it figure out how to execute it. That idea sounded great to me. I didn’t quite know what he meant, but I knew it’s something I desired. I knew I liked Smalltalk’s elegance, because I remembered admiring it years ago.
Some time back I read a post by Ramon Leon on domain-specific languages (DSLs), and how Smalltalk fit into that picture. He linked to an online video of a presentation by Martin Fowler, talking about this. It was very interesting. Ramon continued talking about DSLs in subsequent posts. Each time clarifying it in my mind. Over time I’ve come to realize this is the answer to why I’ve been so interested in these languages.
In languages like Lisp, Smalltalk, and Ruby it’s possible to “embed” a domain-specific language within the language. Lisp programmers (“Lispers”) are famous for this (insofar as they are known at all). It’s often been said that Lisp is a language for writing other languages. I wasn’t sure what that meant, but I think I’m beginning to understand. Fowler explained that in a way it’s possible to create DSLs in mainstream languages like Java as well.
Fowler discussed the different options for how to implement a DSL. One way is to create a language native to the problem domain. This is expensive, because you have to craft your own parser and your own virtual machine/interpreter, or code generator. He mostly discussed the technique of embedding a DSL in a general-purpose language.
Using the embedding technique, a DSL is a collection of objects and methods, or functions and variables (in the case of a non-OO language like Lisp), that enable a developer to totally focus on the problem domain, and avoid language primitives and API calls that are built into the general-purpose language platform, unless they directly relate to the domain. It forms a kind of language that is specific to the problem. Depending on the general-purpose language used, you can see the DSL coming through. In Java and C#, it’s more difficult to see, since they require you to use type specifiers and a method call notation that clearly delineates what is considered code and what is considered data. And it’s difficult to add methods to standard classes. This can come in handy when writing DSLs, because it’s all about terminology. Sometimes a term that would be most intuitive is already taken by a standard class, and you’d like to add a few more operations that apply to your problem.
Some might ask, “Why would you want to add methods to standard types? Isn’t this dangerous?” When using the DSL technique, you are only redefining the language for the problem at hand. You are not redefining the standard types to that problem domain from then on for all other projects. Typically you would start with the standard types as they were when the next project comes along.
In Lisp, Smalltalk, and Ruby (in that order) the native language syntax is minimalist, and you can add methods to the standard types like falling off a log (in Lisp you can redefine standard functions). So the DSL has an opportunity to “shine through”.
If nothing else, adopting the DSL mindset will have you creating code that on its face makes more sense, because it directly relates to the problem at hand. Using the technique Fowler talks about, depending on the language, it may seem more like a hat trick, because you are still using the same syntax and semantics of the general-purpose language. You can think of it as “painting over” parts of the general-purpose language and putting new labels on things, new nouns and new verbs. You can get more creative, replacing what should be a noun with an adjective, and replacing a verb with a noun, so it makes more sense. There’s an example of this at a link below, when I mention a package called Albatross.
Fowler in his presentation showed an example of a simple DSL for parsing a text file with fixed-length fields, and how it can be manifested in a general-purpose programming language. He showed two examples, one in Java and one in Ruby.
Ramon Leon came up with his own version of a DSL in Smalltalk for the problem of parsing the same text file that Fowler showed in his presentation. You can see it at the link above.
You can see another example of a Smalltalk DSL, created for a web app. testing package called Albatross at this post on Ramon’s blog, On Smalltalk. Maybe I’m going overboard here, but this last example is a thing of beauty, in my opinion. What it’s doing is so obvious. It’s a pleasure to read. This is kind of the point. It not only makes code more understandable, it’s also easier to construct solutions to problems.
The future of software development
Microsoft is adding features to .Net that will enable easier renditions of DSLs in that platform. The Linq Project team is adding new features to C# and VB.Net that make them more like dynamic languages. They will introduce dynamic types and lambda expressions, plus what they call “extension methods”, which will allow methods to be added to any type, including a standard type, that is used within a certain scope. Lisp, Smalltalk, and Ruby have, in effect, had these features for years. In any case it will save programmers time, and make their code more understandable.
A few years ago I heard that Charles Simonyi, a former Microsoft developer, had started his own company, Intentional Software. He said he was developing a tool for .Net that he hoped would solve a glaring problem in the software industry. I personally haven’t run into this problem on projects, but from what I hear it’s common for a customer or corporate manager to discuss a project with a development team, or pass spec’s to them, and receive a product months later that is nothing like what they asked for. Something got “lost in translation” between the domain expert (the customer or manager) and the implementation from the dev. team.
Simonyi said that in his experience, large applications like Microsoft Word eventually develop into their own sort of language, or API. Simonyi said he was working on a system whereby the domain expert would be able to graphically specify what they want, on their terms, and that it would translate the specification into a domain-specific model in .Net that the developers could work with. They could then fill it with the logic that would make the application work. He didn’t say it, probably because it would confuse most readers (I think it was in BusinessWeek, something like that), but what he was talking about was creating a tool for creating DSLs.
I have a feeling this concept will become the future of software development. It’s been done in dynamic languages for years. In terms of mainstream technologies it’s not here yet, but it’s coming.