Second, JRuby 1.1b1 has been released and as expected is considerably faster (see item #5 in this link) than the standard “MRI” runtime. JRuby joins Jython and Rhino in providing a JVM-based runtime for a dynamic language, with features designed to help developers mix and match the dynamic language code with Java code.

See the trend here? Python, Ruby, and JavaScript are emerging as the dynamic languages of the future for .NET and Java developers.

The hard work done by Sun and Microsoft to make their VMs work well is being leveraged by the next wave of languages. Threads, high performance I/O, memory management, and portability are all features that are quite expensive to get right, and the .NET and Java platforms have pretty much achieved that at this point. (Piggybacking newer, higher-level languages on these mature runtimes means that you get a mature new language runtime faster than if each language’s runtime were built from scratch and painstakingly debugged in isolation from the others.)

There are still some hurdles (performance, type safety fears, lack of mass market acceptance, ECMAScript 4 standardization and adoption, etc.), but in 2 or 3 years, things are going to change dramatically in the web application development world. The seeds of this change are already sown, and it’s just a matter of time. Threads, SQL, OOP, and garbage collection are all features of web application architectures that were initially controversial, but have now met with general acceptance. Dynamic languages are clearly the next step.

Obviously, Java and C# are far from dead, and in 10 years people will still be coding in Java and C#, because as with other languages like C and assembly, the newest and highest-level language isn’t automatically right for every project. But if you’re building web applications, most of what your code does falls into the categories of string manipulation, collection operations, or file and socket I/O. Image processing, crypto, full text search, and other CPU-heavy, byte-twiddling features may be part of your application, but you’re not writing the image scaler, RC4 cipher, or inverted index yourself; those are done in a library, probably written in C, and you’re just calling it. So your needs are likely to be similar to the sweet spot of dynamic languages: maximum expressivity and the fancy features to let you write clever code, making you productive and making the code as clean and elegant as possible. In other words, they put developer productivity first (lower labor cost and shorter development schedules) at the expense of runtime performance. Since hardware gets cheaper over time but code gets uglier over time, this is probably the right choice to make for most web application projects.

Another interesting benefit of layering instead of starting over is that the integration between dynamic languages and Java or CLR languages is much nicer than managed vs. unmanaged code in .NET or, even worse, JNI in Java. That is, it won’t be a bloody mess to mix and match code, from a technical feasibility standpoint. This matters, because The Big Rewrite is among the Things You Should Never Do. But little bitty rewrites are fine, especially if you have a thorough test suite to help you avoid breaking things. (By the way, dynamic languages are great for writing automated tests.)

Which of these three (JavaScript, Python, or Ruby) is going to be dominant 5 years from now? I don’t think any of them will be. The dynamic language community is fragmented, and the various vendors and big sponsors of these three languages are fairly entrenched already. Microsoft is investing in all three; Google has standardized on Python and JavaScript to the exclusion of Ruby; Sun has hired the JRuby team; Mozilla is heavily invested in JavaScript; Adobe supports JavaScript in AIR but not Ruby or Python, etc.

In fact, if you encapsulated the glue code sufficiently well, you could mix and match JavaScript, Python, and Ruby in your application, and port your hideous hydra between the JVM and .NET. You would be wasting a lot of effort since the three languages are largely similar, but you could do it. Alternatively, you could create a portability layer between the DLR and the JVM a la WxWindows, and write-once-debug-everywhere in a more productive language than Java.

These are all repugnant ideas, but only because as I write this and as you read this, we probably realize that to attempt this today would be a huge task. But what about in 2010? Probably not so gross. What about an application that could be executed on Silverlight, AIR, Firefox, SWT, and Mono, unmodified? How about a mobile app that runs on smartphones regardless of the runtime (.NET vs. J2SE)? Not gross at all, and not unthinkable if your app is written in JavaScript using some kind of portability layer that doesn’t exist yet.

In the longer term, JavaScript (a.k.a. ECMAScript 4) is likely to become extremely popular. As far as I know it’s not quite a perfect fit for Steve Yegge’s The Next Big Language, but it’s the closest thing there is, and it has two critical advantages over Ruby and Python that will make it successful: C-family syntax (which makes development tools cheaper to build) and effectively unanimous buy-in from vendors and developers.

So, what about the other dynamic languages that people are using in large numbers today? What’s going to happen to ActionScript, CFScript, PHP, and VB?

ActionScript and CFScript are pretty close to JavaScript by design; I’ve read that ActionScript 3 is actually compliant with the ECMAScript 4 draft specfication. It’s pretty clear that Adobe is betting on JavaScript. In the near future (2 or 3 years) I predict that Adobe will rev its products and support ECMAScript 4 across the board.

PHP and VB.NET/VBScript will hang around for a long time because they’re approachable and already very popular, but they’ve already peaked, and will steadily decline as developers switch to C# (on the .NET side) and Rails (on the Linux side), and then JavaScript as soon as a serious web app framework and an ISP-friendly runtime exist. Microsoft will keep investing in VB to keep customers happy; Yahoo will keep investing in PHP because it is so heavily invested in PHP already; new developers will find PHP to be an easy starting point for light duty web development, with tons of documentation and free applications that they can download and hack. But PHP will not inherit the kingdom from C# or Java, and the languages which do achieve mainstream success after C# and Java will do everything that PHP does language-wise, and the market momentum around those languages will make them better than PHP at what PHP does. Developers will ask themselves why they would write the client side and server side in two different languages, especially when the server-side language is more expressive and has better portability and libraries. That’s not true yet, but it will be in a couple of years. In 5 years or so PHP and VBScript will go the way of Perl CGIs: still used, but by a community a tenth of the size it is today.

What about the new Java-based dynamic language, Groovy? Groovy is interesting, but it’s too late. The Java mainstream of vendors and developers only recently managed to convince the world of “serious” C++ developers that automatic garbage collection and JIT compiled bytecodes can actually work in a high traffic context. The next battle, to promote the dynamic language features that Java lacks but which Groovy brings, will take years to fight. Once a developer makes a decision to not use standard Java, Groovy is on a more or less level playing field with the JVM-hosted versions of Python, JavaScript, and Ruby, but each of those languages has far greater adoption than Groovy, and each of them has greater opportunity for leverage on other runtimes than Groovy. For a Java developer, once the door is opened to other languages, the only advantage Groovy has is that its syntax is familiar. Compare this to JavaScript which web developers also need to know how to use; why learn a third language (Groovy) in addition to Java and JavaScript? Over time the simplicity of coding and debugging in JavaScript on client and server, together with dynamic-language productivity, will overcome the momentum of the Java standard, and web developers using server-side Java now will gradually replace it with JavaScript on the JVM. Conservative attitudes in the mainstream Java community (including Fortune 500 companies and the many offshore development firms that write code for them) will make this take quite a while – probably 5 years before JavaScript becomes a common part of the architectures that currently use J2EE, and 10 years before Java goes the way of COBOL (maintained forever but not used for new projects).

So in conclusion, keeping an eye on the future value of a technology, including who’s investing in it and who’s talking about investing in it, is critical to making your own investments today. In five years you’re not going to be using the same technology stack that you are today, and your project’s success and your own salary will be tied in large part to how well you invested today.

Eclipse is an IDE so it’s hard to imagine it not being baroque and difficult to use, requiring weeks of effort to become productive. JBidWatcher has saved me a lot of money on eBay so I could probably put a dollar value on how much it’s worth to endure its bizarre UI. Azureus is fairly fugly also but it does a very good job and has a deep, sophisticated UI that’s fairly easy to understand, so despite the eyesore, it’s at least fairly clear. The common thread among all of these is that they are all written in Java, and that they are so valuable that it’s worthwhile to overlook the ugly UIs.

Now imagine those sorts of trade-offs, but on already difficult to use mobile devices, and aimed at consumers. Are you making a strategically wise choice by sacrificing usability and control over the user interface, and probably access to platform-specific features such as dialing the phone, in order to save money on development? Adam Breindel talks about this in When Building a Smartphone App, Resist the Siren Song of J2ME.

Adam and I worked on a J2ME application and I totally agree with him about the disillusionment of trying to write a single app that would work across phones. Issues include:

• Complex and difficult application installation procedures for end-users: How do you install the JVM on the phone? How do you get the plain J2ME app packaged up so that the phone will accept it? Does the app require manual user configuration before use? Is there a different launching process from other apps?
• Lack of control of the user interface: hardware details such as how many buttons you have, whether there’s a stylus, etc. differ from phone to phone, and the API to let you code once and let J2ME handle the layout for each device leaves your code very disconnected from what’s actually happening on the screen.
• Not being able to use recent J2ME APIs because even the latest phones only support older, more minimal J2ME APIs
• Not being able to do things on a phone that would seem obvious, like dialing the phone, opening a hyperlink in the phone’s browser, sending an SMS, or making a network connection. Either these are entirely impossible or require phone-specific or JVM-specific tools and procedures to sign your application, or having the runtime nag the user to request permission to do something that they just asked the app to do for them.

For all the noise Sun is making about broad J2ME penetration, the developer experience is quite disappointing, and as a result, the user experience is also quite disappointing. You can look up J2ME features and APIs and get excited, but when you actually deploy your app to a handset, it won’t load, or runs terribly slowly, or looks awful, or simply doesn’t do the things that the API says will happen when you call it a certain way. Suddenly the strict J2SE and J2EE logo certification programs make sense, because the J2ME approach of making so much functionality specified but optional leaves developers high and dry. The phone supports J2ME version xyz, but write an app coded to that API that works on the emulator and deploy it to a handset and lo and behold, all those optional APIs turn out to be missing even though the handset is capable of that functionality, and some mandatory API functions are not working. Here be dragons.

Case in point: can’t dial the phone on a Treo 650 (at least, not as of a year ago). The J2ME API tells you how to do it. The PalmOS JVM (made by IBM) lets you make the API call, and returns a successful response. Nothing happens. IBM says they’re aware of this issue. The end. The docs say you can, the code you write says you did, the phone just doesn’t do it.

Case in point: Every time you start an application and it accesses the network for the first time on a Treo 650, the user is nagged for permission to access the network. Quit the app and start again, nagged again. IBM has a tool that you can use to sign the app, but you have to use their VisualAge Micro Edition IDE which costs hundreds of dollars to do that. Try and find and download the trial version. A year ago, it was not possible. So, making that persistent nag go away probably costs several hundred dollars. (I never verified that it actually works, just that IBM says the way to sign the app so that it’s trusted is to do that, and that there was no available free way to get that tool.)

These are minor issues, but they certainly interfere with the quick usage pattern of a mobile app, and make it annoying to use your app. Imagine what that would be like if you had a competitor with a native application for that phone, whose application probably cost them more, but their app is better and the user likes it a lot more.

The important distinction here is not cost, it’s ROI. It costs a lot to develop a similar application for each smartphone platform, using that platform’s native tools. It would seem to cost a lot less to develop a J2ME app. But that’s only if you assume that it’s OK to abandon features and settle for a horrid user experience in the course of development.

It’s likely that your goal as a development team is to develop an app that has a predefined feature set that you know the device can support, and a predefined UI design that your mobile-savvy UI people are sure will go over well with users accustomed to that particular kind of smartphone. In that case you will almost certainly fail to accomplish that goal using J2ME. You have to scale back your goal so that you’re satisfied that you got something kinda like what you wanted working on a bunch of phones, and determined users will probably be able to figure out how to install it and make it work.

I call that phenomenon “write once, be disappointed everywhere.”

Let’s continue talking about cost, though. The native apps may require (or suggest) different programming language skills for different devices. It might seem wise to just write everything in C, but I think that’s a false economy as well. The phone APIs will differ so much that you will really need a native developer for each platform, not a team of generic C developers who will figure out the individual phone stuff and be freely floating resources that you can assign to whatever app version needs their attention. Smartphones may run Linux, may run Windows Mobile, may run PalmOS, may run Symbian… these are different operating systems with very different ideas of how applications run and coexist. The platform specific knowledge (APIs, appropriate UI feel, device capabilities) is probably an order of magnitude harder to learn and maintain than the ability to get an application working in a given programming language.

How much of a great C programmer’s skill is really the C language, and how much of it is proficiency with the available libraries on the platform he or she is accustomed to? I think close to 90% of their professional skill set is platform and library familiarity, and 10% syntax and low-level understanding of how the language actually works.

In light of this (just using C doesn’t mean developers or code are portable across smartphones), consider that there are high level languages available for some smartphones. What if that 90% platform familiarity means they can use a language and/or development environment that makes them 5 or 10 times as productive, after spending just a few days or a couple of weeks learning the language syntax?

After our very disappointing J2ME-on-PalmOS port experience, Adam found Handheld Basic which initially appalled me (oh no, BASIC!) but turned out to be a great choice. It’s a flavor of BASIC, so learning the language didn’t take long, and the support happens to be quite good (lots of code samples) so picking up the library portion of that 90% didn’t take long at all. I imagine that C# on Windows Mobile is similar. As more and more phone start to use Linux as their OS (which will be a particularly huge improvement for PalmOS based phones), you’ll be able to use Python, Ruby, Mono, J2SE (a whole different animal from J2ME), TCL, or pretty much any other high level language available for Linux. At LinuxWorld 2006 I saw a development device running unmodified GNOME desktop apps running on a PalmOS device alongside PalmOS apps. There are more options appearing all the time, and with the exception of Handheld Basic, most of them are ports of familiar, mature, well understood desktop languages, with class libraries relevant to mobile devices.

So, I recommend that you work in whatever high level language lets you do all the platform specific stuff you want, and if that means a different language per phone, that’s actually going to be the least expensive way to get to the apps you actually wanted to build. A big chunk of the scary cost of developing native apps goes away, because native doesn’t necessarily mean abandoning Java for C. (In fact I found Handheld Basic to be a more productive environment for me after a couple of weeks than J2ME was, despite my ~8 years of full time Java experience before starting that project.)

That pretty much means no reuse for you. Sorry, but that’s the deal right now.

If that unique-app-per-platform cost is too scary, consider a few ways to save money:

• For networked apps (aren’t they all?) ask yourself if there’s some logic that could just as easily be done on the server as on the client. Is there some complicated parsing code on the client that could be simplified by changing the response format from the server to something that’s easier to parse?
• Can you remove or alter certain features from a subset of platforms you intend to support, so that your premium supported platforms get your ideal app, whereas a few less popular phones still get a nice app, but perhaps one that doesn’t have every feature available on your premium app. You might be reading this and thinking “but that’s what you said J2ME would force me to do! Why is this any better?” The distinction is that you are in control of the decision of what to leave out to save money, whereas with J2ME it’s the platform vendor who makes that decision for everyone using their platform. If you’re writing an address book, not being able to dial the phone is lethal; not writing the code to let the user attach a photo to the entries is not.
• Can you move some user-facing functionality to the server to make the client simpler? Maybe there are some rarely-used features that could be done via a desktop or mobile web browser, so you can focus on putting the ten-times-a-day features on the handset, and making those features fast and convenient. You probably already made that trade-off in general, but perhaps for some kinds of handset, you’ll move the dividing line a little further, so that for users of particularly rare phones, some moderately frequent features can’t be done on the phone. This could also be a good approach for new handset types: design a “lite” app and a “full” app version, build the “lite” app first on each platform, and let user demand tell you whether the full app is worth it. Your developers can tell you much more accurately how much the incremental functionality would cost, since they already have done the lite version. Maybe you’d provide a VoiceXML interface, or mobile web browser interface, for that feature so that the user can still do whatever the feature requires while they’re far from a desktop PC.

A final consideration: labor. Maybe you have some Java developers, or C developers, and don’t have developers good at Handheld Basic or C#, so you’re not inclined to fire them all and hire new developers to do native apps, you’re thinking maybe C on every handset, or J2ME, is still the right choice. I still say that you should probably use native apps in native high level languages on every smartphone platform. If you’re committed to a strategy of good apps on a bunch of different phones, I think I’ve made clear that native apps are the only way to currently get there; J2ME simply doesn’t let you make good apps. So what’s left is C code written by C developers vs. high level code written by C developers who have to retrain.

As I said above, I don’t see much chance that your developers or your code will be portable across different smartphones if you use C. Maybe you’ll get 5-10% savings that way (some code ported across phones, or some developer hours shuffled between platform teams). But you’d get a 5-10x cost saving from using something very modern and high-level instead of C, and the overhead of training for the language would be very very small as compared to the large and unavoidable overhead of having to learn what the handsets can do and how the APIs for the handset’s OS work.

I suppose that means that if you have a bunch of C developers who lack smartphone skills, you’re in a pickle, but that situation seems kind of unlikely to me (a mobile app company hires a bunch of Unix and Win32 C developers with no mobile phone skills?). More likely is that you’d find a mobile developer who is proficient with one or more handset OSs and the best tools for each one, and they may be of the opinion that C is the best choice since they can reuse skills and some code across platforms. I would say that in that case you need to convince them to (or more likely encourage them to do what they were already considering, which is to) go ahead and find a highly productive high level programming language/environment for each smartphone platform.

A final possibility if you have a ton of platforms to target and a pile of killer C programmers is to try and port something like Python across most of your target platforms, or to make a very high-level API or domain-specific language that runs inside your own custom C portability layer. You might be able to find an open source option that you can invest some developer hours in, so that the actual application-specific code that you write on each platform is minimized and portable. But I suspect that this would still be expensive and would result in some J2ME-like UI abstractions that ended up being very unsatisfying in the end.

Best of luck!

Hell no. Firefox and Thunderbird are built using an incredibly complicated set of tools and technologies created by the Netscape and Mozilla folks, specifically for the purpose of building Firefox and Thunderbird.

Low-level stuff is written in C++, but if you know C++ already you still have to learn their coding standards and class libraries. Yeah, it’s great that they have coding standards and libraries; I’m just sayin’, that’s more stuff you have to learn in order to write a single line of low-level code.

High level stuff is in a combo of JavaScript and XML. Of course you have to learn the XUL platform stuff, so just knowing JavaScript and XML isn’t sufficient. You’ll also probably need to learn to use the low-level classes from the XUL side, so that you can get anything done.

Now if you want to actually build it, you’ll also need to understand their build system, and maybe their installer. You might need to learn how their help system, documentation system, internationalization sytem, etc. etc. work.

Now, I’m in awe of the monumental amount of new and working code that the Mozilla community has created. But the problem is just that: they built this huge application stack, and in order to get working on Firefox or Thunderbird, you have to learn a big chunk of that stack.

In theory, this wouldn’t be any more of a barrier than if they had coded to a single platform’s GUI widget set, class library, etc. But if I were to learn .NET and Visual Studio, or Cocoa, or GTK+, I would then have skills that I could use to work on thousands of other applications, either open source or commercial. That’s because those technologies are designed for general purpose application development.

The Mozilla application platform, however, has no life of its own outside of Mozilla. There’s an effort to extract it and make XULRunner a viable standalone platform that you could build your own XUL applications on, but that’s currently just a “stable developer preview”. What that means is that it’s still not suitable for use if you wanted to, say, build your own iTunes killer with it. It’s almost there, and maybe if you throw a lot of effort at it you could hack it so it’s usable, but as-is, it isn’t readily useful.

So, when a developer (such as myself) looks at this technology set, they ask themselves the question, “can I justify spending this much time learning all this stuff just to fix a little annoying bug?” If you spend a week or two downloading and learning and hacking and that gets you a certain amount of skill with the technology set, was it worth it? Maybe you could go get a job working with this technology set, except almost no one else is using it, not even open source projects.

When a corporation looks at this technology set, they have to ask a similar question: “is this the platform that will give me the best bang for my developer buck?” If they invest the time and money in hiring developers to work on this mostly-complete platform, so that they can then build an application with it, will it pay off? What about alternatives? Will the pool of available developers be so small that the project will fail before it starts?

These are not insurmountable barriers; some folks know some of these technologies already or only want to work on a corner of the application, so the overhead is smaller. Or, they may really want to work on Firefox or Thunderbird very badly. For an employer or open source project, their goals may match the goals of the original platform designers very well, in which case the payoff of using the Mozilla platform would be much higher than if you were (for example) making a Windows-only IM client.

But the barriers do exist, and they discourage the platform and the applications built with it from being improved. Developers don’t want to spend all that time learning things they can’t use elsewhere, and employers don’t want to pay developers to fix the platform so that other projects could find it useful. A few projects do exist and obviously somebody somewhere is working on Thunderbird bugs, but for a platform this sophisticated with this much mindshare (via Firefox), it’s remarkably unsuccessful in terms of adoption by developers.

All it would take to fix this would be some cash. Somebody could donate (or spend, in pursuit of a business goal) money toward the completion of XULRunner for general use, and toward clear and useful beginning developer documentation for the rest of the Mozilla platform.

Any takers? I could thrown in a hundred bucks or so, but I have a feeling this is more like a $50,000 undertaking. Otherwise we’re stuck waiting for Joost and a handful of others to slowly move it along.