Tuesday, 18 July 2017

Goodbye to Eclipse - the end of an era as an Eclipse plugin developer.

I've been an Eclipse plug-in developer for over 7 years now, for nearly all of my software engineer career. Notably, I have been the lead developer of 3 open-source Eclipse IDEs: RustDT, Goclipse, and DDT. But with some pain, I've decided to change my career tech focus, and stop working with Eclipse IDE development altogether.

For several years now, Eclipse has been declining in popularity, and in my opinion also declining in comparative quality to other IDEs. I've seen the issues Eclipse has, both external (functionality, UI design, bugs), and internal (code debt, shortage of manpower). I was holding on to the hope it would recover, but now I don't see much of a future for Eclipse other than a legacy existence, and I realize exiting was long overdue.

I'm writing this post for two main reasons: I want to document my career path and explain the reasons for my tech change. And I want to give some clarity and thoughts to the users of my Eclipse IDEs, about the future of those projects, and IDEs in general.

A bit of background

I have roughly 6 years of cumulative FOSS work on Eclipse IDE plug-in development. I started in university with my MSc. thesis, but the bulk of this work occurred from 2011 to 2016[1]. First with DDT, later Goclipse[2], and then RustDT.

It so happened in those early times that I wanted to contribute to the IDE tooling of these new languages, because I was a massive fan of JDT (the Eclipse Java IDE) and all the cool things it could do - code completion, refactoring, etc. - and I wanted to help bring some of that functionality to these newer programming languages.

Also, I was hoping that if any of these languages took off in popularity, I would be in a position of importance of sorts, as the lead developer of a popular IDE for such a language. This was a big, unlikely gamble, I knew that from the start. And indeed it didn't materialize, although for different reasons in two different, distinct situations:

The first case was with DDT and the D language. It failed to get the popularity I was hoping for, and by 2013 or so I realized it was very unlikely D would ever gain significant adoption.[3]

I almost stopped working on IDE development then, but, at the same time a new player had emerged on the scene: Go. Go seemed to have a lot more potential to take off, so I then began to generify my DDT code and adapt it to an existing Eclipse Go IDE - Goclipse. A year later, Rust followed suit with RustDT[4]. These languages were gaining traction, and by 2016 I would say Go had clearly achieved mainstream popularity[5]. Unfortunately, by then, the Eclipse IDE had itself declined in popularity so much that having written Go's Eclipse IDE meant little in terms of influence or importance. Most users were using the Go plug-ins for IntelliJ or Visual Studio Code[6].

Moving on

Eventually I ran out of personal funds and had to go back to a normal, full-time, paid job. So in early 2017 I joined Renesas Electronics as an Eclipse plug-in developer, working on extensions to Eclipse and CDT for their embedded/hardware development tools. By then I thought it was unlikely I would do any significant work on those OSS projects again, but career-wise I still saw myself as a Eclipse plug-in developer.

As those initial months of employment passed, I started to question this. First I noticed there was a scarcity of Eclipse RCP/plug-in development jobs available in London and the UK generally. This tech was always a niche, but it looked like there were now even fewer roles in the market than 4 or 5 years ago.

In addition, Ericsson began pulling out their developers from CDT, and more significantly, Google too announced they were pulling out their developers from the core Eclipse team:
"Eclipse usage within Google has dropped to the point where it no longer makes sense to have a team of engineers contributing to the Eclipse project. This means that in the next few months my team will be dismantled and I will be reassigned to something else."
Google abandoning Eclipse was the wake-up call that prompted me to re-evaluate my career path, and it didn't take long for me to decide I too should move on entirely. I didn't know if wanted to retrain into Android development, into Java back-end work, or even JavaFX/Swing UI development, but I knew didn't want to work with Eclipse any more.

Hindsight

I had already been on the fence about the future of Eclipse for some years, but I guess I still hoped it would recover somehow. That wasn't happening:

https://zeroturnaround.com/rebellabs/java-tools-and-technologies-landscape-2016-trends/


They say of course, hindsight is 20/20, and one shouldn't beat oneself for mistakes one did not see coming. But I still think that at the very least in 2013 I had enough information to realize I should have seriously considered quitting. In particular, there was a big, solid sign there was trouble ahead for Eclipse. It was this:

Google announces Android Studio: An IDE built for Android (based on IntelliJ)

This was significant because, for those not familiar with the history of Android tooling, by this point Google already had a full-blown, mature IDE for Android based on Eclipse (Eclipse ADT). So what they chose to do was deprecate it and build a new one based on IntelliJ instead. For them to throw away so much existing code and effort, it must have meant IntelliJ was considered by them to be significantly superior, not just marginally so.

I mean, if Google had no previous Android IDE, and were just choosing whether to start one with Eclipse or IntelliJ, and had chosen IntelliJ - it could have been said to be a choice down to just a small margin of preference. But that wasn't the case. They were abandoning an established code base, in favour of writing a new one based on a different IDE platform. This meant the rewrite was preferable to fixing the various issues Eclipse had with platform code, API, UI, and so on... and if Google wasn't going to do it, even though they had so much invested already, who would? Nobody.

This is the point I regret not having left Eclipse earlier, saving myself three years of my professional life. And a lot of personal funds.

Where to, then?

Well, for me, since I want to work with Java and potentially other JVM languages (Scala looks nice!), the IDE choice is clear: IntelliJ IDEA.

As for Rust and Go, IntelliJ is still a solid choice, but no longer necessarily the clear winner. Visual Studio Code is quite nice and is becoming more and more popular. And particularly the Language Server Protocol is very innovative and also getting a lot of initial traction and interest.

For Scala the situation seems similar: a choice between IntelliJ and Visual Studio Code. Sure, there is also the Eclipse-based Scala IDE, which has been professionally developed and very actively maintained, but those guys too are jumping ship:
"Over the years, Eclipse turned out to be no longer the IDE we love because it doesn't help us to make us and our users happy. We therefore looked for alternative editors that could act as a new graphical frontend for Scala IDE and VSC turned out to be the most promising editor that could handle our needs."

The future

All things considered, it looks like the LSP is the future of IDEs. VSCode will likely be at the forefront of this change, but I imagine other IDEs and editors will easily pick that up and not be far behind. VSCode is also developing something similar to LSP, but for debugging: the Debug Protocol. Looks very promising.

I will give a shout-out to the work being done on Eclipse IDE support for LSP, which is nice. But ultimately I don't think it will be enough to keep Eclipse IDE relevant to a modern audience. Even if Eclipse gets widespread support for new languages through LSP, there is still a lot of fundamental problems with the Eclipse UI that at this stage are too hard to change satisfactorily - they would require a complete redesign of the UI[7].

As for IntelliJ, it will be interesting to see how LSP will affect it, since IntelliJ typically relies on its own internal IDE engine and language framework to provide language support. Sure, IntelliJ can also implement language support using LSP, but can it do so whilst providing the same level of quality and robustness that it does using its own platform? I guess we will have to wait and see. I wouldn't be surprised if JetBrains, the company behind IntelliJ eventually became partners in shaping the evolution of LSP.



[1] - As a side note, I want to point out that this FOSS development was actually full-time unpaid work. How I managed to fund myself in London (a very expensive city) is another story, but not for this post.

[2] - Unlike my other IDEs, Goclipse was not started by me, but a different developer. Rather, I took over an existing codebase once the previous developers retired from contributing (kudos to their work, BTW). However, because Goclipse was gradually rewritten to use my generic IDE code (MelnormeEclipse), after two years Goclipse's code was almost entirely rewritten/replaced.

[3] - This is a very broad subject to comment on, but in summary, in my opinion D struggled to get enough manpower to develop (to a sufficient degree for an emerging language), the compiler, core libraries, and other tools. That the main D compiler, DMD uses its own compiler backend, instead of LLVM or GCC, illustrates the general failure to integrate with the larger FOSS community and re-use more of other people's code. This approach hampered D's growth, and then when Rust emerged, that sealed D's fate - not only has Rust got vastly more resources behind it, but the language itself was built from the ground up on a superior paradigm: no garbage collection but instead ownership semantics, something that D is now playing catch up to. Good luck with that...

[4] - By now it was doable for a single developer to write a IDE for one of these newer languages because there were people already working on external tools doing code completion, find definition, etc.. In other words an IDE developer only needed to implement the IDE UI - the semantic engine itself was being developed by other people. With Goclipse and RustDT, as the UI elements were gradually made generic from DDT since they were conceptually language-agnostic, it meant it was possible for a single developer to create a sizeable amount of IDE functionality on its own. In the past, before Github and all that, this would have been very hard to achieve.

[5] - What constitutes "mainstream popularity" is imprecise and subjective, but I'm basing this threshold on seeing a small but steady flow of job ads for Go developers in London. I would say a language can roughly be considered mainstream when you can find in any major tech city a job that primarily uses that language.

[6] - Or alternatively, the various editor environments - Vim, Emacs, Sublime, etc.. But unlike IntelliJ, I would not consider those editors as actual competitors of an Eclipse IDE, because most of their users have a different editor mindset and would not be interested in using a heavyweight IDE, regardless if whether it was quality or not.

[7] - Not an extensive list, but: Package Explorer vs. Project Explorer dichotomy; Project Explorer issues with nested projects and duplicate resources; No implicit auto-save like IntelliJ; UI based on per-language perspectives instead of a more generic model; Functionality mismatch between external files and workspace files; 
And then, all the thousand paper cuts...

5 comments:

  1. "No implicit auto-save like IntelliJ"
    you missed the new feature of Eclipse Neon about auto-save: https://www.eclipse.org/eclipse/news/4.6/platform.php#autosave-dirty-editors

    ReplyDelete
    Replies
    1. Apologies for late reply. I had forgotten to enable notifications for blog comments (seems it was not turned on by default!)

      I know well the auto-save feature from Neon - I even attempted an alternative implementation at some point when auto-save was being proposed. But that feature is far from the same as what you have in IntelliJ.
      In Eclipse even with auto-save enabled, you still get asked if you want to save a file if you close an editor (and there's no other preference to enable that). And that makes a big difference! I think unsaved editors also interact badly with refactoring operations in certain situations, but can't remember the details. Plus, you still get the dirty editor mark in the editor tab which is distracting.
      In IntelliJ none of that happens, you never get asked to save, it just happens implicitly all the time (and by default).

      Delete
  2. Bummer. RustDT is my IDE of choice for Rust. Thanks for your work on it.

    How long were you working full-time, unpaid, on Eclipse plugins??

    ReplyDelete
    Replies
    1. Apologies for late reply. I had forgotten to enable notifications for blog comments (seems it was not turned on by default!)

      The amount of work that was unpaid, hum, that must have been cumulatively, equivalent to about 3 years of full-time work... and yes, that is quite a lot! ~_~
      I was fortunate in a way, that I had amassed enough funds to do that comfortably. Unfortunate in how I choose to spend that time. That mistake is on me though, I don't blame anyone else.

      Delete
  3. How can you work on Android Studio(Idea) without incremental build which is showing the all problems right on the project tree immediately on save and easy to catch the problems on the fly, can run your project almost immediately and do not wait for the build processes? If it fail you have to find the bug, fix it and build again, get another bugs and again and again... That's just like back to 90th so I feel miss something about Idea or Visual Studio. Is there any way to turn on the Eclipse incremental build for this IDEs of future?

    ReplyDelete