Eyes Above The Waves

Robert O'Callahan. Christian. Repatriate Kiwi. Hacker.

Thursday 30 June 2005

Eclipse CDT

Eclipse 3.1 is out. Eclipse is an IDE framework written in Java and includes a Java IDE that is incredibly powerful. It also has amazing CVS integration. And it's open source, cross platform, has industry support, huge plugin community etc etc. I used Eclipse a lot when I was working on Java code and it was fantastic then ... it's even better now with full Java 1.5 support and lots of new features. On my machine at work it just flies --- I suspect due to a combination of performance improvements in Eclipse, improvements in JVMs, and the power of this machine. If you're writing Java code, there is no reason to not use Eclipse or something of similar power.

Unfortunately I don't work with Java right now, so I took the C/C++ tools for a spin. These tools are called the CDT, now at version 3.0. The CDT feature set is pretty good for a C++ environment --- intelligent (parse-tree based) code completion, intelligent navigation and searching, debugger integration, on the fly builds, and basic rename refactoring. This is very impressive given the nightmarish world of C and C++ parsing and semantics. I'd love to be able to use the CDT to work on Mozilla ... I currently use emacs, which I detest.

The problem is that Mozilla is a huge and complicated codebase. There are more than 4,500 C++ files and 1,700 C files. The build process is complex, and generates a lot of code from IDL in our own special way. This creates two major problems for tools: scaling performance and making sense of the codebase. Last time I tried the CDT (2.1) it just collapsed. The good news is that CDT 3.0 is much better, perhaps partly due to improvements in Eclipse itself and my improved hardware. There's now a built-in tool that reads the 'make' output from a complete build and extracts all sorts of information useful to CDT --- include file paths, defined symbols, which source files are part of the build, and so on. Everything is also much more scalable now; I've actually been able to index most of Mozilla and get code completion and some other things working.

The bad news is that there are still showstopper problems. The make output scanner doesn't work reliably so some of my source files don't get the correct information. When I worked around that by hand, I was able to index most of the Mozilla code but code completion popups and navigation operations took 5-10 seconds to happen, which is unusable. In general while playing with the tools I kept getting into strange situations including hangs, out of memory situations, and runaway background tasks.

It's a real shame because the tools look so good and they seem so close to working --- much closer than six months ago. If a few bugs and glaring performance problems get fixed, I'll be able to throw away emacs and step up into a much more productive environment. I'm really looking forward to that.

It's important to remember that Mozilla is an extreme test, a particularly large and complicated project. No IDE has ever been usable on Mozilla (beyond basic text editing); if CDT gets there, it will be the first. For smaller and simpler projects --- i.e. most C/C++ projects --- the CDT probably works just fine.


Jeff Walden
What's the situation with Microsoft's Visual Studio? (I seem to remember Ben posting a rant asking why there was no free IDE to compete with it, so I wonder how well it meets your criteria.)
Ludovic Hirlimann
Codawarrior used to wok on the mozilla code back in the mac os 9 days.
Btw any news for your ld patches (the ones you talked about here back in february) ?
Robert O'Callahan
Ludovic: those ld patches were committed and have started appearing in distributions (FC4 has them).
Jeff: I haven't used VC++ for a long time, but Eclipse CDT is broadly comparable in functionality.
I used to use VC++ to edit my Mozilla code, and I'm sure you could use Codewarrior and other tools for that too. What I want from an IDE is deeper understanding of the code as a whole similar to what you get with modern Java tools. So for example if you have "x->foo" in your code, it knows that x is of type T* and 'foo' refers to the particular field of T, not various other things in your program which also happen to be called 'foo'. So then you can navigate immediately to where 'foo' is declared, or if you type 'f', code completion will offer you 'foo' but not 'fing' which is a field of some other type U.
I understand VC++ can do this, but again as far as I know you still can't stuff Mozilla into it. If it still works the way it used to, it wants you to define a project file referring to all the sources and that's very hard to do when you have a huge project like Mozilla with thousands of files. And maintaining that file as the code changes is hard ... I remember what troubles we had in the old days when we had to have Codewarrior project files in CVS and update them every time we changed file structure. That's why I think this new Eclipse build log scanner --- once it works properly on Mozilla --- is a winner.
Allan Knudsen
I haven't tried out CDT, but I do use eclipse and the random lockups always seem to have been a problem. The lastest builds have made good progess on eliminating these issues and there are hacks which improves it even more. If you want to play around with eclipse again, I'll recommend reading some of the blogs on eclipsepowered.org, especially http://www.eclipsepowered.org/archives/2005/05/15/31m7-memory-options/ and http://www.eclipsepowered.org/archives/2005/03/17/windows-performance-tip/.
I was beginning to think I was the only one that actually used the CDT. At work I develop software using a wide range of languages/technologies (Java, C++, Perl, XML, etc; often writing code that crosses between these languages as well -- C++/Java and C++/Perl). Because of this diverse situation alone, I can't imagine using any other IDE (I agree, the tight CVS integration is so awesome). Netbeans is supposed to be as extensible as Eclipse, but as far as I know, their C/C++ plugin hasn't been updated to work with their last 3 or 4 releases which has kept me from even downloading it.
Being the huge Firefox and Eclipse fan that I am (http://planet.mozilla.org/ and http://planeteclipse.org/planet/ are both daily reads), I strongly considered working on a small project that was based on helping other developers build Firefox in Eclipse using CDT (mainly documenting the setup but also some hacks here and there where necessary). One thing that stopped me (aside from some of the issues that still remain with CDT) is that you would have to use MinGW/Cygwin if you wanted to debug within Eclipse. This is probably not a huge deal since you could still use VC++ compiler if you didn't want to use the built-in debugger.
Overall, the CDT is really cool, I am liking 3.0 quite a bit so far. I use the plugin on a daily basis at work, however, the projects I work on are, of course, no where near the size of the Mozilla/Firefox projects (I disable the code completion/navigation even in my small projects because it is so slow; I have a feeling there is a lot of room for improvement here). One thing that is kind of bugging me about the latest version of Eclipse and CDT is that there appear to be some issues with capturing the console output (https://bugs.eclipse.org/bugs/show_bug.cgi?id=102043). I'm sure it will be fixed along with several other somewhat annoying bugs (mainly related to debugger and filenames).
By the way, I follow the Firefox development quite a bit and I like the work that you do, keep it up. :)
Looking forward to 1.1, thanks for the cool product.
Robert O'Callahan
Joel, I think many or most Mozilla developers actually use Linux so the Cygwin thing is not an issue.
I just fixed one CDT performance bug and submitted a patch :-).
You're right about the cygwin thing, most developers (even on Windows) could use gcc and everything would be neatly integrated into Eclipse (idealy). However, since VC++ is the tier1 development platform for Windows, I was hoping to find a catch-all solution (this was probably an unrealistic goal). Either way, the VC++ thing is not a huge deal. I think I'll give my little project some more thought as the CDT matures some more.
By the way, which bug did you submit your patch to?
I've noticed the Eclipse people tend to not be very responsive on Bugzilla. I guess I'm spoiled by the Mozilla folks. :)
Robert O'Callahan
Doug Schaefer
One reason you may see improvements using CDT on Mozilla is this is one of the main test beds we like to use when working on scalability. Hopefully we can get some of your bugs fixed in time for the final CDT due out at the end of July -ish.
Our thinking is if we can work with Mozilla, we can work with most large projects. Thanks for your support and help! Keep the patches coming :)
John Camelon
FYI We do our best to be responsive to bugzilla entries.