polled Travis CI stats for languages on GitHub, Julia was basically tied for last in uptime. This isn't just a statistical curiosity: the first time I tried to fix something, the build was non-deterministically broken for the better part of a week because someone checked bad code directly into master without review. I spent maybe a week fixing a few things and then took a break. The next time I came back to fix something, tests were failing for a day because of another bad check-in and I gave up on the idea of fixing bugs. That tests fail so often is even worse than it sounds when you take into account the poor test coverage. And even when the build is "working", it uses recursive makefiles, and often fails with a message telling you that you need to run
make clean and build again, which takes half an hour. When you do so, it often fails with a message telling you that you need to
make clean all and build again, with takes an hour. And then there's some chance that will fail and you'll have to manually clean out
deps and build again, which takes even longer. And that's the good case! The bad case is when the build fails non-deterministically. These are well-known problems that occur when using recursive make, described in Recursive Make Considered Harmful circa 1997.
And that's not even the biggest barrier to contributing to core Julia. The biggest barrier is that the vast majority of the core code is written with no markers of intent (comments, meaningful variable names, asserts, meaningful function names, explanations of short variable or function names, design docs, etc.). There's a tax on debugging and fixing bugs deep in core Julia because of all this. I happen to know one of the Julia core contributors (presently listed as the #2 contributor by GitHub's ranking), and when I asked him about some of the more obtuse functions I was digging around in, he couldn't figure it out either. His suggestion was to ask the mailing list, but for the really obscure code in the core codebase, there's perhaps one to three people who actually understand the code, and if they're too busy to respond, you're out of luck.
I don't mind spending my spare time working for free to fix other people's bugs. In fact, I do quite a bit of that and it turns out I often enjoy it. But I'm too old and crotchety to spend my leisure time deciphering code that even the core developers can't figure out because it's too obscure.
None of this is to say that Julia is bad, but the concerns of the core team are pretty different from my concerns. This is the point in a complain-y blog post where you're supposed to suggest an alternative or make a call to action, but I don't know that either makes sense here. The purely technical problems, like slow load times or the package manager, are being fixed or will be fixed, so there's not much to say there. As for process problems, like not writing tests, not writing internal documentation, and checking unreviewed and sometimes breaking changes directly into master, well, that's “easy”4 to fix by adding a code review process that forces people to write tests and documentation for code, but that's not free.
A small team of highly talented developers who can basically hold all of the code in their collective heads can make great progress while eschewing anything that isn't just straight coding at the cost of making it more difficult for other people to contribute. Is that worth it? It's hard to say. If you have to slow down Jeff, Keno, and the other super productive core contributors and all you get out of it is a couple of bums like me, that's probably not worth it. If you get a thousand people like me, that's probably worth it. The reality is in the ambiguous region in the middle, where it might or might not be worth it. The calculation is complicated by the fact that most of the benefit comes in the long run, whereas the costs are disproportionately paid in the short run. I once had an engineering professor who claimed that the answer to every engineering question is "it depends". What should Julia do? It depends.
Update: this post was edited a bit to remove a sentence about how friendly the Julia community is since that no longer seemed appropriate in light of recent private and semi-private communications from one of the co-creators of Julia. They were, by far, the nastiest and most dishonest responses I've ever gotten to any blog post. Some of those responses were on a private discussion channel; multiple people later talked to me about how shocked they were at the sheer meanness and dishonesty of the responses. Oh, and there's also the public mailing list. The responses there weren't in the same league, but even so, I didn't stick around long since I unsubscribed when one the Julia co-creators responded with something bad enough that it prompted someone else to to suggest sticking to the facts and avoiding attacks. That wasn't the first attack, or even the first one to prompt someone to respond and ask that people stay on topic; it just happened to be the one that made me think that we weren't going to have a productive discussion. I extended an olive branch before leaving, but who knows what happened there?
Update 2, 1 year later: The same person who previously attacked me in private is now posting heavily edited and misleading excerpts in an attempt to discredit this post. I'm not going to post the full content in part because it's extremely long, but mostly because it's a gross violation of that community's norms to post internal content publicly. If you know anyone in the RC community who was there for the discussion before the edits and you want the truth, ask your RC buddy for their take. If you don't know any RC folks, consider that my debate partner's behavior was so egregious that multiple people asked him to stop, and many more people messaged me privately to talk about how inappropriate his behavior was. If you compare that to what's been publicly dredged up, you can get an idea of both how representative the public excerpts are and of how honest the other person is being._
Aside from that, they also claim that the issues in this post have been addressed, and that (for example) test coverage is now good. But if you look at any thread about this, people find that they still run into a lot of bugs. I hear about this all the time because I know one of the co-authors of the O'Reilly Learning Julia book and they have to re-write examples to work around core bugs all the time. And that's for basic examples for an intro book, stuff you'd expect to work because it's so simple. This isn't to say that Julia is a bad language, but there's a trade-off between moving fast and breaking things. If you're ok with the trade-off that Julia makes, that's great! But you shouldn't believe the claims that there's no trade-off and that everything is great.
In retrospect, the initial response to my post from the Julia community is pretty amusing. Here's a response from a core member of the Julia community that's representative of the response at the time:
Other folks thought it was great that the community responded to criticism so well. And then one of the co-creators of Julia goes around for a year telling people that issues that haven't been fixed have been fixed while also dredging up an extremely misleading and heavily edited set of quotes from private communications to discredit me. If that's what it looks like when the community responds well, I'm afraid to even ask about what it looks like when it doesn't respond well.
Thanks (or anti-thanks) to Leah Hanson for pestering me to write this for the past few months. It's not the kind of thing I'd normally write, but the concerns here got repeatedly brushed off when I brought them up in private. For example, when I brought up testing, I was told that Julia is better tested than most projects. While that's true in some technical sense (the median project on GitHub probably has zero tests, so any non-zero number of tests is above average), I didn't find that to be a meaningful rebuttal (as opposed to a reply that Julia is still expected to be mostly untested because it's in an alpha state). After getting a similar response on a wide array of topics I stopped using Julia. Normally that would be that, but Leah really wanted these concerns to stop getting ignored, so I wrote this up.
ALso, thanks to Leah Hanson, Julia Evans, Joe Wilder, Eddie V, David Andrzejewski, and Yuri Vishnevsky for comments/corrections/discussion.