In Defense of Cargo Cult Programming
Cargo cult programming has a bad name, and for good reason. But is it always that bad?
by Mike Nicholaides
If you've been programming for long enough you've undoubtedly heard developers malign cargo cult programming. You hear how awful it is and how only bad developers do it. But is it always that bad?
There are a few definitions of the term. While the "cargo cult" metaphor was originally coined by Richard Feynman, the most widely used definition in the software development community was put forth by Eric Lippert in an article where he complains about novice developers who know the syntax of the language but aren't fluent enough to understand the intricacies of the semantics:
There are lots of cargo cult programmers—programmers who understand what the code does, but not how it does it. Therefore, they cannot make meaningful changes to the program. They tend to proceed by making random changes, testing, and changing again until they manage to come up with something that works.
The community has taken this definition and expanded it to include any activity where a developer is making a choice without fully understanding its consequences, and only chosen because he or she saw the choice pay off for someone else.
It's easy to criticize new developers, but I posit that cargo culting is an effective tool that we all use every day and benefit from. Let's evaluate it in terms of its tradeoffs like we would any other coding practice. Could there possibly be good reasons to write code that you don't understand how it works?
Cargo culting is unavoidable
It's uncontroversial that new developers can't help but cargo cult. They simply haven't learned enough of the stack to really understand what's going on under the hood and why things must be done in a certain way.
For example, in one of my first programming courses we learned that in Java the entry point to the program is:
public static void main(String args)
It took years to get to the point where I understand every part of that method signature. Even if it was explained to me at the time, I wouldn't have understood it. I would have had to understand method visibility, class vs. instance methods, return types, parameter types, arrays (as opposed to lists/vectors), command line arguments (and how they are parsed by the shell), and various details about Java's implementation.
And that's OK. It turns out you hardly need to understand any of that stuff to write command line or even GUI programs in Java.
We don't expect new developers to know the ins and outs of every decision they make. They just haven't been programming long enough to understand every detail about every part of the stack.
Nor do we expect them to know that git stores commits in a directed acyclic graph and each branch is a pointer to one node in the graph.
It's good enough that they can do
git checkout and
git commit and get on with their work.
And it's good enough if they follow a rule like "never fast-forward merge" (or "always fast-forward merge").
But even for senior developers it's unavoidable.
What senior dev really understands all the git commands they have to look up on Stack Overflow?
sed portion of the one-liner they looked up yesterday on GitHub?
Or every header in the HTTP response they have their web application API respond with?
Or when to use the HTTPS vs. the SSH URL from your GitHub repo?
Or how to best configure your local PostgreSQL instance?
Or your Neo4j instance?
Or how to debug your rvm/rbenv/chruby/virtualenv setup without blowing it away and rebuilding from scratch?
The only way to not cargo cult is to know everything. And nobody knows everything.
Cargo culting is good
Cargo culting is good when used responsibly. It saves us from having to know everything. When used well, it is the act of leveraging the hard-won expertise of everybody else in the field. Arguably, using any abstraction at all without understanding how it works (which is the whole point of abstraction!) is cargo culting by Lippert's definition.
Furthermore, cargo culting is often a good choice. "These people seem to know what they are doing, so I'll follow their lead," is a perfectly rational strategy.
In fact, the programming community actively encourages cargo culting. When we tell people, "don't use global state," or, "follow PEP8," we are telling them to cargo cult. For example, a developer would ideally understand the (sometimes not so) subtle ways global state can bite you, but until the understanding is there, it's better to just avoid it.
Some other great uses of cargo culting:
- "Convention over configuration" as seen in Rails (and its myriad clones), Django, and Ember.js
- Style guides and their tools, like Rubocop, JSLint, and Auto PEP8
- Many static analysis tools, like CodeClimate, Reek, Flay, and Flog
- The 12factor app guidelines
- Code smells
How to cargo cult responsibly
Obviously, cargo culting has some major downsides, but now we've seen some up-sides, too. We'd like to use tools and techniques that get the job done now, without always having to fully understand why they work. Instead of avoiding cargo culting altogether, how can we cargo cult effectively?
The one big caveat: the inherent risk of cargo culting is that you won't be able to predict what will go wrong, or even the class of problems you may encounter, because you fundamentally don't understand the decisions you are making. That said, here are a few tips.
You can write automated tests to ensure your code works as expected, even if you don't understand why the code works. A simple example would be if you copy and paste a sorting algorithm from Stack Overflow, you can write a test for it to make sure it's working.
Be careful, though, because testing isn't as good at proving there aren't unexpected side effects or edge cases. In the example of the sorting algorithm, you won't know which edge cases you need to worry about.
The cleaner you keep the code, the easier it will be to reason about when a cargo culted decision needs to be changed. And remember, you can't refactor effectively without tests.
Isolate cargo culted code
When possible, isolate cargo culted code into a class or module. The more you can isolate it, the more you limit its potential damage. Also, the more isolated it is, the easier it will be to replace or rewrite once you understand more about it.
Don't cargo cult in critical parts of your app
For example, if high concurrency is an absolute necessity for your app, you probably can't get away with copying some threading code from Stack Overflow. Even choosing a language that inherently handles concurrency well (like Elixir, Go, or Clojure) won't magically solve your problem. Unfortunately (or fortunately), you'll really have to dig in and learn something.
Don't cargo cult if failure leads to bad things happening
If peoples' safety, privacy, money, etc. are at risk, don't cargo cult. As an obvious example, if someone's pace-maker runs a certain C one-liner you copied from Stack Overflow, you damn-well better know why your little trick works and what the failure cases and side effects are.
Avoid cargo culting when dealing with varying input
For example, if you don't know awk super well and the input to your awk one-liner will come from user input, it's likely to break. If you can predict the different forms your input can take, that will help, but you should keep in mind that it's likely to break on input cases you weren't able to foresee.
Monitor in production
Just because it works on your machine, don't assume it will work in production where the environment differs and users do things you never considered. Monitoring errors, exceptions, assertion failures, and performance can let you know if something is wrong before your users, clients, or boss realize it.
Code review, pair programming, static analysis tools, etc.
Get other eyes, virtual or otherwise, to look at your code. Pair programming and code review are obvious examples, but static analysis tools (e.g. CodeClimate) can be helpful, too. Besides catching obvious mistakes, they are especially helpful for pointing out code that might not always work as expected.
Move to deeper understanding
By starting with cargo culting, when you do decide to dive deep into the subject you'll already have some context. The path to deeper understanding can be self study, mentoring, pair programming, or just doing it yourself for the sake of learning.
This is old news
The tips enumerated above are likely all things you've heard before. That's because we're all working with limited knowledge. We all cargo cult all the time and we've developed intuition, techniques, and guidelines so we can move forward with partial knowledge and limit the damage we can do. Rather than never cargo cult, good developers learn to cargo cult effectively.