On the UC-Santa Cruz campus where I work, there are a number of walking paths between buildings that cross through forested and hilly terrain. One of them leads down to a bridge over a ravine, and winds around and down the slope of a hill to reach it. The paved path has a sharp bend far away from the bridge, and at the point where it bends, students have worn a second path that leads straight towards the bridge. However, because the bridge is not quite where it appears to be from that angle, thanks to the slope of the hill, this impromptu path must take a sharp jog further down the hill, closer to the bridge.
Students have worn the second path leading down to the bridge because they perceive that it’s faster to head straight ahead instead of staying on the paved path, because the paved path appears to detour away from the bridge. However, the paved path, because it takes an earlier detour, approaches the bridge in a straight line, while the “shortcut” path must alter course. Therefore, when coming down the hill towards the bridge, the shortcut path appears to be shorter; however, when approaching the hill from the bridge, the paved path appears to be shorter, because it progresses in a straight line while the second path takes a detour. And thus – and I have observed this in person – when descending the hill towards the bridge, students prefer the shortcut, whereas when ascending the hill from the bridge, students prefer the paved path. Why?
In fact this is a simple real-world case of graph traversal, or the process of selecting the most advantageous path from A to B from a network of possible routes. These flip-flopping students are demonstrating that when solving a graph traversal problem, the human brain likes to use the “greedy algorithm”; that is, the brain selects the path that appears the most immediately advantageous, even though there may be no functional difference overall – the two paths are approximately the same length. So students heading in either direction, faced with two options, unconsciously choose the one that appears to be headed straight for their destination, even if they know there is no ultimate difference.
The time it takes you to get from one building to another on campus is not a pressing issue, except when optimizing exactly how late you can sleep in before that morning class, and no undergrad has ever halted at the fork between these two means of reaching the bridge and simply stayed there, frozen, for the rest of time. But what if you were confronted with a thousand means of getting to the bridge, all of which appeared to head straight for it in the same amount of time? How would you choose which route to take? You might stare down several, proceed down one or two to make sure there were no obstacles or advantages not immediately visible. No big deal, you say to yourself. I’d just pick one at random. But what we observe in practice is that a surprising number of people – and it happens to everyone – will find themselves staring at the mouths of those thousand paths, unable to make a decision. They are trapped in analysis paralysis, and it’s the subject of today’s essay.
Analysis paralysis refers to the human brain’s natural tendency to seize up when confronted with a large number of options that each offer the same cost/reward ratios. The brain has, in effect, no natural tie-breaker. It can happen in situations as simple as trying to buy laundry detergent at the supermarket (so many kinds!) or as complex as trying to publish a key piece of research. It is often called “over-thinking a problem” or being “spoiled for choice,” although there is no real useful thinking going on – your brain is just stuck in an infinite loop, waffling. Any situation that presents a large number of choices that have similar risks and similar costs can cause analysis paralysis, and the process of research is one that often has many options for moving forward at a given time. It strikes most frequently, however, at two particular points: at the beginning and at the end of a research project.
At the outset of a project, you have a desired result, represented by some destination node in the graph, and many possible options for obtaining that result, represented by the network that must be traversed to reach that destination node. Especially at the beginning, that network is often nebulous and filled with ways to proceed. Projects that involve a large amount of raw data are the most vulnerable to analysis paralysis, as the researcher is presented with dozens of different ways to begin analyzing the data, all of which cost roughly the same and offer, as far as you know, about the same reward. On the other end of a research project, analysis paralysis strikes when deciding when to call a piece of work finished. This freeze sometimes goes by the name of “Zeno’s Paper,” that twist on the well-known Greek paradox that states that first you write half of the paper, then you write half of what’s left, then half of that, and so on. You are spoiled for choice as to where to end your paper, and without a clear place to stop or an external pressure such as an obvious threat of being scooped, your brain can’t pick where to put it down and call it done.
Analysis paralysis is a serious threat, and one that can ensnare anyone. Any situation that presents a large number of options with no clearly superior choice can cause it, and it can lead to getting scooped, not publishing anything, or, in the worst case, inability to complete a thesis. Having outlined the threat and the cause, how do we escape it?
The first and most important step to breaking free from analysis paralysis is to recognize when you are trapped in it. From inside the paralysis, it is easy to convince yourself that you are just taking your time to construct an ideal plan of attack when in reality you are spinning your wheels hoping to suddenly discover a reason to prefer one of the options set before you. It may feel completely natural to continue on in the same manner. Don’t fall for it! Analysis paralysis is so insidious because it is what your brain wants to do. It can’t break a tie, so it wobbles around until either the options change or it is forced to make a decision by external pressure. Recognize when you’ve been staring at the problem instead of trying to solve it. Ask yourself “what’s my plan of attack?” or alternately “where is the finishing point for this project?” If you can’t answer these questions, you may be in trouble. An aimless frustration, by the way, is one of the sure signs of analysis paralysis. Breathe. Relax. Then make a plan.
Second step: narrow down your options. You must prune down the tree of possibilities to a manageable size. Don’t try and figure out a metric by which to make the best choice; the problem you are struggling with is that there is no best choice. You have to break the tie. If you are unable to decide how to begin a project, defaulting to a known or standard approach can get you out of this rut; when you can’t figure out a plan of attack in the first couple of weeks, go to your default approach and don’t obsess over whether it is the correct one. You can always change course later if it’s clear that it won’t work; the important thing is not to sit forever wasting time worrying that you might waste time.
Handling this step on the other end of a project takes a different kind of approach. Here you must cut down the list of things to do. But how can I leave out Figure 63b!? you protest. All I need to do is perform one more statistical test to nail down my results for good. And then of course do another type of analysis, and test it against the data from an obscure paper from 1973, and… Stop right there. Make a list of everything that needs to be done to complete your project. Cross off about a third of it. Set that list in stone. That’s what needs to be done, and that’s what you’re going to do. The truth is you can always do more research, which is good, because otherwise we’d be out of a job. Don’t obsess over the idea that you have left something critical out of your paper. If you forgot anything, your advisor will mention it, and if you both forgot something, the referee will be happy to remind you.
Third and last: just do it. Research is about thought, so it is perilously easily to begin over-thinking; but at some point you must stop thinking and do. Commit to a plan of attack and follow through, and don’t get frustrated if it doesn’t work. Get back up and try again. While it may sting to imagine coming to your advisor at the end of the week and saying, “The method I chose isn’t panning out, and I need to change it,” contrast it to the idea of explaining how you whiled away the entire week reading papers and doodling around on Facebook because you couldn’t get started on real work. When writing up results for publication, recognize that papers, like teenagers, must leave the nest some time; and papers, like teenagers, will get more surly and uncooperative the longer you delay that moment. Kick them out. Note that this doesn’t mean a blind charge straight ahead. There is a happy medium between spending all your time appreciating the problem and spending it knocking your head against a brick wall when a step back would let you know that there’s a perfectly serviceable door. No one can tell you where to strike the balance between the two; that, unfortunately, is something you have to figure out on your own.
Analysis paralysis is an easy trap to fall into in grad school, and a hard one to claw your way out of. Grad students often have no pressure to make a choice right now, and hence have little perceived cost in delaying. The five years of a PhD program can seem like forever, so what’s putting off a decision another couple of days? But while putting a little time into planning will save you effort down the road, it’s important to recognize the difference between planning and doing. Recognize if you are falling into analysis paralysis. Cut down the tree of options to a workable plan. And carry it through. These three steps can save you a lot of pain in your (hopefully) years of research to come.
So, readers, now that I’ve explained what analysis paralysis is, send me your thoughts: what strategies do you use to overcome it? Have you ever been trapped in it? How did you manage to get out?
Oh, and before I forget: the ultimate fate of the two paths I mentioned at the beginning of this article? Annoyed by students tromping through their formerly pristine forest, UCSC blocked off the second “shortcut” path with a sign stating “Forest Renewal in Progress,” and planted several large rocks to bar the entrance for good measure. Now students must take the paved path. Problem solved.