Faced with complicated choices and shrinking timelines, many leaders rely on intuition. But how reliable is your gut instinct in reality?

This week, my message focuses on deciphering the true value of intuition. How far should you trust your instincts? And under what conditions can gut feelings be translated into sound business decisions?

The human brain is capable of incredible feats. It can swiftly process and assess information at a subconscious level, delivering conclusions that we call “intuition”.

In The Future of Decision Making: Less Intuition, More Evidence, Andrew McAfee shares a telling example:

If I ask a group of people “Is the average price of German cars more or less than $100,000?” and then ask them to estimate the average price of German cars, they’ll “anchor” around BMWs and other high-end makes when estimating. If I ask a parallel group the same two questions but say “more or less than $30,000” instead, they’ll anchor around VWs and give a much lower estimate. How much lower? About $35,000 on average, or half the difference in the two anchor prices.

Our reliance on intuition is a legacy from our cave-dwelling days. When confronting dangerous situations on a daily basis, our ancestors relied on gut instinct to survive – careful analysis wasn’t an option. Today, the decisions we make in the corporate world aren’t life or death, and they don’t need to be made in a split second. Still, we continue to depend heavily on intuition to show us the way. Why?

In the Harvard Business Review article, Don’t Trust Your Gut, Eric Bonabeau breaks down the allure of intuition:

For one thing, it’s romantic. It raises business above the drab world of spreadsheets and income statements and turns it into something of an art form… For another, it simplifies. It says that we needn’t worry if we can’t decipher complex challenges rationally-our subconscious mind will automatically deliver the right answer… Finally, it makes us feel special. Any idiot can run the numbers, but the gift of a good gut-that’s reserved for the true business elite.

However, as Bonabeau goes on to explain, our perspective is quite selective. We forget the gut-based decisions that failed, choosing to focus only on legendary stories of success. The truth is that hunches often don’t work – but the facts get swept away in our overwhelming belief.

This is not to say that gut instinct isn’t a valuable guide. As Laura Huang explains in When It’s OK to Trust Your Gut on a Big Decision, intuition is the product of a sophisticated mental process:

Recognize that gut feel is not a separate piece of information but it draws on both objective and subjective information that is already available. Understand that gut feel is not quick, impulsive, and emotional – it’s actually something much more cultivated and nuanced and based on experience.

Intuition can be astonishingly accurate in the right circumstances. Think of chess players who seem to “see” their opponent’s future moves or investors who simply “know” which startups will succeed. Intuition deserves to be paid attention to, especially when it’s based on years of expertise and is functioning in an environment with cues about what might happen next and feedback on what’s working. Plus, when you don’t have much time for analysis and need to act very quickly, intuition can offer a handy shortcut.

Despite all the benefits, however, gut feel isn’t enough on its own. It’s one of several important data points, and should be treated as such. (Take this quick Harvard Business Review quiz to see if you’re prone to getting tricked by intuition.) Bonabeau highlights the danger of relying overtly on hunches:

Intuition has its place in decision making-you should not ignore your instincts any more than you should ignore your conscience-but anyone who thinks that intuition is a substitute for reason is indulging in a risky delusion. Detached from rigorous analysis, intuition is a fickle and undependable guide-it is as likely to lead to disaster as to success.

So, the next time you’re tempted to make a big decision based largely on gut feeling, consider the following 8 recommendations to increase your likelihood of success:

1. Factor in experience

In Intuition Isn’t Just about Trusting your Gut, Modesto A. Maidique describes his interviews with several CEOs on how they had made the most important, fruitful decisions of their careers – nearly all the responses came down to “gut instinct”. Follow-up questions led to an interesting discovery: the success rate was far higher in domains where the CEOs had deep knowledge. In other words, the quality of your intuition has a strong link with the quality of your expertise in the given area. According to research, it takes about 10 years of domain-specific experience to develop accurate gut-based judgments. Which means that trusting your instincts in new, unfamiliar territory is not the smartest option.

2. Use intuition where data doesn’t suffice

If the problem you’re tackling is well-structured, has clear decision rules and criteria, and plenty of available data, take an analytical approach. As Huang advises in the Harvard Business Review article mentioned above, you should rely on intuition only when data can’t plug gaps of uncertainty or when the data itself is ambiguous:

What kind of decision are you being faced with? What is the level of “unknowability”? Reserve your intuition for those decisions that go beyond routine, where calculations of probabilities and risks are not only unrealistic, they are infeasible.

At the same time, there needs to be a certain amount of predictability for your intuition to have value. As psychologist Gary Klein explains in an interview with McKinsey Quarterly:

If a situation is very, very turbulent, we say it has low validity, and there’s no basis for intuition. For example, you shouldn’t trust the judgments of stock brokers picking individual stocks.

3. Know yourself

One way to guard against the inherent cognitive biases and flaws that present themselves as “gut feeling” is self-knowledge. When you become aware of the destructive tendencies within yourself, such as confirmation bias or the desire for show someone up, you can supplement intuition with measurable evidence to make better decisions.

4. Pay attention to the numbers

As McAfee reminds us in the article mentioned above, we should make the maximum use of available statistical techniques, which can locate patterns in vast amounts of data and identify causal relationships. These techniques aren’t 100% perfect, but they’re still extremely valuable – especially when compared with intuition alone:

A 2000 paper surveyed 136 studies in which human judgment was compared to algorithmic prediction. Sixty-five of the studies found no real difference between the two, and 63 found that the equation performed significantly better than the person. Only eight of the studies found that people were significantly better predictors of the task at hand. If you’re keeping score, that’s just under a 6% win rate for the people and their intuition, and a 46% rate of clear losses.

5. Explore technology

A number of innovative decision support tools are now entering the business domain. These sophisticated programmes have the potential to rigorously analyse a mind-boggling number of choices in a complex, rapidly changing environment. The insights thus generated can support your decision-making process.

6. Go back to basics

Don’t dismiss conventional decision-science techniques, such as decision trees, portfolio management and system dynamics. These offer an excellent way to analyse multiple options. They do have their limits, however, as Bonabeau points out:

In highly complex situations-when there are many dependencies among possible solutions or no clear way of measuring the solutions’ value—traditional tools become unwieldy and tend to provide unreliable answers.

7. Watch out for overconfidence

Leaders need to be particularly careful of the tendency to become overconfident about their gut instinct. As highlighted by Nobel laureate Daniel Kahneman in an interview with McKinsey Quarterly:

Overconfidence is a powerful source of illusions, primarily determined by the quality and coherence of the story that you can construct, not by its validity. If people can construct a simple and coherent story, they will feel confident regardless of how well grounded it is in reality…. That’s one of the real dangers of leader selection in many organizations: leaders are selected for overconfidence. We associate leadership with decisiveness. That perception of leadership pushes people to make decisions fairly quickly, lest they be seen as dithering and indecisive.

8. Get feedback

When you make an important decision, record the process by which you arrived at it and seek out reliable feedback on it afterwards. By objectively assessing the outcome of your decision, you strengthen your expertise, which in turn sharpens your future intuition.

In When It’s Safe to Rely on Intuition (and When It’s Not), Connson Chou Locke sums up the role of intuition in decision-making wonderfully:

Ultimately, it may be that we should use both intuition and analysis. There may be times when intuition helps narrow down the options, which can then be analysed in a logical and rational way. Or the reverse: an initial detailed analysis may identify a few options that seem equally good, and intuition is needed to single out the right one. But before you decide to trust your gut, ask yourself: Am I an expert? Is this an unstructured problem? And how much time do I have to choose?

How far do you listen to your intuition? Have you found other ways to strengthen your decision-making process?