Tag: Judgements

Hindsight and foresight in judgments under uncertainty

As I began reflecting on my earlier posts on You are intelligent: have you done something dumb? and Judgments, I rolled back to this classic article, so beautifully titled, Hindsight ≠ foresight. Written by B Fischhoff, way back in 1975, this paper provides two intuitive results (at least in hindsight): once the outcome of an event is known, people associate higher probability of its occurrence; and people were unaware that they have been influenced by hindsight (the knowledge of what actually happened).

Case class in a business school – hampered by hindsight bias

Take a business school case class for instance. As a case teacher, I face this a quite a lot of times. In a typical strategy cases, the primary question to the class is “what should the company do?” And since most cases are set a few years in the past, simple Internet search (by the students as part of their class preparation) would have informed them about what had actually happened. Given that students come into class with this hindsight, they try very hard to fit their preparation and theoretical arguments to the actual outcome, however irrational, or improbable it might have been. A good management teacher ought to therefore provide for this “hindsight bias” in  students and ensure that a fair discussion happens in class on all possible outcomes.

Should I therefore, as a management teacher, provide my students with only cases for which the outcomes were not known? What therefore are my criteria to choose cases for a class? Do I fight to eliminate this “hindsight bias”? Let me come back to this later.

An air-crash investigation

Let us take another example. A special team has been tasked with investigating the cause of an air crash. Any investigation of an accident would inevitably entail putting together pieces of information to arrive at a causal relationship between the antecedent factors and the event, which is known to have happened. It is impossible to eliminate the source of bias here, the event. The investigation team has to be trained to create a counter-factual (good) outcome from the evidence at hand. They need to recreate the antecedents to the event in a manner that they evaluate if anyone in their place would have made the same decisions as the actors (pilots and crew who made certain decisions) did. Experimentally, it is akin to creating a control-group that knows all the facts leading up to the case, but not the actual outcome.

Investigating white-collar crime

In the case of white-collar crime, especially when it involves financial fraud, another significant factor interferes with hindsight bias, the size or impact. Larger frauds are fraught with more pronounced biases. Media coverage on “select” white-collar crimes are testimony to such biases. Nicholas Bourtin (read the article here), adds how armed with hindsight bias, financial crime investigators might ascribe malicious intent to even innocent mistakes or poor judgement.

As I was thinking about this issue, I just saw the breaking news of an earthquake of 7.2 magnitude hitting the Iraq-Iran border (Sunday, 12th November 2017). Hopefully, there isn’t much damage. And it triggered a thought.

Fighting hindsight bias – learning from geophysics

A great learning for fighting hindsight bias comes from geophysical studies. Imagine how geologists and geophysicists study earthquakes and volcanic eruptions. These are events that just “happen”, and then the scientists “reconstruct” the events through carefully collected data. Can we learn something from the way they fight hindsight bias? Sure.

Strategy #1: Conduct stability studies. Not just fault studies, but stability studies. Take the context of highly quake-prone areas, and go study why earthquakes aren’t happening! Such data would provide the ‘normal’ distribution of data with the occurance of earthquakes being the outliers.

Strategy #2: Broaden the search. Take all retrospective data for analysis. Study all the quakes that happened on a plate/ all eruptions of a volcano. Such events may occur very infrequently, and may be randomly distributed over time. However infrequent they may be, it would be worthwhile to study the antecedent conditions every time. Maybe, one can find a cause-effect relationship. Like concluding that most road accidents happen between 2.00am and 4.30am because there is a high likelihood of drivers sleeping behind the wheel (that is if they are still being driven by human drivers!).

Strategy #3: Combine the two, and seek patterns. Conduct stability studies and say why events do not happen, and conduct (with big data) longitudinal studies to infer why events do happen. Combine the two and create patterns. Such patterns can be immensely helpful in studying antecedents of events, and effectively fighting hindsight bias.

Fighting hindsight bias – applying it in managerial judgement

Straight, let us try and apply the learning from geo-physics to managerial judgement. First, consider prior probabilities of an event happening appropriately. Imagine an angry boss (no, I would like to believe that all bosses are not always angry!). In trying to understand what angered her today, use prior probabilities appropriately. She may be angry because she was being held accountable for something beyond her control (like your productivity), or just that she gets angry when she is frustrated about not being able to communicate or convince others. Strategy #1: ask yourself, when is she ‘not angry’? She is not angry when you complete your work on time, when you present your work properly (as she likes it), and when your work is of good quality. Then why is she angry today? You have the answer.

Second, stop thinking sample size and probability. Unless you have a really large sample size of such events, stop thinking about probability. Imagine predictions in sport or financial services. I was taught in my first finance classes, “past performance is not an indication of future performance”. And my brief indulgence with sports tells me that the law of averages is that “sustained good performance does not last long”. Would you be confident in predicting the goals scored by a football team if their prior performances were [4-1, 3-0, 5-2, and 1-0] or [1-4, 4-2, 2-2, 1-0] with the second number in each pair representing the goals scored by the opposition? Most would be confident of predicting the performance of the former scoring pattern than the latter. It might just happen that the next game is against the league leader (including someone with initials of CR7) and all these performances do not matter at all. You really need to collect loads of data on each team’s performance, including historical performances of all the opposition teams before you make any predictions.

Third, stay away from causal relationships (no I did not say casual relationships!), unless you have really “big data” on both the normal distribution of the event not happening, as well as the outlier chance of the event happening. Remember the wonder batsman, Pranav Dhanawade, the 17-year old kid who scored 1009* runs for his local cricket team. After a few years, his father has decided to return the scholarship he received, since he has not performed up to expectations (read it here). It was important that when an event of this nature (an extraordinary performance) occured, one needs to not just reward, but also invest in nurturing the talent. Without an adequate support structure to hone his talent, the financial reward was insufficient to sustain even acceptable performance.

So why do some firms perform better than others?

The answer may not lie in analyzing why those performed better, but in understanding what the others do that make them not perform as well as the high performers; longitudinal and cross-sectional (big) data on multiple firms’ performance; and being very cautious about making causal assertions. Isn’t this the core of strategy research, today?

Cheers!

(c) 2017. Srinivasan R

 

Judgments

This is that time of the year when Indian business schools welcome their new students. As a self-proclaimed proponent of the case method of learning, I am often invited by my school to teach a session on “case method of learning” to the first year students. And one of my key messages to one such group of students this year was this: “a lot of what you will learn in the business school in terms of content, can be read from a variety of sources; what you will learn in class through continuous, repeated practice is the ability to make sound judgments.” This post is an elaboration of my understanding of the role ‘judgments’ play in business and life.

Judgment: what is it, anyway?

In the legal world, a decision made by a ‘learned Judge’ after hearing out all the arguments from all parties involved. The judge makes up her mind after providing equal and fair opportunity to all concerned parties to present their points of view; a detailed analysis of the evidence presented; collating expert opinions; gleaning through precedents and cognizant of the opportunity of this particular decision setting a precedent; and keeping the law of the land as well as the changing (socio-economic) contexts. When a judge presents a judgement, it provides a guideline for what is good/ bad; preferable/ not-preferable; acceptable/ not-acceptable in that particular context. To that extent, there is a subjective evaluation of the options, given the specific context; and a specific preference for one course of thought/ action over another.

Is it different from decision-making?

At a basic level, a judgement is a decision. But it is more than a just a decision. A decision by definition is a choice. Professor William Starbuck famously distinguished policy making (where resource allocation is a continuous process) from decision-making as ” … the end of deliberation and the beginning of action” (for more details on this quote, and in general, a history of decision-making, read this classic HBR article).

I see the primary difference between decision-making and judgement as managing risk and uncertainty. In his classic book titled “Risk, Uncertainty, and Profit”, Frank Knight (1921) defined uncertainty in a situation where the outomes could not be comprehensively enumerated and the probabilities of their occurances cannot be estimated. On the other hand, risk is a situation where all possible outcomes could be listed and the probabilities may be calculated.

Instructions and advice

One of my favourite assertions in my case learning sessions is the difference between instructions and advice. Instructions as we all know are directions for performing an activity step-by-step, a sort of a standard operating procedure. No thinking involved here – just go ahead and do what is written up/ told to. Whereas advice is contextual. Someone tells you, “it worked for me/ others in a similar context, you may try it yourself”. Of course, this implies that if your context is different, feel free to ignore/ adapt. Isn’t that why advice is always free?!

I bring in an example of how a little boy is taught to cross the street. Imagine his mother’s instructions: “before you cross the street at the zebra crossing, look for the policeman at the intersection; and when he signals you to cross, run across the street as fast as you can!” Wonderful … as long as the context is fixed. What happens if at the intersection, there is no policeman … does the boy keep waiting? What happens if the policeman does not notice him waiting to cross the street? Of what happens when the policeman signals him to cross the street, but a car is speeding towards him? What happens if …. ? Here is where judgements come in handy. Instead of providing him instructions to cross the street, his mother should develop judgement skills in him.

Imagine how you cross the street … if any of you have tried crossing the street in India, you know it better. I distinctly remember when my German colleagues while attending a conference in India, had a harrowing time crossing streets! When you cross the street, you look both sides of the road, spot a car a fair distance away (163m farther), driving relatively slow (at 26 km per hour). You estimate that at your speed of walking (4.5 km per hour), you will be able to cross the 80ft wide street a well 29.5 seconds before the car crosses the point where you intend to cross. You get the point, right? Nobody does all these calculations, we know it. Decision scientists call it intuition, gut, judgements.

It is developed through practice, accumulated through experience, and through active experimentation. Acculturation through socialization and mentoring may help in developing judgements; but no guarantee that just by repeating an action again and again, one would develop judgement. Apart from this practice and experience, a critical component of judgement is intent. Plus, an ability to weigh the pros and cons (in almost real time), as is in decision-making.

Intent in Judgement

One needs to have a specific intent to learn from experience. It is very likely that someone can continue to do an activity repeatedly without developing a sense of judgement. Something like a rote learning or Pavlovian Conditioning. How many times have you experienced people doing the same activities again and again not knowing why they are doing it, and why that way? Inefficient bureaucracies are built on the separation of thinking from doing; the doers are refrained from thinking … they are told to just do, and suspend thinking. Imagine blue-collared workers in the Taylorian world, or even BPO workers, or some customer service executives in modern-day organizations. It requires concerted intent to learn judgement.

Will I lose my job to automation?

The question in most cases is not if, but when? Judgement has never been more important as it is today. Roles where judgements are not required, activities that can be codified into detailed processes (where all possible outcomes can be enumerated and probabilities calculated), automation will take over. Bots and robots dominate the internet world today. Almost every website that has a customer interface has a bot running … and sometimes the responses could be hilarious. For instance, an airline customer thanked an airline sarcastically for misplacing his luggage and the airline responded with a big thanks for his compliment. Obviously, the sarcasm was lost on the automated response. The machine could not “learn” enough. And the entire twitterati took over (read about it here).

We live in a world today where the buzzwords include “big data”, “analytics”, “business intelligence”, and “artificial intelligence”. I recently saw a cartoon on a blog (futurethink.com.sg) that I can relate to very well.

Artificial-Intelligence-and-Real-Intelligence

As machine learning, automation, robotics, and augmented reality dominate our industrial vocabulary, natural intelligence and human judgement should take centrestage in our discourse.

Learning judgement

My advice to budding managers, invest in learning judgement-making. Consciously, with intent. Practise, make mistakes, experiment. Define outcomes and build expertise. After all, what you want to make out of your life and career depends on your judgement, right?

Cheers.

(c) 2017. R Srinivasan.

 

%d bloggers like this: