Halo Effect Impact for Tender Evaluation (and Ways to Fix It)

The Phenomenon

Halo effect is a type of cognitive bias. Cognitive biases are errors in thinking that influence how we make decisions. This phenomenon affects us both at conscious and sub-conscious levels. It affects personal and professional life decisions. Nobody is an exception. Not recruitment specialists, not IT and not procurement or any other specialist area. Procurement professionals, however, are exposed to it even more – they have to deal with multiple external sources of influence every day. This subject is very wide, but this time let’s look into a very narrow area: tenders and their evaluation.

Cognitive bias in procurement

“Procurement Academy” provides some very simple and good explanations of the most common biases. According to them, cognitive biases appear during the negotiation process or in daily communication with suppliers and would not apply to tender process. Therefore, the tender would make your buying process efficient. That is not necessarily so. Let’s leave all the malevolent intentions aside – this time we talk only about unintentional cognitive bias.

Cognitive bias in tenders

There are a lot of different types of biases. Too many to analyse all of them here. Also, tender processes vary across different companies. However, here are the examples how, without even knowing about it, your decisions on supplier / product / service selection might be influenced:

  • Overconfidence. I wrote about it some time ago. I loved Jennifer M Wood’s article on similar subject, where she points out: “Experts are more prone to this bias than laypeople, since they are more convinced that they are right”. How is the connected with the tenders? Well, WHO is running your tenders for you?
  • Halo effect: the tendency for an impression created in one area to influence opinion in another area. Think about Account sales manager David. During the tender presentation meeting, he was charming: professional, knowledgeable, personable. Did you not automatically expect the whole company to be just as David is? Another example is the sequence of replies to the questions. It is proven, that if the first questions are replied extremely well, the evaluator tends to evaluate the following replies much higher. Was there a time pressure to submit the tender proposal? Most likely the greatest effort was put into replying the first questions.
  • Framing effects: You will tend to make different conclusions from the same information, based on it’s layout: sequence of words, mathematical representation. Think about it: would you rather have a half-empty or a half-full glass of water? Would you choose 90% fat free or 10% fat yogurt? All tender submission documents will be “adjusted” to communicate “the right” message.
  • Base-rate neglect: it is a tendency to ignore base rate information (say, historical supplier performance information) and focus on specific information (a tender submission data). I did my own small research on this: results are here. We are all guilty of this.
  • Confirmation bias: I tend to call it selective hearing or memory. People tend to search for, focus and remember information in a way that confirms their perceptions. The questions asked in tender documents here become critical. Compare: “Is Sam friendly?” and “Is Sam unfriendly?” These questions are about the same thing, but the answers would give you a different picture.
  • Post-purchase rationalization. It is not a joke. There is such a thing.

 

Ways To De-Bias Tenders

First of all, we need to remember, that as long as there are people involved, there will always be various types of communication and thinking variations. Also, the tender itself is a very limited tool to ensure procurement efficiency. However, sometimes you will have to use them. When you find yourself running a tender:

  • Consider confidence calibration. No, I am not suggesting NOT hiring experts overall. I am suggesting calibrating your experts. And, sometimes, to can ask yourself: why exactly am I hiring a SME? What do I expect from him / her? Why do I think, that having run a tender for OTHERS several times means they will know what is best for OUR company? What exactly am I looking for?
  • Evaluate the company, not the person that is representing it. Meet all of the team (or the core team), that will be running your account and servicing your company. Use testing platforms to get some data to back up your impressions. OR do not meet anyone at all.
  • Remove the supplier names from their replies before handing tender submissions for the evaluation.
  • Improve question evaluation process: split the questions and evaluate so that you can compare question 1 from supplier A with question 1 from supplier B as opposed to firstly evaluating all questions from supplier A prior to moving on to supplier B.
  • Use reframing techniques: challenge assumptions; ask wicked questions (create paradoxes); use “5 Why?” process; adopt multiple stakeholder perspectives).
  • Scientists and philosophers by-pass confirmation bias by trying to disprove theories. Challenge and doubt all sales claims. It is better, if you have historical data which you can analyse. If your suppliers provided you with 10 statements of how well they will meet your business requirements, try to disprove all. If you succeed in half of them… well, then should your tender evaluation reflect that?
  • On the other hand – be critical about the questions you ask in your tender questionnaires.
  •  

Knowing how our brain works helps us not to only avoid our own errors. You can use this knowledge to your advantage. Are you taking any exams any time soon? Reply firstly the questions you know best. And make sure to make a magnificent impression for all of the further questions.

Advertisements
Halo Effect Impact for Tender Evaluation (and Ways to Fix It)

If I had to put my money on it…

One sentence. That is all you need to reset your brain from simple biases (a.k.a. mistakes in other words). But let’s start from the beginning.

To sum up, logic is not that straightforward

It all started during my holidays. I thought I would do some reading. Which I did. But, besides giving me some tools for my work, the book also challenged my understanding about humans being logical creatures. The book (“How to measure anything” by Douglas W. Hubbard) also talks about perceptions and games that human brain plays. It talks about classical statistics and Bayesian statistics (I am preparing a book overview for my next post. This post is about my experiment).

One of the phenomenons that the book describes is how various biases affect people’s decisions. How perceptions win over logic (say what???). How people choose to believe few pieces of new information and disregard all of the data and facts that they have at hand historically. Having read that, I felt almost insulted for all of us. Oh my saint naivety!

Off to experimenting!

So I decided to test the methodologies and experiments, described in the book. I took exactly the same question, which was used in the book and started asking people around me: “there are 100 specialists in the room. 95 of them are criminal lawyers and 5 – paediatricians. One person was selected randomly. It turned out to be Jane. Jane likes science and she loves kids. What is Jane’s occupation?”.

I started with my partner. First big surprise! On the other hand… knowing how he bought mascara for me… I decided I needed to increase the sample. I tested it with one of my CIPS class students and I was also surprised, again. The final step was to broaden the audience and get the feedback from more people – which I did with a LinkedIn post. I have received many different replies:

  • Statistically most likely. Lawyer. There, I said it – most likely, Jane is a lawyer.
  • Statistically less likely. Paediatrician. This is the grey area. It is not entirely incorrect answer. Explanation will follow later in the text.
  • Different: politically correct, challenging, sensitive, thoughtful, show-off. Knowing, that everyone is first of all a human with a personality, before becoming a professional – you would not expect anything less, right?

Overall the conclusion is one – people are not statistical machines and they are always prone to biases. The book was right. I was wrong to think that people, as a rule, are logical.

The theory

The reason why I avoided to give “the right” answer is complicated. Even statistics is not that straightforward. There are different approaches to it. The book analyses two different approaches – classical and Bayesian.

Classical statistics makes two assumptions:

  1. The data you are analysing has normal distribution.
  2. You know nothing else about the phenomenon you are analysing apart the data you are given.

Needless to say, both of these assumptions many times are wrong. Bayesian statistics, on the other hand, says that whenever you are making a [business] decision, you should take into account (or not ignore) other available information.

In the example given above, you have two portions of information: 95 criminal lawyers and 5 paediatricians; and some new information – Jane’s personal characteristics. Which piece of information do you choose to believe? Which part is more facts based and which is more biased (“standards”, sales and marketing statements, clichés)? That is up to the specialist to decide.

The Application

If you think this is not relevant to you, you are wrong. It is not just a story from a book:

  • Remember every time you hear a sales pitch of yet another P2P system, which “will solve all of your problems”.
  • Remember all the times, when you get a feedback, based on wrong biases and assumptions: no matter if it is recruitment process, annual performance feedback meeting or any other type of feedback.
  • Remember tender evaluation process. I have seen too many tenders, where all of the historical information about supplier performance is put aside, purely because during the tender the new sales team made a great sales pitch. A supplier was always late and delivered poor quality service? And you choose the same supplier again because the sales team promised that everything will change? It is up to you to choose what you believe.

People make decisions based on biases and perceptions more often that they think they do. Scary?

The Fix

There is a fix to it! And it works! I also tested it on myself and my test groups. Almost ALL people who initially chose paediatrician as an option changed their decision when I asked this question: “If you had £1000 and had to put it on your option – what would you do?” (That is also a suggestion from the book). If they had to put their own money on one of the options, they would choose lawyer.

Without going into details, I will sum up: if you are asked a question and you want to give a more confident reply, start it by saying “if I had to put my money on it, I would…”. It is proven, that this helps to re-calibrate your brain. You might still choose the same option, but by saying that, you switch on a different part of brain – that means your reply will be considered from different perspectives.

My reply to the tender evaluation could be: “If I had to bet my money, I would say that the supplier will continue being late, despite what they claimed during the tender. However, even knowing this, I would recommend awarding the contract to them. It is a risk that we already know about and we need to work on it together with the supplier. Instead of being undefined threat, it becomes an action plan and, potentially, a strength”.

 

Hope this is helpful. I thoroughly enjoyed receiving the answers to my question on LinkedIn. Thanks to everyone who contributed! Thank you for reading!

 

If I had to put my money on it…

Types and levels of proofness

For instance, I blog under Futureproofitable: that is future proof and profitable in one word. You can buy things cheap. But if they are of no use – you are not getting far as a business.

The same rule applies across all of the organisations (private or public). You can have the strongest Procurement processes and function in place, but if the decision itself to buy something is wrong, then nothing can save you.

There are few levels of proofness, as I categorise it:

  • Idiot proof: if you know there is a risk of completely failing at the task or a project, just do not do it.
  • Future proof: you know there is a risk, but you choose to address it in advance.
  • Future proof and profitable: you know there is a risk, you choose to address it in advance in a way so that you can turn it into revenue stream in the future.

I will be sharing more examples in the future as I come across them. Today, I already have a bit of a library gathered. Here’s one example.

A PFI object. A health institution. You would expect the highest standard of cleanliness and hygiene. And still… you come across an artwork hanging from a ceiling like this:

ceiling art.jpg

Can you think of all the things that can go wrong? Cost of cleaning it? More like not affordable to clean? And if you cannot afford to clean it – what kind of risks are you exposing everyone to?

What has that got to do with future proofing? Here you go:

  • Level 1, #idiotproof: don’t do it. Just don’t. It adds no value, costs a fortune itself, the maintenance costs even more; without maintenance – it becomes a source of various risks.
  • Level 2, #futureproof: make it machine washable. I am not suggesting I know HOW to do it. But if you really need it in the first place – make it machine washable.
  • Level 3, #futureproofitable: make it machine washable “hall of fame”: hang a small artwork in the name of everyone who has contributed through charity to the purpose of the object.

Do you have any examples like this?

Types and levels of proofness

RF-Spy: Procurement on a Mission

I am on a mission. Yes, this is what I think every time I start a review of a project or a spend category – I am on a mission. I have a goal in front of me: to do whatever it takes, to get the best result. I was discussing with my colleagues, what should be the starting point for the procurement process. I heard many replies. The most frequent reply was – RFI (request for information) is (or should be) the first step. Not for me. I start with a game my kid taught me: “I spy with my little eye…”

The understanding on an RFI varies. Wikipedia says it is “a standard business process whose purpose is to collect written information about the capabilities of various suppliers. Normally it follows a format that can be used for comparative purposes… sent to a broad base of potential suppliers for the purpose of conditioning suppliers’ minds, developing strategy, building a database and preparing for an RFP, RFT, or RFQ”. The key negative aspects of this definition to me are “standard”, “comparative”, “written information…to condition suppliers’ minds”. How can you choose a standard method of measurement, if, at the point of going into the market, you do not yet know what you are looking for? How can you aim to compare outcomes, if by default, RFI stage should allow you to collect completely different methods, business cases, options? How can you gauge someone’s minds over standard ten page questionnaire? By the time any supplier reaches the end of such a questionnaire, they hate you anyway. You need a dialogue to gauge someone’s mind, views and approach. RFI is a one way street.

I have asked our colleagues over LinkedIn post (big thank you for those who responded): “what is the real purpose of RFI for you?”. People use RFI:

  • To be sure the supplier is able to fulfil the needs.
  • To validate assumptions, prove or disprove hypothesis.
  • To validate buying organization’s requirements.
  • To develop buying organization’s requirements.
  • To ensure transparency and business ethics.
  • To assess suppliers on qualification requirements.
  • To capture the information for short listing purposes.
  • To make sure everyone is given a chance to participate.

Ultimately, RFI is a communication method. It is neither good nor bad. It has to serve the purpose of what you want to achieve:

  • To check suppliers’ financial capabilities? Choose independent information sources instead.
  • To “condition suppliers’ minds”? Choose a meeting and/or a conversation.
  • To evaluate supplier’s capability to fulfil the needs (quality, capacity)? Choose supplier audits.
  • To act ethically and transparent? Do it, instead of talking and making manifestos about it. Inviting two hundred suppliers (fine, slightly exaggerating) into a RFI / RFP process is expensive, not transparent. You don’t trust your own employees? Well, you already know what’s coming next – why do you employ them in the first place?
  • To make sure the supplier’s project team will see through to the end of the project? Ask for staff retention statistics, not for a standard list of CVs.
  • To find out what is happening in the market? Do your researches. Nobody these days can complain of lack of information.

It might seem funny (or silly – I do not mind calling things their real names), but I tried to find “instructions for spying”, “private investigator processes” and similar keywords on the internet. You know, to validate if my work principles are any similar to the real spy games (fine, I have been watching too many movies, I know). And I did find things. Mostly, governmental process descriptions, and they were of not much use. However, I did come across some indications. Mostly those were lists of tools that they can use, based on the situation and their own expert judgement. Yes, the tools can be standard, but the combination, that you use, is what makes it unique.

Because of this approach, I choose to do my homework (the spy game) first. Then, when I know my alternatives, I issue pre-qualification questionnaires (if there is anything else that I still want to ask the suppliers and it was not validated during my homework stage). Getting onto my first stage long-list (which is no-where near 200) is already an achievement. And there is no need to question my motivation: I represent business interests. Making sure, that procurement process is not too long and not expensive is one of the objectives. Political correctness? I choose to be polite instead. Those suppliers, who were not invited to the process, should appreciate that I respect their time and choose not to waste their resources by dragging them into a process if I know they are not suitable for it.

Do you have your own unique approach to this? Please share in the comments below!

RF-Spy: Procurement on a Mission