Multi-Criteria Evaluation

Multi-Criteria Evaluation – The Foundation of Rational Decision Making Process

Multi-criteria evaluation is a fundamental step of the rational decision-making process. The purpose of evaluation is to gain reliable information on strengths, weaknesses and overall utility of each option. Even though the ability to conduct multi-criteria evaluation is a crucial decision making skill the process of evaluation is often poorly organized or even omitted.

The general concept of multi-criteria evaluation is presented on the picture below:

Multi-criteria evaluation concept. From multiple criteria, alternatives and evaluators to the common group decision

Multiple Alternatives

There are multiple alternatives that we are going to evaluate. For example: projects, suppliers, software packages, ERP systems, locations, investments, candidates, strategies, technologies, processes…

Each alternative has strengths and weaknesses. But we can’t look at alternatives only through the lens of characteristic features. We should ensure common evaluation standards for each alternative so we need a common set of criteria...

Multiple Criteria

Even in the case of such a trivial decision as choosing a movie to watch on Friday night, we can identify at least several criteria that can be used for evaluation: the type of movie, actors, reviews, awards, nominations, duration, director.

When you buy a car you look at the price, maintenance costs, appearance, safety, comfort, etc. When you choose a job you can consider salary, promotion and career development opportunities.

In the case of complex business or governmental decisions such as vendor selection, projects prioritization or investment selection there are usually dozens of criteria.

Multiple Evaluators

Often, there are multiple people whose opinions must be taken into account during decision-making process. So we need a framework that can be used for collaborative evaluation.

Common Results

Multi-criteria evaluation gives you alternatives’ utility values. Utility values are based on aggregated and collaborative information on partial utilities (in context of each criterion separately) and criteria weights (criteria are not equally important).

The result helps you to make rational decisions, like:

  • choosing the best option e.g. ERP system provider,
  • selecting multiple options e.g. projects from portfolio,
  • performing prioritization e.g. of projects or tasks,
  • allocating resources e.g. capital, consultants, time machines.

Multi-Criteria Evaluation for Different Types of Decisions

Type of decision Importance Decision Maker Multi-Criteria Evaluation
Tab. Recommendation for multi-criteria evaluation based on the type of decision
Everyday Decisions Low Individual Not Recommended
Important Personal Decisions High Individual Recommended
Decisions Which Must Be Justified High Individual or Group Highly Recommended
Important Group Decisions High Group Highly Recommended
Strategic Decisions Extreme Group Absolutely Needed

Insignificant Everyday Decisions

An accurate multi-criteria evaluation takes time so it should not precede all your decisions. Every day you make a lot of choices and this approach wouldn’t allow you to live a normal life. In case of unimportant decisions, when consequences aren’t significant, you can rely on your feelings and intuition.

Important Personal Decisions

Multi-criteria evaluation requires some effort but it greatly increases the chances of making a good decision. When you make a choice that will have significant impact on your future, it is worth to invest some of your time to do it right. Consider decisions related to your career or money. Don’t they deserve a special attention?

Rational decision-making is like a diet. Everybody knows that this is good for us but often we choose the worse but easier way. Now you know that your decisions can be better. Are you going to take advantage of this knowledge or do you still trust your feelings? It is your choice.

Decisions Which Must Be Justified

Multi-criteria evaluation allows you to justify and document your choices.

Group Decision Making Process

Imagine buying a car together with a few friends. You have defined the budget and on this basis selected 6 car models from which you are going to choose one. It was quite easy but now it is getting harder…

Everyone has a different opinion. The most important criterion for you is safety, for someone else – price, the other appreciates performance and someone else – comfort. Besides, everyone likes a different color and each has a favorite brand. How to make a collaborative choice?

Group decisions often cause conflicts. Decision making process can take a long time and be filled with emotional discussions that don’t lead to a solution. It is impossible to find a solution that fully satisfies everyone. However, you can ensure objective and fair decision-making process which guarantees participation of everyone on equal terms.

So voting ???!!!

Voting is the simplest solution that allows you to make a fast choice and everyone participates on equal terms. Unfortunately, voting has little in common with collaborative decision making. Car selection through the voting will be the brand popularity contest rather than the rational decision-making process.

So, what is better than voting?

Solution that will help you to avoid conflicts, save time and make choice that best fits needs of group is the collaborative multi-criteria evaluation. Collaborative multi-criteria evaluation is a series of voting on specific aspects of decision. This solution combines the advantages of voting (the speed of reaching the outcome and fairness of the process) with the rules of rational decision-making.

And what if we don’t do this?

Maybe you’ll make a good choice and nothing wrong will happen. Maybe conflicts will occur and your relationships will suffer. Maybe you’ll suffer the consequences of bad choice. When there is a lack of repetitive process the consequences are difficult to predict. Sometimes it will be good, sometimes bad. You just never know.

However, there are decision making problems when multi-criteria evaluation is not only a help but an absolute essential decision making must-have.

Strategic Decisions

You already know that the formal process of multi-criteria evaluation is useful when decision is important (improves quality of decision, helps to justify and document) and while the group decision-making (helps to avoid conflicts, saves time, everyone participates on equal terms).

But what if we combine both factors and turn them up a bit? What if decision is extremely important and its consequences affect a large group of people such as company’s owners, employees, partners and customers? What if decision is being made by a large group of directors, each of which represents the interests of his department?

In this way, we have received the characteristic of strategic decision. It is complex, difficult, very important, there are different groups of interest and it affects a large group of people. When making a strategic decision it is in the interest of all stakeholders to organize the reliable process of multi-criteria evaluation.

So if you face a strategic decision the right question is not whether you should conduct multi-criteria evaluation, but how to do it. And you will find the answer in the next section...

How To Evaluate Alternatives In Context of Multiple Criteria?

As you see there is a certain category of problems where a systematic multi-criteria evaluation is very helpful, even indispensable. Nature abhors a vacuum so there are a lot of MCDM (or MCDA) methods that satisfy this need. MCDM methods differ from each other but they are all based on the rational decision making process described below.

  1. List alternatives that will be evaluated.
  2. Identify criteria that will be used to evaluate alternatives. You can organize them into hierarchy. You can find sample hierarchies here: decision models
  3. List evaluators, the people who will take part in evaluation process.
  4. Assign weights to criteria – they are not equally important.
  5. Evaluate alternatives in the context of each criterion separately.
  6. Collect it all together and calculate the total utility.

Analytic Hierarchy Process

Intuitive Multi-Criteria Evaluation Method

One of the most popular MCDM methods is the Analytic Hierarchy Process. AHP is very intuitive, easy to use and understandable and thus beats most of the other MCDM methods that have a solid mathematical background but are so complex that can be used only by scientists and qualified decision analysts.

MakeItRational is easy to use software for collaborative multi-criteria evaluation based on the Analytical Hierarchy Process.

Find out more