Weighted Decision Matrix
A weighted decision matrix is a tool used to compare alternatives with respect to multiple criteria of different levels of importance. It can be used to rank all the alternatives relative to a “fixed” reference and thus create a partial order fo the alternatives.
Introduction
Making decisions is both important and difficult. You must make decisions that are justified and in which all stakeholders have confidence. It is also important to document decisions in clear, structured ways to ensure that others will understand your reasons for having made a decision, long after the decision is made.
There are often many different criteria that need to be considered in making a decision. It is essential to identify the criteria, and to make the decision with respect to those criteria as precisely as possible. You cannot just maximize each criterion - that's just not possible - and maximizing one criterion very likely restricts you to fail to meet other important criteria. That is, the “best” design almost always involves a design trade-off.
The matter is further aggravated when there are many alternatives from which to choose; in these cases, not only does each alternative need to be examined, but all the alternatives must be treated consistently to ensure that a final comparison of all the alternatives is justifiable.
A weighted decision matrix (WDM) is a simple tool that can be very useful in making complex decisions, especially in cases where there are many alternatives and many criteria of varying importance to be considered.
WDMs are often used in design engineering as a qualitative tool to evaluate alternatives. This page explains how they work in general; other topics will show how they are used in specific design tasks.
WDMs are often used in concept evaluation.
A template Google Sheet for WDMs is available here.
Prerequisites
To use a WDM, you need certain information:
- a set of well-defined criteria;
- a set of weights that define the relative importance of the criteria;
- a reference against which comparisons will be made; and
- a well-defined set of alternatives to be ranked.
How you generate these required data will depend on what you want to use the WDM for. In engineering design, the required data are typically developed during design stages leading up to concept evaluation.
Structure of a WDM
A sample WDM is given below. The problem it addresses is deciding which of a few possible travel itineraries a specific person might follow for a European summer vacation1).
| CONCEPTS | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| REFERENCE TRIP | Trip A | Trip B | Trip C | ||||||
| Criteria | Wgt | Rating | Score | Rating | Score | Rating | Score | Rating | Score |
| Travel cost | 0.25 | 0 | 0 | 1 | 0.25 | 0.0 | 0.0 | -1 | -0.25 |
| Total Cost | 0.20 | 0 | 0 | 0.0 | 0.0 | 1 | 0.20 | -1 | -0.20 |
| Novelty | 0.15 | 0 | 0 | 2 | 0.3 | 1 | 0.15 | 2 | 0.3 |
| Locations | 0.10 | 0 | 0 | -1 | -0.1 | 0.0 | 0.0 | 2 | 0.2 |
| Travel time | 0.10 | 0 | 0 | 0.0 | 0.0 | -1 | -0.1 | 1 | 0.1 |
| Safety | 0.10 | 0 | 0 | 2 | 0.2 | 1 | 0.1 | 2 | 0.2 |
| Accommodation | 0.05 | 0 | 0 | -2 | -0.1 | -1 | -0.05 | 2 | 0.1 |
| Travel quality | 0.05 | 0 | 0 | -2 | -0.1 | 0.0 | 0.0 | 2 | 0.1 |
| TOTAL | 0.0 | 0.45 | 0.30 | 0.55 | |||||
| RANK | 2 | 3 | 1 | ||||||
| CONTINUE? | yes | no | yes | ||||||
The criteria were established via the experience of the person who built this WDM about past vacations, and reflective thinking about how best to organize them.
- It's important that the criteria be independent of one another, or as nearly so as possible.
It is best to rate the concepts before calculating the weights of the criteria.
- This is because of a natural human bias to not think hard about things we “know” are not important. If the weights had been calculated before the ratings were done, the person doing the WDM may have tended to unconsciously neglect the lowest weighted criteria, which could have ruined his vacation.
Here's how you generally go about filling in a WDM, based on the example:
- Consider the first alternative, Trip A.
- Now consider the first criterion – Travel Cost – and ignore all the other criteria.
- This is particularly important: ignore all the other criteria. The whole point of a WDM is to divide a complex problem into many simpler problems. This means treating each criterion as completely separate from every other one. The WDM will take care of putting everything together again for you, later.
- Compare Trip A to the Reference only for Travel Cost.
- Assign a
Rating(typically with a standard 5-point Likert scale) for that comparison. - Repeat this for each criterion.
- Repeat all this for all the other alternatives.
- Develop the criteria weights using pairwise comparison.
- Multiply each
Ratingby theWeightfor that criterion. This gives theScorefor that item. - Sum the
Scores; this goes in theTOTALrow. - Fill in the
RANKrow by ordering the concepts in descending value of total score. - Decide whether to continue considering a given itinerary. This is given in the very last row of the matrix.
However
It should be clear that a WDM is not a static document; it can change and evolve in parallel to both your understanding of the problem and the development of a solution. So it's important to keep old versions of your WDMs, to track the history of your work.
Remember the ladder design brainstorm? Ladder design evaluation shows a decision matrix for that problem.