THIS PAGE IS OBSOLETE AS OF 1 JAN 2020 for MEC325.
Pairwise comparison is a tool to rank a set of decision-making criteria and rate the criteria on a relative scale of importance.
Making decisions requires comparing alternatives with respect to a set of criteria. If there are more than two criteria, determining which criteria are more important can itself be a serious problem. One would like to be able to rank the criteria in order of importance, and to assign to the criteria some relative ranking indicating the degree of importance of each criterion with respect to the other criteria.
Here are some examples:
It can be very difficult to rank and weight criteria. It can become insurmountable in complex problems because every criterion must be weighted with respect to every other criterion simultaneously; this is a problem that grows arithmetically. (For example, given 5 criteria, there are 4+3+2+1=10 relationships to consider, and for 10 criteria there are 45 relationships to consider).
Pairwise comparison is one way to determine how to evaluate alternatives by providing an easy and reliable means to rate and rank decision-making criteria. It is often used to assign weights to design criteria in concept evaluation.
Pairwise comparison is implemented in two stages:
For our approach, this step has already been done, through the definition of a PRS. The criteria we will use are the product characteristics. Remember that each PC is really just a label for all the requirements flowing from it in the PRS.
Here is an example, using some fairly typical, general PCs. You would use the PCs from your specific PRS.
Each cell in the matrix corresponds to a comparison of a pair of items (hence the name “pairwise comparison”). The cells will contain the item that is considered the most important of the pair.
Obviously, we only need one triangle of the matrix, because comparing A to B is the same as comparing B to A. Since the rows and columns contain exactly the same things in the same order, one triangle of the matrix will contain just a mirror image of the other.
Furthermore, the diagonal of the matrix is irrelevant – it simply doesn't make sense to consider how important one criterion is with respect to itself!
So now we have:
For each row, consider the criterion in the row with respect to each criterion in the rest of that row.
The first comparison in our matrix is
durability. Which is more important? Discuss this in your teams and reach a consensus to that question. In the corresponding cell of the matrix, put the letter of the criterion that is most important. Do not consider any other criteria beyond the two that you are comparing.
Then go on to the next pair (in our example,
quality). If we really, really think that two criteria are of equal importance, then we put both letters in the corresponding cell.
Note that the individual comparisons are pairwise – we completely ignore the other criteria.
Say we decide that
functionality is more important than
durability. We would then put a
A into cell
(2, 4) of the matrix. We continue doing this till all the empty cells have been filled.
We might end up with something like the following:
Notice the double letters in some cells. We have used this convention to indicate that the two criteria being compared are of equal importance; that is, you just could not decide which of the two criteria was more important.
This step must be documented to provide justifications for your decisions.
Now we simply make a sorted list of the criteria, ranked by the number of cells containing their flag letter. For example,
functionality is marked by the letter
functionality gets a rank value of 4, because there are 4
A in the matrix. This leads to:
Finally, we want to associate weights with our criteria so that the relative ranking from the pairwise comparison is satisfied.
There are two basic constraints on how we assign the weights.
We can either begin to wrestle with the problem in a strictly ad-hoc manner, or we can try to structure our solution. It's inevitable that some iteration will be required, so there's no point in looking for a method that will give us actual weights in one pass. However, we can try to set up an initial set of values that does satisfy the constraints, and then tweak the values until they are satisfactory for all stakeholders.
One very easy way to set that initial set of values is to assume a linear proportion between all the weights and solve the following equation (for our example):
100 = 7x + 6x + 6x + 5x + 5x + 4x + 4x + 2x + 0x
x = 2.56 (approx), where the coefficients in the equation are the number of occurrences of each criterion in the pairwise comparison matrix. This leads to:
maintainability arose by gathering up all the round-off error from the other calculations.
There are obvious problems with this approach. For example, the lowest ranked item in a pairwise comparison can often turn out to have a weight of zero, as we found in the example above. However, we cannot assume that zero importance implies we can omit it altogether. If we do that, we have to redo the entire pairwise comparison, and then some other criterion may end up having zero importance.
So, when we have a criterion that appears to have zero importance, we borrow a fractional amount from other criteria and give it to that last-ranked criterion, just to make sure it counts for something.
Other strange effects can occur. For example, it is possible to get a situation where given three criteria, say A, B, and C, we find that A is more important than B, that B is more important than C, and that C is more important than A. This is a paradoxical situation (known as Arrow's Paradox) in that it makes no sense. Generally, this happens when people work individually rather than in teams.
No matter what else happens, however, it is essential that all stakeholders in a design agree to the actual weights. Pairwise comparison is therefore best done in teams and not individually.
Salustri has a simple Google spreadsheet that can be copied and used as a template for carrying out a pairwise comparison task. The sheet will track the sums and percentages for you; all you have to do is list the criteria and make the actual comparisons.
Remember to enter into the cells the letters corresponding to the attributes themselves not the column letters.
You may only change cells that have a white background; do not change anything in gray cells, or you will break the spreadsheet.