All designs must be verified before they can pass on to manufacturing. Designs are verified by performing analyses, tests, and simulations on various models and prototypes of the design, which are intended to quantify the performance of the design. In order to evaluate the results of these tests, we must have some scale against which the results can be measured.
A performance metric (PM) is a parameter that is used to measure the performance of a design with respect to some criterion. It is usually given as the difference a fixed, baseline value and some comparable value measured by way of a test of some sort – in other words, it measures how far off the target performance values a given design is. A good set of PMs will let us accurately determine the overall validity of a design.
The most basic requirements of a product are those supplied in the product design specification (i.e. the design problem itself). Defining the initial product design specification is the first step of a design process. Since any acceptable design must meet the PDS, the most important performance metrics are those addressing the requirements in the PDS. It makes sense that these PMs be set at the beginning of a design process, right after the PDS has been set. This means it is the designers' job to determine the PMs.
Once PMs are set in place, test engineers can immediately begin to plan for the tests that will be needed to actually measure the design along those metrics. (This will be more evident when we look at some examples below.) Based on their expertise and experience, test engineers can begin to lay out a timetable for the tests, and for the analysis of their results, and for the manufacture of test equipment and specimens and instrumentation. In this way, they can "get the jump" on the whole test phase and thereby help shorten the lead-time for getting the product to market.
A PM need not be tied to a single component or system. It might be a quantity that is emergent, that only occurs in the product as a whole, once the product is in operation.
Only some PMs can be defined at the beginning of a design process. As a design matures, more PMs usually arise quite naturally from the need to validate components and sub-assemblies of the product.
Typically, PMs are defined with respect to minima, maxima, or optima; that is, they are based on the constraints in a design problem. Here are some examples.
| Baseline Constraint | Performance Metric | Sample value | |
|---|---|---|---|
| Maxima | The automobile cannot weigh more than 2000 kg | dW = (Wmax – W) / Wmax | dW = (2000 kg – 1980 kg) / 2000 kg = 0.01 |
| Minima | The headroom in the car must be at least 36 inches | dH = (H – Hmin) / Hmin | dH = (38 in – 36 in) / 36 = 0.06 |
| Optima | The smelting furnace operates optimally at 600 C. | dT = (-abs( T – Topt )) / Topt | dT = (-abs( 610 C – 600 C )) / 600 C = -0.02 |
Note that for optimal values, any variation – above or below the baseline – is undesireable
Notice that only for the second case is the PM a characteristic of a part of the product. Indeed, for the furnace, there is no single component that actually ever operates at the operating temperature!
Notice that the way we calculate the values of the performance metrics changes from case to case. The different kinds of formulations are to ensure that positive values always denote preferable designs. The more positive, the better.
We want to keep all PMs measured this way because it greatly simplifies the comparison of alternative designs for the same PDS. Say we have 10 performance metrics derived from a PDS, and say we have three competing designs. We can test each design and tabulate how they compare for each performance metric. Since large positive values are preferred, it becomes very easy to just scan a table of performance metric values for the designs to see which one is best. We can also use such tabulated test results to spot areas of concern in a given design: if a design is tested and is getting good (positive) values for all but a few of the performance metrics, then we know to focus attention on those areas of the design that are not performing well.
Since PMs are tied to constraints, and since the number of constraints grows as a design evolves, one can also expect the number of PMs to grow with time. So design teams need to pause regularly during their design work and make sure that they have identified any new PMs that have arisen.
