Standardising new approaches for validation of diagnostic assays and estimation of measurement uncertainty in veterinary diagnostic testing

Download Full Version MS Word format [81.5kb]

Axel Colling

Summary

Because the measurement process is not entirely reproducible, there is no exact value that can be associated with the measured analyte. So the result is most accurately expressed as an estimate together with an associated level of imprecision. This imprecision is the Measurement Uncertainty. MU is limited to the measurement process. It is not a question of whether the measurement is appropriate and fit for whatever use to which it may be applied. It is not an alternative to test validation, but is rightly considered a component of that process.

There are two main approaches to estimate MU:

1) The “components” or “bottom-up” approach identifies all sources of uncertainty individually in a “fish-bone” diagram. Chemical and physical testing laboratories tend to follow this approach because potential sources of uncertainty are usually readily identifiable, and their magnitudes can be estimated and combined. There are also published attempts to validate this approach in the medical testing field. For example, for serology, the uncertainties for time, temperature, volume, reading (OD), operator and reagent batch were identified to estimate the overall MU of the method [Dimech et al. 2007]. The advantage of this approach is that the major sources of uncertainty are clearly identified and weighted individually. The results from Dimech et al. indicated that reagent batch-to-batch, lab-to-lab and operator variation contributed significantly to the total variation whereas reading, volume and temperature contributed to a lesser extent. The disadvantage is that it is a time-consuming process because it requires a complex statistical model and repeated measurements of each component.    

2) The “control sample” or “top-down” approach is suitable for medical and veterinary diagnostic test methods because of the availability of quality control samples, which can be used to monitor whole-of-procedure-performance and directly estimate the combined MU of the test procedure. Upper and lower limits to approve or reject MU will depend on the purpose of the test. If the MU goal is not met it may be necessary to analyse the procedure to identify and modify uncertainty sources using the bottom-up approach. The advantage of this approach is the availability of repeatability data in diagnostic testing laboratories and simple calculations. The disadvantage is that the result is a global MU for the entire procedure and it fails to differentiate between individual contributing components. This example given in this paper focuses on the control sample approach because of its suitability in veterinary diagnostic testing and prooven acceptance by accreditation bodies.

Alternatively, the method characteristics approach, where performance data from a valid collaborative study are used as combined uncertainties (other sources may need to be added). Laboratories must meet defined criteria for bias and repeatability for the MU estimates to be valid. These should be larger than would be obtained by competent laboratories using their own control samples or components model (Dimech et al 2006).