Cut-off values
The determination of cut-off values
ACOMED statistics is a statistics service provider with a focus on the design and analysis of diagnostic studies. We also offer statistical consulting
and advanced training in statistics. We support companies in the pharmaceutical industry, medical device industry
and CROs in statistical design and analysis of clinical trials as well as in SAS programming.
On this page you will find some hints on cut-off values, their calculation, their application and their interpretation. If you need support in the design and analysis of your diagnostic study, please do not hesitate to contact us (Tel.: +49 (0) 341 3910195).
The determination of cut-off values as a regulatory task
Manufacturers of diagnostic tests are required to provide values that divide the value range of the measured values into test positives and test negatives. This value - the cut-off value - is one of the properties of a diagnostic test. This is also provided for in the new EU regulation 2017/746.
The determination of the cut-off as a medical task
Different cut-off values are associated with different values for sensitivity and specificity, in the sense of a trade-off: higher sensitivities are associated with lower specificities and vice versa. This trade-off can be clearly seen in the following figure.
The definition of the cut-off is therefore primarily a clinical / medical, possibly also health-economic problem: Do you want to use the test with a high degree of specificity (few false positives) or one with a high sensitivity (few false negatives)?
The following figure shows such definitions using the example of the tumor marker Cyfra 21-1 for bronchial carcinoma (data from: Keller T, Bitterlich N, Hilfenhaus S, Bigl H, Löser T, Leonhardt P (1998): Tumor markers in the diagnosis of bronchial carcinoma: new options using fuzzy logic based tumor marker profiles. J Cancer Res Clin Oncol 124: 565-574),
being the ROC curve
is taken to help.
An example of such a definition is the PSA test for diagnosing the prostate Ca of men: Here the cut-off is chosen so that a high sensitivity is achieved (approx. 90%) and takes a very low specificity (30 ... 40%) with the corresponding consequences (e.g. invasive follow-up examinations).
Conclusion for the determination of cut-off values:
From a clinical medical perspective, the decisive step is to determine the specificities or sensitivities that the test should have. The associated cut-off value is then determined, and the associated sensitivity or specificity is specified.
This has an important consequence: the frequently encountered determination of the cut-off value using criteria such as the maximum diagnostic quality is not expedient.
The determination of the cut-off as a statistical task
The estimate of the cut-off using statistical methods corresponds to the estimation of a percentile in a distribution. If the cut-off related to 90% specificity is to be determined, the 90% percentile of the distribution of the non-sick should be specified. A number of methods can be used for this purpose, as a rule the percentile is determined nonparametrically. The corresponding CLSI guideline EP28-A3 also provides for this. This guideline specifies a sample size of 130, which however depends on various boundary conditions.
It is important to keep in mind the uncertainty of the percentile estimate. This can be specified by the confidence interval, whereby the CLSI EP28-A3 guideline uses the 90% confidence interval. This is (unexpectedly) broad, the following figure shows this for the cut-off of Cyfra 21-1 from the study mentioned above:
The figure shows the distribution of the benign cases (N = 131), together with the cut-off (3.2 ng / ml) for a predetermined specificity of 85%, together with the 90% confidence interval (2.8 - 3 , 9 ng / ml).
Although the sample size is quite large at 131, the cut-off is rather uncertain.
Conclusion: The cut-off value must be given with its confidence interval.
It can also be seen that the distribution of the benign cases is heterogeneous, in particular the right "tail" of the distribution obviously consists of sub-distributions. This makes cut-off determination with parametric methods virtually impossible.
Incidentally, the non-adequacy of determining the cut-off value via the maximum diagnostic quality can also be seen from a statistical point of view: If you determine the confidence interval for this cut-off estimate (lower figure on the left), it is much broader than the one above shown example.
Cut-off and confidence interval (gray horizontal lines) for max. Youden index (left) and fixed specificity (right):
Youden index: 3.2 ng / ml, 90% CI: 2.4 - 5.0 ng / ml
Fixed specificity: 3.2 ng / ml, 90% CI: 2.8 - 3.9 ng / ml
Note: It is a coincidence that the cut-off values are the same.
