NLMC Standardisation of Pathology

DRAFT REPORT OF THE WORKSHOP ON STANDARDISATION OF REPORTING OF
PATHOLOGY DATA
A Workshop organised by the National Pathology Programme
Held at the Royal College of Pathologists on 12 February 2014
The workshop brought together representatives of the Royal College of Pathologists, Association
for Clinical Biochemistry, Institute for Biomedical Science, International Federation for Clinical
Chemistry, UK National External Quality Assessment Service, Wales External Quality Assurance
Scheme, Health and Social Care Information Centre, Care.Data, GP IT Committee, British In-Vitro
Diagnostics Association, National Institute for Biological Standards and Control, Medicines and
Healthcare Products Regulatory Agency, United Kingdom Accreditation Service, Research
Information and Intelligence, Department of Health, NHS England and the National Pathology
Programme, together with other academics working in this field.
The purpose of the workshop was to:
1.
2.
3.
4.
5.
To review data on the variation between methods and the potential to scale data to a
common reporting base.
To review the potential of large scale population data to determine reference ranges by
age, sex and potentially ethnicity.
To advise the National Pathology Programme on how to agree/progress work in this area.
To recognise the need for standardisation both for patients reviewing their data and the
benefits for the research community.
To review timings, milestones and resource in each area of work.
Although pathologists have developed a number of approaches to standardisation, there is much
to be done to ensure results from different sources convey the same clinical meaning and can be
combined to show patient progress. These approaches include the development of the National
Laboratory Medicine Catalogue (NLMC), revision of Pathology Bounded Code List (PBCL),
standardisation of units of measurement, creation of data combination indicators, histopathology
cancer report data sets and standardised microbiological investigations.
This report is divided into three sections:



The evidence of the need for greater standardisation
Roles and responsibilities in moving toward standardisation
Key issues raised and recommendations for further action.
The evidence of the need for greater standardisation:
Pseudonymised data from a three week collection of pathology reports sent to GPs were analysed
using statistical methods to exclude outliers to determine a national reference range for TSH and
fT4. This reference range differed significantly from those reported by some laboratories. Further
analysis of larger numbers will be able to determine variations by age, sex and methodological
differences. The latter could be used to harmonise results using different IVD manufacturers as the
multiple isoforms of TSH make true standardisation impossible.
The same data collection was used to determine if there was an age and sex variation in serum
albumin. With over a million results it was possible to show significant change with age and by sex.
1
The impact of these differences when applied to the adjustment of serum calcium for albumin
yielded clinically significant differences from those reported by laboratories. Further the use of age
and sex related reference ranges demonstrated much tighter reference ranges than those
currently used, thus offering the possibility of earlier detection of disease. The data also showed
marked differences in population mean serum albumin concentrations that would lead to
significant differences in adjusted serum calcium results. Such differences are very unlikely to be
due to population factors.
Review of UK NEQAS reports for PTH assays demonstrates consistent methodological (IVD)
differences when purified PTH is present in normal serum or serum from patients with CKD. In
addition for each method the same results are obtained for a given purified PTH addition indicating
the methods are not measuring PTH fragments significantly. Recovery as shown by the EQA data
indicates one method has double the recovery of the lowest although the differences in the
reference ranges do not reflect this. These methodological variations are reflected in clinical
interventions related to renal bone disease.
Measurement of serum creatinine by enzymic techniques overcomes many the problems
associated with the more commonly used cheaper Jaffe reaction methods; in particular the
clinically significant effects of acetoacetate and glucose. The differences in results by IVD
manufacturer are significant and have clinical implications when results are used in the calculation
of doses of cytotoxic drugs. Differences in results shown in EQA data can lead to a difference in
carboplatin dosage of up to 20%.
The work by pathologists and the renal community to gain a common approach to measuring eGFR used a factor to align creatinine methods but this was not used for routine creatinine
estimations. The end result is that some laboratory methods perform well for e-GFR on UK
NEQAS returns but this is not the case for serum creatinine itself.
Even where there is an international standard used by IVD manufacturers, there remain
methodological differences because methods measure different aspects of the substance in
question. Affinity chromatography, ion-exchange chromatography and immunoassay methods
deliver different results for glycated haemoglobin although using the same IFCC calibrant.
There is not a precise scientific measure of free thyroxine and hence all the laboratory trimmed
mean demonstrated in UK NEQAS returns is wide for a euthyroid patient pool, whereas individual
methods show a much tighter spread of results. Taking five commonly used methods the mean
values for each method show a range from 10/11 pmol/L to 15/16 pmol/L. However, the
consistency of these differences and the lack of any suitable reference standard open the
possibility of scaling results by IVD manufacturer to a common value and tighter clinical reference
interval.
WEQAS are working to the standards of Joint Committee for Traceability in Laboratory Medicine
(JCTLM) listed Reference Measurement Service, accredited to ISO 17025 & ISO 15195. This
allows comparison of EQA results with true values for analytes where this is appropriate. EQA
participant scoring is based on a hierarchy of target values: 1) reference method (if available for all
samples), 2) trimmed method mean (if n > 8) and 3) trimmed overall mean. These reference
targets enable comparison of results with the ‘True’ value. This enables any method/calibration
changes to be identified in real time. The risk of using laboratory trimmed means is that dominant
method groups may hide standardisation issues when reference target values are not used and
may even penalise methods closer to true value.
2
From the perspective of the research community the quality of laboratory data being used was
concerning, as a key requirement for quality good evidence based medicine is high quality data.
The potential clinical and safety risks are very concerning from a patient and public perspective
especially following the publication of the Francis Report with its emphasis on transparency. From
a DH perspective a conclusion with regards to these standardisation issues must be rapid.
Roles and responsibilities in moving toward standardisation:
The issues raised in the previous section are reflected in recommendations in the recently
published Pathology Quality Assurance Review (PQAR):


‘The continued development of the National Laboratory Medicine Catalogue (NLMC) to
ensure consistency of data and information across the NHS in England should remain a
priority. The professional bodies, the IVD manufacturers and others, should work towards
minimising the differences between analytical processes, requesting and reporting.’
Recommendation 4.66.
‘Laboratory processes should be harmonised so that patients can be confident about the
consistency of their test results, especially as they start to gain access to their personal
health records that may contain reports from different pathology services.’ Press Release.
The implementation of the recommendations of PQAR will inevitably require greater transparency
in relation to methodological differences and how to mitigate clinical risk arising from them.
Both, WEQAS and UK NEQAS are keen to continue working with IVD manufacturers to enhance
method performance and have already achieved considerable improvements in many areas.
EQA schemes will play a significant role in the standardisation/harmonisation processes through
identifying the degree of variation between methods/manufacturers, indicating analytes suitable for
standardisation/ harmonisation and monitoring the progress of standardisation/ harmonisation
initiatives. Analytes with metrological traceability should be standardised (scientific approach)
otherwise harmonisation (pragmatic approach) is indicated.
NIBSC plays an important role in international standardisation and approximately 50% of NIBSC’s
WHO International Standards have diagnostic applications. These standards were often never
intended to be used for diagnostic assays but were established as standards for the calibration of
biological assays and subsequently adopted as diagnostic calibrators e.g. for immunoassays as
nothing else was available. However, there are very different needs for immunoassay versus
bioassay including matrix considerations (sugars, protein, serum, plasma,) heterogeneity of the
reference material (subunits, fragments, viral genotypes, different antigens/antibodies), the purity
of the reference material (e.g. tissue extracts) and native versus recombinant materials. In
recognition of the diagnostic use of international standards, there is careful selection of starting
material for new and replacement standards, evaluation of commutability though WHO
collaborative studies or EQA schemes through inclusion of a range of clinical samples alongside
candidate reference materials, development of new methods to statistically evaluate commutability
and closer working with international organisations such as IFCC, EQA schemes and IVD
manufactures.
MHRA is responsible for regulating all medical devices including IVDs in the UK by ensuring they
work and are acceptably safe on behalf of DH. In addition its links with other organisations yields a
global role. Within the remit of MHRA is a safety, quality and performance role that comprises a
3
cycle of manufacturer reports (vigilance), user reports, EQA reports, field safety notices and signal
detection.
Issues remain of commutability of reference materials including PTH, TSH because of multiple
isoforms and other serum proteins.
Pathology professional organisations have an important educational role for pathology staff,
clinical users and the public. Inclusion of agreed steps in standardisation and harmonisation in
developing professional key performance indicators will provide significant leadership.
The office of the CMO will be an important conduit with DH and HSCIC. In addition, the Director
General, External Relations who has responsibility for clinical governance could be used to
support the uptake of standardisation.
The IFCC Harmonisation/Standardisation Project brings together international experts who are
currently reviewing twelve or more analytes. Standardisation requires metrological traceability and
where this is not possible, harmonisation is the best option.
Although these international developments are important, it is essential that the unique
opportunities offered by the NHS are used as a beacon for work on harmonisation and
standardisation.
Key issues raised and recommendations for further action:
General points
It is recognised that there is more than one approach to standardisation:
o
o
o
Standardisation of analytical techniques
Use of World Health Organisation (WHO) and similar reference materials
Mathematical alignment of methods
– Mean values of different methods from EQA
– Population mean values of different methods from GP reports
• Where there are no international reference materials
– To WHO/NIBSC reference based methods where used in IVDs
• With caveats concerning commutability
4
"An inevitable precondition for the establishing of traceable results to calibrators and control
materials is the specificity of the measurement procedures applied. Results of measurement
cannot be traceable when the procedure applied partially detects components which are not
consistent with the definition of the measurand." Lothar Siekman.
“If we have “Fact/Truth” on one side and “compromise” on the other we may have to make a
choice. What I want to ensure that when all the caveats and appendices have been
lost/deleted/forgotten that if we do go for the compromise then it is clearly stated that it is a
compromise and we do not re-badge compromise as Fact/Truth.” Finlay MacKenzie.
There was agreement that using data as a scientific basis for determining reference ranges
overcame the problems associated with consensus approaches such as breadth of participation,
insufficient information on age and sex differences and risk of setting a very wide reference range.
Methodological aspects:
NIBSC is keen to develop fit-for-purpose, commutable Reference Materials but this only has
benefit if assay methods measure the same thing with similar specificity. There is a need for clarity
on what is being measured and it is important to identify ‘quick wins’ as well as longer term goals.
In working with IVD manufacturers there is a need to recognise that the UK is only 3% of the
market, although there is a high regard for UK pathology and potential international influence.
There is a need to communicate UK work on standardisation and harmonisation to other countries
and join like-minded individuals and organisations in generating momentum.
CE marking is not a guarantee that the IVD is fit for all the clinical purposes that clinicians may
require. There is a need for pathology services to have a dialogue with clinical users to ensure the
analyses provided are appropriate for the desired clinical use.
Service user perspective:
Concerns were expressed in reference to GPs whose practices use more than one laboratory and
had assumed the data was suitable to monitor patient progress despite the different sources. This
indicated a need for local dialogue and education. Both GPs and patients need to be made aware
of the ‘fuzziness’ around reference ranges.
It is essential that contracts for systems involved in the transmission and storage of reports
incorporate reflex tests and clinical interpretive comments with the requested items. There is a
need to create an Atlas of Variation of reference ranges and methods by laboratory. NPP will liaise
with UK NEQAS to determine how to take this forward as soon as possible. This will be important
for both patients and researchers.
There needs to be wider recognition of the targets of total allowable error relative to biological
variation by both, pathology service providers and IVD manufacturers.
There is a need to determine how to inform the public of this work especially in the light of the
patient views expressed to PQAR and the need to maintain patient trust. The need was to inform
without being alarmist and continue with standardisation and harmonisation as rapidly as possible.
Patients need to be re-assured that the variation in the vast majority of tests poses no clinical risk
and that the work on standardisation and harmonisation should focus on areas of potential clinical
risk. These areas include serum creatinine, serum albumin and calcium, tumour markers where
there are numerical cut-off values, parathyroid hormone and units where variation poses clinical
5
risk e.g. ethanol. The limitations and variation in current creatinine methods should be minimised
by the universal uptake of enzymic creatinine assays. It may be useful to link this with the change
to CKD-EPI for detection of CKD and this recommendation needs to be incorporated into the
Pathology Commissioning Toolkit. In addition, it is important that expectations of creatinine assay
performance to meet clinical need is formally raised through MHRA.
Exploiting ‘Big Data’:
Large databases of primary care records covering many millions of patients are selecting units of
measurement for analytes. Whilst the principle of a single unit of measurement for any given
analyte is necessary, the appropriate unit should be determined by the pathology community and
firm action in this area will be applauded by these system suppliers. The same principle is
supported by LIMS and middleware system suppliers.
The proposed continuous download of GP reports to HSCIC may permit analysis of greater
numbers and for thyroid data may permit links with other lab data such as TPO concentrations and
thyroid related drugs.
The power if big data needs to be exploited not just for pathologists and researchers but also IVD
manufacturers. Big data has a phase of discovery followed by a phase of utilisation as shown by
the findings in relation to serum albumin measurement. Discovery of more appropriate reference
ranges for analytes protects those providing and using pathology results from claims of negligence
concerning patient misclassification.
There is a need to progress the implementation of NLMC and ensure smooth transition from PBCL
to NLMC as highlighted in PQAR.
The implementation of any decisions re standardisation and harmonisation requires a coordinated
process involving LIMS, middleware and receiving systems such as GP records systems. Making
changes mandatory in GP systems will involve GPSoC and there are currently no equivalent
approaches for LIMS or middleware.
Concluding thoughts:
The use of nationally derived reference ranges and methodological mathematical alignment needs
to be based on sound scientific methods. There also needs to be an authority for making changes
that does not absolve IVD manufacturers from their legal responsibilities. Professional standards
via KPIs monitored and enforced via CPA/UKAS provide an approach to uptake. In addition, the
requirements in the commissioning of pathology services will provide an important lever – this
should be included in the next version of the Pathology Commissioning Toolkit.
There is an appetite for progressing standardisation of methods and reference materials with
harmonisation of data where appropriate. This will require a co-ordinated initiative involving
pathology professional organisations, clinical service users, NIBSC, MHRA and BIVDA with
overarching support from NHE England.
The National Pathology Programme agreed to review the information and views from this
workshop and develop a work programme including workshops in collaboration with the Health
and Social Care Digital Service.
GIFFORD BATSTONE
NATIONAL PATHOLOGY PROGRAMME
04.03.14
6

Download Report