Are peer comparison statistics obtained from Internal Quality Control and External Quality Assurance programs comparable?- An analytical study

Introduction

Quality control process in a clinical laboratory refers to all the procedures that are designed to monitor the routine performance of the testing processes in order to detect any possible errors and correct them before the test results are reported.^1-3 The two types of quality control used in a laboratory are Internal quality control (IQC) and External quality assessment (EQA). The main emphasis of internal quality control program is to monitor the precision of analytical methods while external quality assurance is able to detect accuracy and precision in the performance of analyte.^4-6 All laboratories have IQC program and in addition participate in Proficiency testing (PT) program, such as EQA.^7,8 With the introduction of newer software and programs, IQC results can be compared with peers and is thus able to detect both extent of agreement with peers and precision.

The performance of analytes in EQA is expressed as Z score.⁹ IQC data are analyzed mainly using statistical parameters such as standard deviation and coefficient of variation which is a reflection of precision. A statistic similar to Z score in IQC is SDI (Standard Deviation Index) which is helpful to evaluate agreement with peers by comparing a laboratory’s results to its consensus group. Laboratories can obtain this using a software which has Inter Laboratory Peer Comparison Program that compares the laboratory mean with consensus group value as the target value.⁷ There is limited literature available regarding the comparison of these two scores in the evaluation of analyte performance between the two quality control programs.

This study was undertaken to compare the monthly SDI of common analytes such as glucose and urea obtained from IQC data with the corresponding monthly Z score of EQA program. If the study shows concurrence, then it can be suggested to have an EQA program only where IQC with peer comparison is not available, thereby reducing the overall cost of quality control.

Materials and Methods

This was a retrospective analytical study conducted in the department of Biochemistry. Since it was a pilot study, one year data from Jan 2018 to Jan 2019 was used to compare the two accuracy scores. The accuracy scores from consecutive monthly IQC reports and the corresponding monthly EQA reports of glucose and urea were included in the study. Data during any faulty runs were excluded from the study. This resulted in a total sample size of 24 from IQC data (12 at level 1 QC and 12 at level 2 QC) and 12 corresponding EQA scores for a year. The Institutional ethics committee approval was obtained for the study.

Glucose and urea were selected as they are one of the most common chemistry analytes for which both IQC and EQA data are available. Glucose was estimated using hexokinase method and urea by urease method on fully automated Siemens Dimension ExL Chemistry auto analyzer.

The Z score and SDI were the two scores used for peer comparison in this study. The Z scores of glucose and urea were obtained from monthly EQA report. The corresponding SDI for glucose and urea was calculated from the monthly IQC peer comparison report using the unity real time software. The SDI calculated using level 1 and level 2 IQC data for glucose and urea was then compared with the Z score of EQA reports for the corresponding months.

Unity real time software was procured from Biorad EQA provider. In addition to monitoring day to day performance of IQC, this software also allows the user to participate in interlaboratory comparison of IQC data. Thus, in addition to precision, extent of agreement with peers can also be monitored.

The lab participated in Biorad monthly EQA procedure for chemistry analysis.

Statistical analysis

SPSS 16 was used for statistical analysis. The quantitative variables were expressed as mean and percentages. Data has been plotted to analyze the trend between the two scores. The intraclass correlation (Bland Altmann analysis) for agreement has been used for comparison between the two scores. Values less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability and agreement between the scores.

Results

Figures1 to 4 represent the trend in the monthly SDI and Z score for glucose and urea at level 1 and level 2 IQC, respectively. The graphs show similar trend between the two scores across most of the months 10 out 12 months (83.3%).

Table 1 shows the intra class correlation between the two scores for both the parameters. The intra class correlation between SDI and Z score for both the parameters was less than 0.5 indicating a poor correlation (p value more than 0.05)

Discussion

Internal Quality Control samples are used to monitor and evaluate the day to day analytical performance of parameters in a laboratory.^1,2 The most common performance statistics obtained from IQC is standard deviation and coefficient of variation which are indicators of precision. EQA however can detect both precision and agreement with peer data. With the introduction of newer software and programs, IQC results can be compared with peers and is thus able to detect both precision and extent of agreement with peers.⁷ Several statistics can be obtained in the evaluation of quality of analyte performance using IQC and EQA.^10,11 This study was undertaken to compare the performance of the analyte using the Z score available from EQA reports and the calculated SDI from IQC peer comparison data.

Results of our study from the graphical representation of trends between the scores show that there was similar trend observed between the two scores across most of the months for both parameters across both the IQC levels. This shows that the findings in EQAS tends to be reflective of the trend observed in IQC. Hence the analysis of laboratory monthly review of IQC can indicate the expected trend in EQAS since both are evaluated using peer comparison although the number of peers may vary between the two programs.

Although the two scores were calculated using similar variables, there was no statistically significant agreement between the absolute values of the two scores. The intraclass correlation coefficient was not significant in the analysis for agreement. This shows that although the trend was similar across most of the months, there was statistically significant difference between the absolute values of the two scores. This could be due to difference in the number of peers participating in the program and also due to difference in the total number of individual laboratory measurements involved in calculating the two scores in EQA and IQC.

Conclusion

This study shows that although the performance of an analyte can be obtained from both IQC and EQA monthly peer comparison reports, it may not be appropriate to extrapolate the analyte performance observed in IQC to EQA in terms of actual value of the scores as there is no agreement between the two scores in terms of absolute values.

Limitation

This study was conducted using one year data. Analysis of trends using larger sample size from two to three year IQAC and EQA data would give more accurate comparison.

Criteria for inclusion of the authors

• Dr Jaya Kumari S: Concept, design, analysis, interpretation and writing up of manuscript
• Pawan: Data collection, statistical analysis, interpretation and write up

Disclosure

This manuscript has been read and approved by all the authors, the requirements for authorship of each author has been met, and that each author believes that the manuscript represents honest work.

Conflict of Interest

Nil