Performance Evaluation of Real Time Polymerase Reaction Assay (SCODA) for Detection of SARS-CoV-2 ()
1. Introduction
In the past few years, the world has been faced with a great challenge caused by COVID-19 pandemic which has had devastating effects on all other sectors of the economies of nations: developed, developing or low- and medium-income countries. The year 2020 will go down memory lane as the year the globe stood still for SARS-CoV-2 pandemic. The lack of movement across countries also affected movement of goods and services including movement of diagnostic test kits to many low-and-mid-income countries of the world that could not produce the diagnostic kits and other reagents needed to manage the infection in-country. This limitation affected testing and hence the knowledge of the true burden of the infection in most countries. The importance of testing cannot be underestimated in the battle to effectively manage the spread of COVID-19 in any society because it is what gives a clear picture of the burden of the pandemic and how it is spreading. Without testing, one cannot tell who is infected or not; that means tracing or treatment cannot begin let alone molecular studies into the possible variations circulating in the country. In addition, it becomes difficult to ascertain the true burden of the infection in any society if testing data available is low relative to the population.
Testing had remained a problem in many countries especially in Africa because of the high cost and expertise required for conducting a polymerase chain reaction (PCR) test which is the gold standard for diagnosis. This challenge is major because the kits needed for PCR testing are expensive and because they are not produced in many countries, there is a huge global demand for limited products. Hence, to reduce this challenge, there is a need for development and production of diagnostic kits in many countries in the world for SARS-CoV-2 and subsequently other pathogens of great challenge to our health sector. Many diagnostic techniques such as antibody and antigen rapid diagnostic testing, enzyme-linked immunosorbent assay (ELISA), Isothermal amplification are available and are being used for diverse studies on pathogen detection and characterization, however, assays that utilize real-time PCR technology have shown higher sensitivity and specificity which have extended the scope of diagnosis [1]. Real-time PCR technique has currently become the mainstream of molecular diagnostic tools.
SARS-CoV-2 Detection Assay (SCODA) makes use of RT-qPCR method for the detection of SARS-CoV-2 in clinical samples. It is an in-country developed kit that targets the ORF1ab and N genes of SARS-CoV-2 in a multiplex reaction. It is a well-developed and validated kit in-country. It also has an internal control which acts as a housekeeping gene for the reaction.
This study was aimed at evaluating the performance characteristics of SCODA in comparison to commercially available and approved COVID-19 testing kits.
2. Methods
SCODA was validated at Centre for Human Virology and Genomics (CHVG), Nigerian Institute of Medical Research using Foundation for Innovative New Diagnostics (FIND) protocol adapted from WHO assay validation protocol [2]. CHVG was one of the laboratory networks approved by NCDC to carry out COVID-19 testing in Nigeria. Beyond testing, the laboratory had over the years being the foremost research institute in the country carrying out cutting edge researches into diseases of public health concerns. It is ISO 15189 accredited and WHO pre-qualification laboratory for diagnostic kits. In addition to the validation from CHVG, the assay has been validated by three other external laboratories.
2.1. Study Design
This was a retrospective study which made use of stored samples from previous tests.
2.2. Sample Collection
One hundred and fifty (150) were used for this analysis: fifty (50) PCR positive and one hundred (100) negative samples for SARS-CoV-2. Clinical samples were collected from individuals who came to the sample collection drive-through put in place by the Nigerian Institute of Medical Research in collaboration with some non-governmental organizations to boost national testing capacity for COVID-19. Both nasopharyngeal and oropharyngeal swabs were used for this study. The samples were earlier tested on LifeRiver SARS-CoV-2 multiplex RT-PCR kit.
RNA Extraction
Viral RNA were extracted from the samples with Qiagen RNA mini kit (QIAGEN, Hilden, Germany) according to the manufacturer’s instructions.
2.3. Preparation of Standards
The assay was validated using ATCC (VR-3276SD) and WHO SARS-CoV-2 panels. In addition, lyophilized synthesized plasmids were constituted using TE buffer and the solutions quantified using Qubit 4 Fluorimeter double-stranded DNA assay (Thermo Fischer Scientific). The number of copies of the virus in the quantified plasmid was calculated using Avogadro’s formula: Number of copies/µl = [weight of plasmid (ng)/length of plasmid × 109 × 660] × 6.022 × 1023 [3] [4].
2.4. RT-qPCR Protocol
A 25 µl reaction volume contained 5 µl of RNA and 20 ul of PCR mix was plated on a 96-well reaction plate and ran on Quant Studio 5 machine (ThermoFischer scientific) using optimized PCR condition of 52˚C for 10 minutes, 95˚C for 10 seconds, 95˚C for 5 seconds (40 cycles) and 56˚C for 30 seconds (40 cycles). The ORF1ab target on SCODA runs on 5’-HEX…BHQ1-3’ while N-gene target and IC run on 5’-FAM…BHQ1-3’ and 5’-ROX…BHQ2-3’ channels respectively.
2.5. Limit of Detection
This defines the lowest quantity of a virus an assay can detect in a sample. To determine the LoD95 of SCODA, 10 different concentrations (1, 5, 10, 50, 100, 500, 1000, 5000, 10,000, 100,000 RNA copies/µl) were ran in 10 replicate each. The LoD95 was then calculated from the resultant Ct values using Probit Regression Analysis [5].
2.6. Sensitivity and Specificity
To test for analytical sensitivity, 24 positive samples (from the positive controls used for LoD) were analyzed. On the other hand, the specificity of the assay, replicates of RNase free water and Viral Transport Medium (VTM) without any sample inside were analyzed. Clinical sensitivity and specificity of SCODA were determined by analyzing fifty (50) positive samples and one hundred (100) negative samples against a commercially available kit (Sensitivity (%) = {number of true positives/[number of true positives + false negatives]} × 100. Specificity (%) = {number of true negatives/[number of true negatives + false positives]} × 100. Positive predictive value (%) = {no. of true positive/[no. of true positive + no. of false positive]} × 100. Negative predictive value (%) = {no. of true negative/[no. of true negative + no. of false negative]} × 100 [6].
2.7. Linearity and Limit of Quantification
This is the ability of an assay to return values that are proportional to the concentrations of the target pathogen in the sample. To determine this for SCODA, 6-tenfold dilutions (101 - 106) of the standard controls were run on both ORF1ab and N-gene and the data obtained analyzed using regression equation: y = ax + b, where a is slope and b is intercept.
2.8. Repeatability and Reproducibility
Repeatability was evaluated by calculating the intra-assay (within runs) while reproducibility was evaluated by calculating the inter-assay (within days) of the experiment. Intra-assay is the measure of the ability of an assay to return values that are proportional to the concentrations of the target pathogen in the sample when run at different replicates on the same plate at the same time. To determine this on SCODA, 6-tenfold dilutions (101 - 106) of the standards were run on both ORF1ab and N-gene.
Also, Inter-assay is the measure of the variance in the concentration between runs of sample replicates on different plates and days. To determine this, four different sample concentrations were tested across 5 days. The coefficient of variation was then calculated for both targets.
3. Results
The standards assays when constituted and quantified, yielded 16.0 ng/µl and 21.2 ng/µl for ORF1ab and N-gene respectively. Serial dilution (1:10) was carried out from 100 to 106 cp/µl using the PCR protocol above. The result is as shown in the graphs in Figures 1-3. The ORF1ab target amplified all the dilution except 100 cp/µl while N-gene target amplified all the dilutions.
![]()
Figure 1. Amplification plot of the standards for ORF1ab-gene of SARS-CoV-2 (100 to 106 cp/µl).
![]()
Figure 2. Amplification plot of the standards for N-gene of SARS-CoV-2 (100 - 106 cp/µl).
![]()
Figure 3. Amplification curves for ORF1ab, N-gene and IC.
3.1. Linearity
The ORF1ab target showed a linearity of 98.2% while N-gene target showed a linearity of 99.8% (Figure 4 & Figure 5).
3.2. Intra Assay Precision
The Coefficient of variation percentage for both N-gene and ORF1ab are ≤3% as seen in Table 1.
3.3. Inter Assay Precision
The Coefficient of variation percentage for both N-gene and ORF1ab are ≤3% (Table 2).
3.4. Limit of Detection
The limit of detection at 95% confidence interval for both ORF1ab and N-gene targets are 74.04 (95% CI: 25 - 1000) cp/µl and 1.119 (95% CI: 1 - 1) cp/µl respectively (Figure 6 & Figure 7).
![]()
Figure 4. Graph of linearity for ORF1ab.
![]()
Table 1. Intra assay analysis on both N-gene and ORF1ab targets.
![]()
Table 2. Intra assay analysis on both N-gene and ORF1ab targets.
![]()
Figure 6. Limit of Detection analysis for ORF1ab using Probit analysis.
![]()
Figure 7. Limit of Detection analysis for N-gene using Probit analysis.
3.5. Sensitivity and Specificity
The sensitivity and specificity of the assay were both 100% in comparison to the comparator used.
4. Discussion
For effective management of any outbreak, availability of highly sensitive and accurate diagnostic tools is very important. Several diagnostics platforms are available nowadays, however, molecular methods of diagnosis have higher sensitivity in detecting early infection. At the advent of Sars-CoV-2 outbreak, WHO rolled out an assay developed by Charite Research Institute in Germany in 2020 [7] which was used in several countries before more testing kits began to come out. Although very sensitive, the protocol of usage was a bit tedious and the result turn-around-time was long especially for an outbreak. There is one thing for diagnostic kits to be available for use and another thing for it to be available in some specific places at the right time. Movement of goods from the developed countries to low-income countries like Nigeria is usually cumbersome because the several protocols that are in between. COVID-19 pandemic showed clearly the gaps in research and product development between the developed and developing countries. Ability to develop assays that are highly specific to the strains circulating in a particular region helps to minimize giving out false negative results. It also provides easy and cheaper access to kits within such region. The development of SCODA was to meet an in-country need in a pandemic which at a time made it impossible for inter country movement. Before SCODA was developed, more than seven different COVID-19 diagnostic kits had been used at CHVG. This provided a lot of room to validate SCODA against many commercially available kits. The sensitivity and specificity of 100% relative to the comparator and other panels used in the validation process implies that the kit can detect all positive samples as positive and all negative as negative. This cuts off the possibility of false positive or false negative results. In comparison with other kits such as Tib Molbiol kit (sensitivity 100%, specificity – 100%); BGI (sensitivity – 100%; specificity – 100%); DAAN gene (sensitivity – 100%, specificity – 93.3%) and Liferiver (sensitivity – 100%, specificity – 93.3%) [2], the specificity of SCODA was better than Liferiver and DAAN gene kits. From the studies by Onwuamah and colleagues, it is clear that SCODA can compete well with commercially approved kits COVID-19 RT-qPCR test kits. The sensitivity and specificity of SCODA as recorded in this study exceeds the criteria set by WHO for an acceptable sensitivity of ≥80% and specificity of ≥90%; desirable sensitivity of ≥90% and specificity of ≥99% for the validity of RT-PCR result [2] [8].
One of the quality control criteria for passing a test is the nature of its amplification; oftentimes a sigmoidal amplification is acceptable; Figures 1-3 in this study shows that the amplifications generated with SCODA are sigmoidal in shape. This is a pass for SCODA.
The in-silico assessment of the specificity of this assay showed no cross reaction with any related pathogens at the regions of ORF1ab and N-gene where the primers and probes were designed from. The wet laboratory assessment in comparison with other approved kits showed no cross reaction. The limit of detections of (LoD) 74.04 copies/µl and 1.119 copies/µl (Figure 6 & Figure 7) for ORF1ab and N-gene targets respectively indicates its capacity to detect an infection with a low viral load of SARS-CoV-2 when compared with other commercial kits [2] [7].
One of the major concerns of an assay is its precision. This is the ability of an assay to replicate similar result when a particular concentration is run consistently on the same day or different days either by the same person or not. This is measured by the inter assay and intra assay analysis of the assay. This study showed that there is consistency in the results from the precision analysis as seen in Table 1 & Table 2 above with %CV < 3 for both targets. This means that SCODA can produce results that can be repeated and reproduced.
Limitation
The ATCC panels that were purchased took a long time to arrive. In addition to late arrival, the synthesized panel did not cover the part of the ORF1ab gene where the target was designed from. This limited us to validating the assay on only the N-gene target of SCODA.
5. Conclusion
SCODA can be used as an alternative kit to other commercial detection kits for SARS-CoV-2. It is easily assessable and cheaper. The advantage of having an in-country developed assay helps for prompt diagnosis of SARS-CoV-2.
Acknowledgements
I want to acknowledge all my colleagues at Centre for Human Virology and Genomics, Nigerian Institute of Medical Research (NIMR), Yaba, Lagos State, Nigeria.
Authors’ Contribution
Joseph O. Shaibu: assay design and conceptualization, optimization and validation; Chika K. Onwuamah: Validation; Azuka P. Okwuraiwe: validation and Laboratory management; Rahaman Ahmed: validation; Akeeb B.O Oyefolu: supervision; Salako B.L: supervision and funding; Rosemary A. A: supervision and conceptualization.