NADAL Antigen test Evaluation

 

Evaluation of the performance of the COVID-19 antigen rapid tests in scientific studies

 

Background

Even before COVID-19 antigen rapid antigen tests (RATs) get their approval, they are thoroughly evaluated in a number of clinical settings as part of a performance evaluation for in-vitro diagnostic devices. The results of these studies serve as the basis of the performance section in the instructions for use. To further evaluate the clinical sensitivity and specificity of a test, further extensive independent studies are conducted by hospitals, laboratories and scientific institutes. In all of these studies, clinical sample materials taken from the respiratory tract are used. Using RAT and RT-PCR in parallel, these samples are studied for the presence of the target analyte (N-protein and specific RNA).

 

Differences between studies by FIND and PEI

In cooperation with global partners, the Geneva-based Foundation for Innovative New Diagnostics (FIND) looks at the analytical and clinical performance of rapid SARS-CoV-2 antigen tests on an ongoing basis. 20 COVID-19 RATs have been investigated thus far in more than 40 completed studies with an extensive study cohort (n>200). Freshly obtained nasopharyngeal, naso-/oropharyngeal and nasal swab samples were evaluated prospectively, with sampling carried out exactly as described in the instructions for use. Additionally, the limit of detection (LOD) was evaluated and compared to the manufacturer's specification with the help of a dilution series from a virus culture of SARS-CoV-2 [1].

 

The Paul-Ehrlich-Institute (PEI) carried out a comparative study for the Federal Ministry of Health of billable ‘rapid antigen tests (so-called point of care tests (PoC)) for direct pathogen detection of the coronavirus SARS-CoV-2’ (BfArM list, [2]). Sample materials consisted of 50 pools of up to 10 naso- and oropharyngeal swabs each. These were sent to the PEI either as dry swabs or in viral transport medium. They were then extracted into phosphate-buffered saline (PBS), diluted and frozen. Thawed pools were stored refrigerated for up to 5 days. To date, over 100 different COVID-19 RATs were examined retrospectively. For this procedure, 50 μL of the PBS sample solution from each pool was absorbed using the corresponding sampling tool for the rapid test and then eluted in the extraction buffer of the RAT [3, 4].

 

Retrospective laboratory studies (‘PEI‘) compared a multitude of COVID-19 RATs under identical circumstances, which leads to the following disadvantages:

 

    • Sample material do not comply with the specifications given for the RAT
      Our laboratory studies found the PBS solution used for the pooling and dilution of the swabs to be incompatible with the NADAL® COVID-19 Ag Test. When fresh samples are collected – as it is the case in the FIND study - there is no extraction using PBS and no dilution.

 

    • Incorrect use of the sampling tool
      The swab provided with the NADAL® COVID-19 Ag Test is comprised of hydrophobic material, which is optimal for the collection of swab samples from the respiratory tract. Due to the material properties of the swab, the absorption of aqueous (hydrophilic) solutions is highly limited. This can mean that not enough virus-containing sample material is transferred to the test. In the FIND study, the swab was used for its intended purpose.

 

    • Sample degradation
      The use of virus transport media (VTM, with and without chaotropic salts), dilution with PBS or several days of storage are no problem for RNA followed by RT-PCR. This is not necessarily the case to the analysis of proteins. The adsorption to the walls of the vessel, denaturation, aggregation and digestion of proteins can all lead to a strong reduction in detectable target analytes. In fresh samples, such as those used in the prospective study by FIND, the target analytes are optimal for a RAT.

 

  • Diluted sample material
    If the use of transport media cannot be avoided, our instructions for use advise mixing a sample volume of 350 μL with the extraction buffer provided with the NADAL® COVID-19 Ag Test. The PEI evaluation used just 50 μL (14.3%), which is much too little. The FIND study worked with a normal, sufficient amount of sample material.

 

Conclusion

The test procedure of the PEI evaluation deviated significantly from the manufacturer's instructions for use of the NADAL® COVID-19 Ag test in many respects. ‘Off-label-use’ like this is, in most cases, associated with significantly deteriorated test performance.

 

This can also be seen in the -sometimes significant- discrepancies in the determined sensitivities of numerous COVID-19 RATs in FIND studies in comparison to those by PEI, as shown in the following table:

 

Tabelle Gegenüberstellung der Ergebnisse von FIND und PEI

 

The clinical studies for the NADAL® COVID-19 Ag Test conducted by us as the manufacturer as part of the performance evaluation for in-vitro diagnostic devices were performed using freshly collected naso-/ oropharyngeal or nasal swabs. The results showed a sensitivity of 97.6% and 94.12%, respectively, in the Ct-range 20-30. When stored or diluted samples (e.g. viral transport media) were used, there is a clear reduction in the performance of the rapid test.

 

In accordance with the instructions for use provided with the test, we recommend testing swab samples as soon as possible after collection in order to ensure the best possible test performance. This is especially confirmed by the FIND study. The NADAL® COVID-19 Ag Test demonstrates a clinical sensitivity of 92.4% at Ct ≤ 33 (see FIND study) and 97.8% at Ct ≤ 25 (see table). In the series of experiments using virus culture dilutions, 2.5 x 10² pfu/ml was still reliably detected.

 

By using fresh samples, as well as simply following the instructions for use, the NADAL® COVID-19 Ag Test can perform as expected and demonstrate that it is a qualitative, high grade tool for anyone looking to diagnose a COVID-19 infection.

 

Thomas Zander, CEO, Biologist
Tobias Roth, Biochemist, Specialist in COVID-19 diagnostics

 

 

 

Sources:

[1] https://www.finddx.org/sarscov2-eval-antigen/.

 

[2] https://antigentest.bfarm.de/ords/f?p=110:100:15260540382923:::::&tz=1:00.

 

[3] Scheiblauer Heinrich, Filomena Angela, Nitsche Andreas, Puyskens Andreas, Corman Victor M, Drosten Christian, Zwirglmaier Karin, Lange Constanze, Emmerich Petra, Müller Michael, Knauer Olivia, Nübling C Micha. Comparative sensitivity evaluation for 122 CE-marked rapid diagnostic tests for SARS-CoV-2 antigen, Germany, September 2020 to April 2021. Euro Surveill. 2021;26(44):pii=2100441. https://doi.org/10.2807/1560-7917.ES.2021.26.44.2100441

 

[4] Puyskens Andreas, Krause Eva, Michel Janine, Nübling C Micha, Scheiblauer Heinrich, Bourquain Daniel, Grossegesse Marica, Valusenko Roman, Corman Victor M, Drosten Christian, Zwirglmaier Katrin, Wölfel Roman, Lange Constanze, Kramer Jan, Friesen Johannes, Ignatius Ralf, Müller Michael, Schmidt-Chanasit Jonas, Emmerich Petra, Schaade Lars, Nitsche Andreas. Establishment of a specimen panel for the decentralised technical evaluation of the sensitivity of 31 rapid diagnostic tests for SARS-CoV-2 antigen, Germany, September 2020 to April 2021. Euro Surveill. 2021;26(44):pii=2100442. https://doi.org/10.2807/1560-7917.ES.2021.26.44.2100442