Fluxion Biosciences
A newly published paper coauthored by researchers at Fluxion and Hospices Civils de Lyon Cancer Institute tested the sensitivity and accuracy of Fluxion's ERASE-Seq liquid biopsy technology against a reference test.
In this blog, we provide some background on the paper and the approach we used to benchmark our ERASE-Seq technology in a 356-sample lung cancer clinical study. The paper, "Sensitivity, specificity, and accuracy of a liquid biopsy approach using molecular amplification pools," was published on May 24, 2021 in Nature Scientific Reports.
Shaking up Cancer Diagnostics with Liquid Biopsies
Liquid biopsies are revolutionizing cancer diagnosis and management, especially for treatment selection in late-stage cancers. However, numerous studies have shown that test results vary widely, especially when the allele frequency (abbreviated AF, the relative concentration of mutant DNA to total DNA) drops below 1%.
And it's easy to fall into this range; the vast majority of cell-free DNA (cfDNA) collected in a blood sample comes from normal, healthy cells. Only a small percentage, and often less than 0.1%, comes from the tumor-derived DNA.
Since we can't separate out the tumor DNA from the healthy DNA, we're left with the challenge of developing analytical approaches that allow accurate calling of DNA mutations at these very low levels. And that's not easy - next-generation sequencers, which are commonly used for this analysis, have error profiles in the 1% range when run using standard protocols.
The Strengths and Weaknesses of Next Generation Sequencing
Researchers have pursued improvements in next generation sequencing (NGS) sensitivity as well as development of alternate analytical methods that have an inherently lower error profile. Improvements in NGS sensitivity have typically involved chemistry-based approaches where small molecular "tags" are used as barcodes on the starting DNA. Then errors that occur through sample library preparation and sequencing can be identified by tracking the barcodes.
Of course the approach isn't perfect; the barcodes don't bind with perfect efficiency, leaving a fair percentage of the DNA untagged, and errors that occur in initial tagging can lead to false positives and negatives. However, the "molecular barcode" or "unique molecular ID (UMI)" approach does offer improvements over standard sequencing, but there is significant room for improvement.
To address the limitations of UMIs, Fluxion developed ERASE-Seq, which stands for Elimination of Recurring Artifacts and Stochastic Errors. The approach uses inter-sample statistical analysis to develop a confidence score at each position in the panel, allowing clear identification of true variants from "noise", and increasing sensitivity while reducing false positives in the critical region between <0.1% and 1% AF.
Figure illustrating how NGS can provide superior sensitivity and specificity with molecular amplification pools (MAPs). (a) The MAP approach divides the cfDNA into two or more molecular pools. The pools individually undergo target amplification, then are indexed and sequenced. (b) Variant data from the two or more pools are compared to a background error profile to eliminate both recurring and random errors.
ddPCR vs. ERASE-Seq: Who Comes Out on Top?
The most common alternative analytical technique is droplet digital PCR. In this approach, the sample is dispersed as tens of thousands to millions of tiny droplets within an oil matrix. The droplets are sized so that very few DNA fragments are in each droplet. Individual droplets can then be tested for a specific mutation.
Since the potential prevalence of each mutation is very low, many thousands of droplets must be consumed per mutation. This has the effect of reducing the number of mutations that can be tested in each sample, since the number DNA fragments and droplets is finite.
Typically the number of mutations that can be tested with ddPCR ranges to a maximum of 10 - 30. Not a bad degree of multiplexing, but when compared to NGS, which can test for thousands of variants in a single sample, it doesn't come close.
However, ddPCR has been established as the gold standard when it comes to sensitivity and specificity of the mutations it can detect from cfDNA. Our ultimate goal for ERASE-Seq liquid biopsies is to combine the sensitivity of ddPCR with the multiplexing capability of NGS.
Testing the Accuracy of Liquid Biopsies
Establishing the sensitivity and accuracy of a liquid biopsy is a challenging task, since we don't have a great way to determine "truth" from a clinical sample. Tumor biopsies provide one option, but may not be representative of the mutations in blood. Most studies have shown a concordance of 60-70%, and that's just not good enough to establish performance metrics.
Analytical tests are usually developed as the first step in this validation process, and are also used to optimize and "tune" assay performance. An analytical test is one that uses a known set of analytes; we know going into the experiment what the answer should be, and can then see how well the test does (and dial in improvements). Various samples can be created to increase complexity, and these tests can be used to compare performance against competitive approaches for benchmarking.
But these types of "contrived" samples, as valuable as they are, can never substitute for the randomness and biological complexity of real patient samples. The best approach is to find a benchmark "orthogonal" test that is statistically independent from the subject test and can be used as the reference.
In the case of our lung cancer study, we chose to use the previously-mentioned ddPCR test as our benchmark. This is advantageous because it's a blood test, and therefore both test approaches are getting an identical sample. Concordance testing against tissue biopsies, while valuable, suffers from a number of limitations, including the spatial heterogeneity of the tumor itself (the tissue sample taken may be different from other parts of the tumor), and variability in DNA shedding from the tumor.
Why Compare ddPCR and NGS?
As mentioned before, the downside to ddPCR is that the number of mutations that can be measured in a single test is relatively small. On the other hand, NGS-based tests like ERASE-Seq can assay thousands of mutations in a single test. And in fact this is the real goal - to develop an NGS test that has the same sensitivity and specificity of a ddPCR assay.
The relatively limited multiplexing of ddPCR tests does present limitations in a concordance test with NGS, since we can only directly compare the few mutations that are common between the two tests. However, this study focused on lung cancer, which has several high-prevalence EGFR mutations that show up in a sufficient percentage of patients, providing a large pool of both positive and negative results in this 356-sample study.
Flow chart representing the types of tests conducted as part of our 356-sample study.
Concordance Study Results
Patients participating in the study were enrolled at initial diagnosis and at progression. Sufficient blood (30mL) was collected to allow both the ERASE-Seq NGS test and the ddPCR test to be run by splitting the sample.
Concordance was measured based on the 3 most commonly tested EGFR variants for lung cancer: L858R, exon 19 deletion, and T790M. The first 2 are sensitizing mutations, while T790M is a resistance mutation.
The concordance results in the paper are summarized as follows: "The overall concordance across 1016 ddPCR tests was 98.8%, with a sensitivity of 98.5% and a per-sample specificity of 98.8%. For sensitizing mutations DelEx19 and L858R, 3 discordant calls made by ERASE-Seq were supported by tissue data."
Sensitivity and specificity of sequencing compared to orthogonal ddPCR testing in the blood for three clinically-actionable alterations (exon 19 deletions, L8585R, and T790M mutations).
Key Takeaways From the Paper
- The ERASE-Seq approach demonstrated excellent sensitivity and specificity compared to ddPCR: 98.5% sensitivity and 98.9% specificity.
- In 23% of cases, actionable mutations were detected by the ERASE-Seq NGS test and not by ddPCR because the mutations were not available in the ddPCR test due to multiplexing limitations.
- The researchers developed an approach to identify CHIP (clonal hematopoiesis of indeterminate potential) variants, which are a potential source of error in certain patient populations.
- This study demonstrated that ERASE-Seq meets its intended aim: to deliver the performance of ddPCR with the target breadth of NGS.
For more information, watch our recorded webinar on this study:
We wish to thank our collaborators at Hospices Civils de Lyon Cancer Institute for their work on this project!