Targeted Mini-Validations of Performance Metrics as a Screening Tool

Introduction

As DNA technologies continue to evolve, forensic laboratories are tasked with considering the adoption and implementation of a variety of new, optional methodologies. Prior to implementation, forensic laboratories must perform a full validation to comply with accreditation standards (1, 2). Validation studies are costly, and a significant amount of time and resources are invested to validate and implement a new method into the workflow (3). Since validation is an investment, it is useful to assess potential benefits before committing to a validation effort. This study details a targeted approach to assess performance metrics in order to make an informed decision without an expensive investment in a new system. The targeted approach allows the laboratory to focus on metrics deemed especially important. The data presented in this study detail an example of this approach, specifically assessing key performance features of the PowerPlex® Fusion (5C) versus PowerPlex® Fusion 6C (6C) STR amplification kits from Promega at the Virginia Department of Forensic Science (VDFS), with the goal of providing scientifically relevant information to aid in the decision whether or not to switch from the five-color version to the six-color version..

The PowerPlex® Fusion System detects 22 autosomal STR loci, 1 Y-STR locus and amelogenin (Figure 1) (4). The system uses 5 dyes to simultaneously amplify and fluorescently detect all 24 of these loci. Previous studies demonstrated 5C as a sensitive, specific and robust STR typing system (4, 5). The reproducibility, accuracy and precision of this system has been well documented, as well as its concordance with other STR kits developed by different manufacturers (4–6). The PowerPlex® Fusion 6C System detects 23 autosomal STR loci, 3 Y-STR loci and amelogenin (Figure 1). It was developed with 6-dye chemistry, allowing the amplification and fluorescent detection of 27 loci simultaneously, thus including more loci within a single reaction (7). These expanded loci include all 24 within the PowerPlex® Fusion System , as well as DYS576, DYS570, and SE33, improving mixture deconvolution and increasing the power of discrimination (7, 8). The PowerPlex® Fusion 6C System shows high specificity for human DNA, high sensitivity with minimal DNA input and concordance across laboratories (9). As with 5C, 6C provides a robust, accurate and sensitive STR typing kit for forensic casework and DNA databasing (7–9).

Figure 1. Loci included in the PowerPlex® Fusion (A) and PowerPlex® Fusion 6C (B) Systems.

The two main goals of this study were: 1) to assess a research design for evaluating new technologies, prior to committing to validation and implementation; and 2) to evaluate an example of this approach, specifically targeting performance metrics between two robust STR typing kits.

Methods

Samples were obtained from the VDFS DNA Research Section. Sample processing for DNA was performed using the DNA IQTM System (Promega) and organic extraction following VDFS protocols for manual extraction, and the PowerQuant® System (Promega) with the QuantStudioTM 5 (Applied Biosystems (AB), Foster City, CA) instrument for quantitation (10, 11). The Veriti (AB) or GeneAmpTM 9700 instrument was used for PCR (12, 13). Detection and analysis utilized the 3500xL Genetic Analyzer (AB) and GeneMapper® ID-X version 1.4 (AB) (14, 15). Probabilistic modeling employed the use of TrueAllele® Casework, following the VDFS procedures (16).

Targeted studies included half-volume reaction optimization, alleles obtained in three- and four-person mixtures, bin overlap and allele ambiguity, assessment of mixtures using probabilistic genotyping, and performance with degraded samples.

Results and Discussion

At the VDFS, half-volume reactions are utilized for 5C. In order to directly compare with 6C, half-volume reaction conditions were optimized for 6C with the final amplification conditions at: 12.5μl volume, 28 cycles for PCR, and a target template of 0.625ng (data not shown).

Since a fundamental difference between the STR kits is the five-dye versus six-dye chemistries, the first targeted metric involved percent profile of alleles obtained for three- and four-person mixtures (n = 9 and n = 12, respectively). Mixtures were generated with a range of contributor ratios to induce low-level proportions. The same samples were amplified with 5C and 6C, and to directly compare performance, SE33 was not included. Three-person mixtures demonstrated no statistically significant difference in percent profile obtained between 5C and 6C (α = 0.05; student’s paired t-test). Four-person mixtures demonstrated a significant difference in allele counts between kits, with 6C averaging three more alleles in four-person mixtures than 5C (Figure 2). Even without SE33, 6C provided additional allelic information in four-person mixtures.

Figure 2. Percent profile of four-person mixture samples using the PowerPlex® Fusion and PowerPlex® Fusion 6C Systems. 6C amplified samples were more consistent and provided more allelic information than 5C.

Next, bin overlap and peak ambiguity were assessed, specifically targeting pull-up. Overlapping bins were counted if they exhibited overlap with a bin in any another color channel for both 5C and 6C. Overlapping bins can potentially increase ambiguity as to whether a peak was pull-up or an allele. Fewer overlapping bins were observed for 5C than 6C, presumably due to the additional color channel for 6C. Ambiguous peaks were qualitatively measured in the same three- and four-person mixtures by determining total pull-up into bins, as well as number of pull-up events that led to an ambiguous allele call. Higher pull-up occurrence was observed for 6C, but fewer peaks were ambiguous upon closer inspection. Figure 3B shows an example of an ambiguous allele call at the D19S433 locus in 6C, which was ultimately determined to be an artifact. Less ambiguity leads to less time in data analysis for peak determination and greater accuracy of allele calls for evidentiary samples. However, when the same mixtures typed for both kits were analyzed using TrueAllele® Casework, without including SE33 in the likelihood ratio (LR) calculations, no significant differences were observed for LRs produced for the contributor reference profiles (data not shown). If SE33 were included, 6C would typically produce higher LRs for most contributors.

Figure 3. Peak ambiguity of three- and four-person mixtures. A: Pull-up into bins for PowerPlex® Fusion and PowerPlex® Fusion 6C Systems, as well as ambiguous pull-up that led to a false allele call. B: Electropherogram displaying ambiguous allele (12) at the D19S433 locus in 6C. The peak is in the n–1 stutter position and is also located in a bin directly below a bin in the yellow channel. This allele was confirmed to be an artifact.

Lastly, performance of both typing kits with degraded samples was measured. Fusion 6C has an additional locus under 220 base pairs than 5C, giving it the potential to produce more discriminating profiles from degraded samples (17). The performance comparison of 5C versus 6C demonstrated no statistically significant difference in allele counts; however, more alleles were counted for the majority of Fusion 6C-amplified samples, which can provide additional information in an already limited forensic sample (Figure 4).

Figure 4. Degraded sample autosomal allele counts in the PowerPlex® Fusion and PowerPlex® Fusion 6C Systems. SE33 was included in 6C. Samples amplified with 6C show higher allele counts for most of the samples. Degradation index from the PowerQuant® System is shown above allele counts for each sample, with increasing index indicating higher levels of degradation.

Conclusions

The direct comparison of 5C and 6C based upon targeted performance metrics deemed important to the VDFS yielded no extensive differences. The performance of 6C was slightly better with the complex four-person mixtures and yielded less ambiguous pull-up, which is advantageous to a forensic laboratory. The targeted mini-validation approach resulted in scientifically valid information that will be used to make an informed decision if the advantages detailed in this study warrant the validation and implementation of 6C. Overall, this approach was quite successful in meeting the goals of the study, and this research design can be broadly applied. Laboratories faced with non-mandatory updates in DNA technology can target an approach that focuses on performance metrics valuable to that laboratory and determine if a new technology possesses an advantage over current methods. This targeted approach can ultimately save a laboratory time, money and resources, since a decision to invest in more extensive training, validation and implementation can be made early in the process. Overall, this study presents an approach for informed decision making and may extend beyond the example presented in this study.

References

1. Federal Bureau of Investigation. Quality assurance standards for forensic DNA testing laboratories. 2020.
2. European Network of Forensic Science Institutes. Guidelines for the single laboratory Validation of Instrumental and Human Based Methods in Forensic Science. 2014.
3. Butler JM, Tomsey CS, Kline MC. Can the Validation Process in Forensic DNA Typing Be Standardized? 2014. Proceedings of the 15th International Symposium on Human Identification.
4. Oostdik K, Lenz K, Nye J, Schelling K, Yet D, Bruski S, et al. Developmental validation of the PowerPlex® Fusion System for analysis of casework and reference samples: A 24-locus multiplex for new database standards. Forensic Sci Int Genet 2014; 12: 69-76.
5. Oostdik K, Ensenberger M, Sprecher C, Bourdeau-Heller J, Krenke B, Storts D. Bridging Databases for Today and Tomorrow: The PowerPlex® Fusion System. Promega Corporation. 2012.
6. Pfoser K, Owen S. Evaluation of the PowerPlex® Fusion System for Use on the ABI PRISM® 310 Genetic Analyzer. Promega Corporation. Updated 2013.
7. Ensenberger MG, Lenz KA, Matthies LK, Hadinoto GM, Schienman JE, Przech AJ, et al. Developmental validation of the PowerPlex® Fusion 6C System. Forensic Sci Int Genet 2016; 21: 134-144.
8. Cisana C, Cerri N, Bosetti A, Verzeletti A, Cortellini V. PowerPlex® Fusion 6C System: evaluation study for analysis of casework and database samples. Croat Med J 2017; 58:26–33.
9. Boavida A, Bogas V, Sampaio L, Gouveia N, Porto MJ, Corte-Real F. PowerPlex® Fusion 6C system: internal validation study. Forensic Sci Research 2018; 3:2: 130-137.
10. Forensic Biology Procedures Manual 210-D2004. Extraction of DNA. Virginia Department of Forensic Science. June 30, 2020.
11. Promega Corporation. PowerQuant® System Technical Manual. Madison, WI: Promega Corporation, Revised 01/2020.
12. Forensic Biology Procedures Manual 210-D2007. PowerPlex® Fusion Amplification and Long Term Storage. Virginia Department of Forensic Science. June 30, 2020.
13. Promega Corporation. PowerPlex® Fusion 6C System Technical Manual. Madison, WI: Promega Corporation, Revised 01/2015.
14. Forensic Biology Procedures Manual 210-D2010. CE For PowerPlex Fusion. Virginia Department of Forensic Science. December 27, 2017.
15. Forensic Biology Procedures Manual 210-D2016. Interpretation of PowerPlex® Fusion CE Data. Virginia Department of Forensic Science. December 23, 2019.
16. Forensic Biology Procedures Manual 210-D2020. TrueAllele® Casework System. Virginia Department of Forensic Science. September 26, 2017.
17. Pizzamiglio M, Marino A, Coli A, Floris T, Garofano L. The use of mini-STRs on degraded DNA samples. International Congress Series 2006; 1288: 498-500.