eDNA Validation Scale

eDNA Validation Scale

Introduction

Recent years have seen an explosion in interest in the use of eDNA to detect the presence of individual species in the environment with targeted essays, as opposed to biodiversity assessments via metabarcoding. The number of published assays is increasing every year, while there is increasing interest from policy makers in implementing the tests for decision-making in environmental management.

A challenge faced by potential users is the huge variation in the level to which published essays have been validated, making it difficult to work out whether or not an assay is reliable enough for application in contexts where expensive management interventions may be applied based on the results.

The validation scale presented here seeks to address this by providing a simple framework in which to place assays according to the level of validation they have achieved. The scale is composed of 5 levels.

On Mobile, Scroll Down and Click ‘Download as PDF’ 

1. Validation steps that have been carried out and ‘passed’

1 In silico analysis PCR protocol applied but not optimised
2 In silico analysis PCR Reaction optimised Some in vitro testing of related species suggests test is specific Detection obtained from water samples in laboratory environment or mesocosm
3 In silico analysis PCR Reaction optimised In vitro specificity confirmed – all closely-related co-occurring species tested Extensive field testing at sites of known presence and absence of target species
4 In silico analysis PCR Reaction optimised In vitro specificity confirmed – all closely-related co-occurring species tested Extensive field testing at sites of known presence and absence of target species Limits of detection established
5 In silico analysis PCR Reaction optimised In vitro specificity confirmed – all closely-related co-occurring species tested Extensive field testing at sites of known presence and absence of target species Limits of detection established Detection probability estimates from statistical modelling Good understanding of ecological, temporal and spatial factors affecting detectability
1. Validation steps that have been carried out and ‘passed’

2. Remaining uncertainties at each level

1 2 3 4 5
DNA may have been introduced by other organisms, including humans. If the target is also a food item then it could have been introduced via waste water DNA may have been introduced by other organisms, including humans. If the target is also a food item then it could have been introduced via waste water DNA may have been introduced by other organisms, including humans. If the target is also a food item then it could have been introduced via waste water DNA may have been introduced by other organisms, including humans. If the target is also a food item then it could have been introduced via waste water DNA may have been introduced by other organisms, including humans. If the target is also a food item then it could have been introduced via waste water
It is not known how ecological and seasonal factors may influence detectability It is not known how ecological and seasonal factors may influence detectability It is not known how ecological and seasonal factors may influence detectability It is not known how ecological and seasonal factors may influence detectability Some untested ecological factors may still influence detectability
It is not known how many samples are required for a 95% chance of detection It is not known how many samples are required for a 95% chance of detection It is not known how many samples are required for a 95% chance of detection It is not known how many samples are required for a 95% chance of detection Sensitivity may vary in different environments
Co-occurring species that have not been tested may co-amplify, leading to false positive results. PCR products should be sequenced to verify target presence Co-occurring species that have not been tested may co-amplify, leading to false positive results. PCR products should be sequenced to verify target presence Co-occurring species that have not been tested may co-amplify, leading to false positive results. PCR products should be sequenced to verify target presence Unrelated species may occasionally co-amplify, although the risk is low given the extensive field testing undertaken. Additional specificity testing may need to be undertaken if using the assay in a different region to that in which it was originally validated Unrelated species may occasionally co-amplify, although the risk is low given the extensive field testing undertaken. Additional specificity testing may need to be undertaken if using the assay in a different region to that in which it was originally validated
It is not known if this assay will work on environmental samples It is not known if this assay will work on environmental samples
2. Remaining uncertainties at each level

3. Valid interpretation of results

1 2 3 4 5
Negative Result Impossible to tell if target is present or absent Impossible to tell if target is present or absent Impossible to tell if target is present or absent Target is likely to be absent, assuming sampling has been carried out at an appropriate time of year and with a high level of replication Target is likely to be absent. Assuming appropriate sampling has been carried out, a probability of species presence (false negative) can be given
Positive Result – Amplification Impossible to tell if target is present or absent Impossible to tell if target is present or absent Impossible to tell if target is present or absent Target is very likely to be present Target is very likely to be present
Positive Result – Sequenced DNA of target is definitely present DNA of target is definitely present DNA of target is definitely present DNA of target is definitely present DNA of target is definitely present
3. Valid interpretation of results
Download as PDF
How should the scale be used?

We hope that when new assays are published, reference will be made to this scale as a standardised way to communicate to end users the extent to which it has been validated. It will help end users to decide whether or not an assay is suitable for their needs in terms of its level of validation and the remaining uncertainty around positive or negative results, and will also help guide any next steps for the further development of particular assays in order to move them further along the scale.

Please note that this is version 1 of the scale, and we expect to make updates to it. We are currently carrying out a meta-analysis of published assays to understand the historic level of validation applied to published assays, and this process involves a review of the original scale as presented here. The final version will be published along with the meta-analysis.

The scale was developed at a workshop of DNAqua-net (Working Group 3: field and laboratory methods) in Innsbruck, Austria in March 2018

Is a semi-validated test still useful?

Very few assays reach the top level on the scale, but this doesn’t necessarily mean that they don’t work well. Care should be taken in basing management responses purely on the results of a semi-validated essay, particularly in regulatory contexts. However, there are times when a semi-validated assay can still be a useful tool to gain an idea of whether particular species may or may not be present – especially when used in combination with other survey methodologies (themselves often imperfect). What is critical is that the user understands the assay is not fully validated and bears this in mind when interpreting results. Therefore, for each level on the scale, we also present the remaining uncertainties that users should be aware of, as well as guidance on how a positive or negative result should be interpreted. Importantly, we also include a set of key assumptions about how the analyses have been conducted.

How does the scale work?

An assay should be considered to have reached the highest point on the scale at which it meets all criteria. Note that the place on the validation scale is location-specific, such that if an assay is to be used in a different region from the one where it was validated, additional field trials will need to be carried out in the new region to ensure the same level of performance is achieved. If additional closely related species are present in the new region then some additional specificity testing will also need to be carried out.

The team behind the scale

Kat Bruce, NatureMetrics

Bettina Thalinger, University of Guelph

Kristy Deiner, Natural History Museum in London

Helen Rees, ADAS

Daniela Sint, University of Innsbruck & Sinsoma

Rosie Blackman, EAWAG

Lynsey Harper, University of Illinois

Michael Traugott, University of Innsbruck & Sinsoma

Caren Goldberg, Washington State University

Go to Top