CEA presented 1 platform and 9 posters at SETAC Seville in May 2024. We will be showcasing each of these presentations in a series of ‘SETAC Spotlight’ articles. This week it is:

SETAC Spotlight: Benchmark dose modelling – Expectation vs Reality

Authors: Sarah Priestly, Emma Russell, Helena Crosland, Amy Brooks

Cambridge Environmental Assessments, RSK ADAS, Boxworth, UK

Background

CEA performed BMD evaluations for several bird and mammal risk assessments in October 2023 in accordance with the new EFSA 2023[1] guidance document and using the EFSA online Bayesian BMD tool.

Data Evaluation

Data were initially evaluated and extracted based on apparent treatment-related biological effects, as identified by the statistical analysis of data in the original study reports. These data were then assessed for ecological relevance as set out in the 2023 EFSA bird and mammal guidance document (referenced above), i.e. a ≥10% deviation in biological measurement of treated vs control animals were considered ecologically relevant ^[2].

The first issue encountered was extracting the data:

• Older study reports were harder to extract – scanned/poor quality PDFs require careful manual extraction
• Sheer volume of data meant this task was very time consuming
• Where manual extraction, rather than copy/paste were used, these values required careful quality control checking

Once the files were extracted, they could then be put through the EFSA Bayesian BMD tool.

Continuous data (body weight gain, etc.) were evaluated using individual animal data, whilst quantal data (mortality, etc.) were evaluated using group means, standard deviation (SD) and sample size (n – number of animals in treatment group).

Issues encountered

The first issue we encountered with the tool was that the data need to be modelled for each single time point. For example, eggs were laid and recorded at several time points e.g. days 4, 7, 10; so these time points needed to be evaluated separately. The tool does allow for you to input all of the available data and select a time point, however, a lot of measurements makes this task very time consuming.

The EFSA tool currently does not allow you to model all dates simultaneously.

The second issue was for body weight data. Quite often the data will include negative numbers, which cannot be evaluated by the BMD tool. Upon request, EFSA was unable to offer a standard solution to this common issue, but suggested that adding ‘a positive constant that is larger in absolute value than the smaller negative value observed’ might be acceptable if an appropriate ‘sensitivity analysis’ could also be provided.

Unfortunately, EFSA’s suggested work-around requires statistical knowledge, and many risk assessors may not have the appropriate skills or knowledge.

Thirdly, error messages were lines of ‘code’ without indicating why the tool would not run the dataset. This resulted in time-consuming investigations into the reasons for the errors and the possible solutions to them.

The final issue we encountered was with tool version. Originally runs used version 9060 of the EFSA Tool. We then re-ran some of the same data sets using the most up to date version (9083) and found that some results were noticeably different.

There was no explanation as to what had been changed in the tool, making it difficult to understand the scientific basis for changes, and which datasets might need to be re-run using the most up to date version. This meant that potentially all the data run originally would need to be re-run, which would be very costly and time consuming.

Updates

Since the tool has been updated the error messages have been improved, for example, if there is a zero in the dataset it will inform you and not allow you to go any further. Therefore, you can adjust the dataset to exclude the zero number, and you may be able to exclude this number within the tool itself (although at the time of writing, we haven’t tested this).

Conclusion

Overall, the tool would be beneficial in determining BMD10 values for use in the risk assessment if the current uncertainties are addressed before the guidance is noted by the EU.

[1] EFSA (European Food Safety Authority), 2023.Guidance on the risk assessment for Birds and Mammals. EFSA Journal 2023;21(2):7790, 300 pp.

[2] In some instances, there may be effects that can have a > 10% significant difference but not statistically significant. These should be evaluated according to the original study report.