Establishing a standardised appraisal tool for evaluating internal validity in in vitro studies will enhance the reliability of data used in EU chemical safety regulations, ensuring robust policy decisions.
The comprehensive database of 405 bias items highlights critical issues affecting in vitro study validity, supporting the development of evidence-based chemical risk assessments.
The evolving nature of in vitro methodologies necessitates regular updates to validity assessment tools, aligning policies with the latest scientific advancements to safeguard public health.
Using adaptable resources like the bias item bank enables policymakers to tailor validity assessments to specific regulatory challenges, promoting consistency and transparency across the EU.
Context: In vitro toxicology studies are increasingly being included as evidence in systematic reviews and chemical risk assessments. INVITES-IN, a tool for assessing the internal validity of in vitro studies, is currently under development. The first step in developing INVITES-IN involves the creation of an “item bank”, an overview of study assessment concepts that may be relevant to evaluating the internal validity of in vitro toxicology studies. The item bank and methodology for its creation presented in this manuscript are intended to be a general resource for supporting the development of appraisal tools for in vitro toxicology studies and potentially other study designs.
Methods: We derived the item bank from seven literature sources (one existing item bank created from a systematic review of assessment criteria for in vitro studies, and six purposively sampled study appraisal tools) and the transcripts of three focus groups. Assessment criteria plausibly relating to internal validity were abstracted from the literature sources and focus group transcripts, disaggregated into individual criteria, then normalised to express in the simplest achievable language the core issue in each criterion – an “item bank” of assessment concepts. The items were then mapped onto a set of bias domains. We conducted simple descriptive statistical analyses and visualisations to describe patterns in the dataset and developed recommendations for the use and development of the item bank.
Results: The item bank contains 405 items of potential relevance to evaluating the internal validity of in vitro toxicology studies.
Discussion: To our knowledge, this is the second item bank of any kind to have been created for toxicology studies, and the first to use focus groups as a data source alongside literature analysis. The large number of items contributed by focus group discussions suggests this is an efficient method for capturing internal validity issues that are not easily identifiable in the literature. We believe our item bank and methodology for its creation will be a useful resource for supporting the development of appraisal tools. Due to the broad applicability of many items in the item bank, it may be informative for study designs beyond the in vitro domain.