How many papers are published each week reporting on trials of interventions involving behavioural aspects of health?

Behavioural science as applied to human health and wellbeing is crucial for meeting the challenges facing humanity in the 21 st century. Although only a small fraction of health-related research focuses on behaviour, it still represents a substantial body of evidence that is accumulating rapidly. The sheer number of publications presents a huge challenge for manual extraction of information from study reports for purposes of evidence synthesis. We assessed the extent of this challenge, focusing on estimating the number of published reports of effectiveness trials relating to behaviour, either as outcomes or as key determinants of health outcomes. We adopted a conservative search strategy using words and phrases relating to effectiveness or efficacy trials of interventions involving commonly researched health-related behaviours. We reviewed a sample of the papers identified using the search strategy to estimate the proportion that were in scope. Using the search strategy we found an estimated 6,793 papers published per year over the years 2018-2022 inclusive. Of these, 81% were estimated by manual inspection to be in scope, resulting in an estimated 5,502 per year or 106 papers per week. The true figure is likely to be higher because of behaviours that were not canvassed in the search and trials represent only a small fraction of papers seeking to describe and understand behavioural issues relating to health and wellbeing.


Introduction
Human behaviour plays a key role in health and wellbeing [1] . Although the proportion of health research that directly involves behavioural measures is small, this still represents a substantial investment, costing many billions of dollars or equivalent each year. It has been estimated that more than 80% of research in health is wasted because of inefficient and ineffective research practice [2] . A major part of the problem of waste is that most research is not discoverable or usable because of the cost and time needed to manually extract and integrate evidence from the large numbers of behavioural studies.
To help estimate the size of the challenge we undertook a bibliometric analysis to estimate the numbers of studies Qeios, CC-BY 4.0 · Article, January 16, 2023 Qeios ID: U6VX2Z · https://doi.org/10.32388/U6VX2Z 1/6 published on evaluations of behavioural interventions relating to health. Specifically, we aimed to estimate the overall number of published studies and the number of randomised trials involving these behavioural interventions and their effectiveness or efficacy.
Any estimates of this kind are inevitably approximate because: 1. There are a very large number of behaviours that impact on health and wellbeing, and it is not practicable to ensure that one has included all of them.
2. There is not a 100% consensus on what should be included in the class of 'health-related behaviours'. For example, one might legitimately include ones that have an indirect impact on health and well-being because they affect pollution or climate change. One might also include criminal behaviours, bullying or behaviours that lead to accidental injury.
3. Variability in the terms used to describe behaviours means that studies relating to a particular behaviour may not be discovered in a search.

4.
A search based on matching terms may be overly inclusive in finding publications that include the terms but where the focus of the study is on something that is not a health-related behaviour.

5.
A single study may be reported in multiple papers and multiple studies may be reported in a single paper (e.g., in a review).
6. Databases that are amenable to complex search expressions will not include all relevant studies.
Nevertheless, even a broad estimate is useful in terms of estimating the size of the challenge when it comes to extracting information from studies in this domain. Therefore, this study aimed to estimate the number of published research papers reporting on the effectiveness or efficacy of interventions in which behavioural measures (e.g., dietary behaviours) were included either as factors influencing an outcome (e.g., obesity) or were themselves an outcome (e.g., smoking cessation, alcohol consumption and physical exercise).

Methods
We used PubMed as the database for the search (https://pubmed.ncbi.nlm.nih.gov/). This is a comprehensive database of research papers relating to health and wellbeing and so provided a basic top-level of screening for these kinds of paper. It has an advanced search facility that includes automatic inclusion of MeSH synonyms (https://www.ncbi.nlm.nih.gov/mesh/), the ability to create expressions using Boolean operators, and includes a filter for randomised trials. On the other hand, it does not generally include 'grey' literature and does not have a comprehensive coverage of relevant journals.
We constructed the search expression as a combination of terms referring to behaviours combined with the term 'intervention' and the expression '(efficacy or effectiveness)'. The intention was to find intervention evaluations relating to the behaviours of interest.
To identify terms for health-related behaviours we viewed all the MeSH terms under behaviour and identified those that Qeios, CC-BY 4.0 · Article, January 16, 2023 Qeios ID: U6VX2Z · https://doi.org/10.32388/U6VX2Z 2/6 related to health. In the case of some behaviours such as 'tobacco use' we just included the term 'tobacco' since this was expected to capture different ways in which tobacco use may be referred to. We used the PubMed truncation facility where appropriate to allow searching for different variants of a term (e.g., we used 'drive*' to allowed capture of 'driving and 'driver').
Starting with MeSH terms relating to common behaviours relating to health and wellbeing, we experimented by hand with inclusion and exclusion of terms to attempt to minimise the number of false positives while capturing papers that were in scope. We noted that terms relating to physical activity, alcohol, substance use, gambling and tobacco use, and diet contributed most to the paper found but that other behaviours such as medication adherence and transport behaviour also made a significant contribution.
We downloaded the most recent 100 papers from the searches and manually inspected the titles of the papers to assess whether they were in scope. We calculated the proportion that were judged to be in scope and used this to adjust the estimate of the number of papers in the target domain. Table 1 shows the results of the search. The final search expression found 33,967 papers with the trials filter on and 262,058 papers with the trials filter off. Examining the titles of the most recent 100 papers in each case it was estimated that 81% of those with the trials filter on and 46% of those with the trial filter off were in scope. This led to an estimate of 27,513 papers with the trials filter on and 120,547 papers with the trial filter off. Therefore, we estimated that an average of 5,502 papers reporting on trials involving behaviour were published per year between 2018 and 2022, amounting to 106 per week, and that an average of 24,109 papers involving behaviour, but not necessarily involving trials, were published over that period, amounting to 464 per week.

Discussion
Using the PubMed database, a search expression aimed at identifying commonly targeted behaviours and an adjustment for papers found to be out of scope, we estimated that in the years 2018-2022 approximately 106 papers were published per week involving trials relating to behavioural aspects of health and wellbeing. This amounted to 5,502 papers per year.
When the search was not limited to trials, the figures were 464 and 24,109 respectively. These estimates are necessarily approximate but provide a sense of the scale of the task when it comes to extracting information from the corpus of literature for the purposes of evidence synthesis.
We can use these figures to arrive at an estimate of the person hours that would be required on an ongoing basis to extract information for evidence synthesis using currently available tools. The Human Behaviour Change Project (HBCP) [3] and the development of a Paper Authoring Tool (PAT) [4] (https://paperauthoringtool.com/) to facilitate reporting of randomised trials and feasibility trials indicate that there are likely to be at least 500 key pieces of information that could usefully be extracted from the report of the average clinical trial of an intervention aimed at changing behaviour. The key pieces of information cover the intervention and comparator intervention/s (features of content and delivery), the target Qeios, CC-BY 4.0 · Article, January 16, 2023 Qeios ID: U6VX2Z · https://doi.org/10.32388/U6VX2Z 4/6 population, the setting, the outcomes, the mechanisms of action and methodological features such as follow-up rates.
We estimate that a skilled researcher who is familiar with the content could perhaps extract one paper's worth of information in an 8-hour day. The reliability with which this information can be extracted can be relatively high amongst trained researchers but ideally there should be two people extracting the information and then discussing discrepancies.
We estimate that obtaining reliable information from a single paper therefore would take around 20 person-hours.
To manually extract information from just the trials relating to behavioural aspects of health and wellbeing would therefore One can question the above assumptions and arrive at different estimates for the cost but even if they were half of what is projected, it would still be a mammoth undertaking and something that would be unlikely ever to be funded. If, as seems likely, the pace of publication of behavioural trials on health and wellbeing increases, this figure would grow accordingly.
It is clear that building an integrative and cumulative science of behaviour as it relates to health and wellbeing will require considerable automation. Attempts to extract information from papers using natural language processing are at an early stage, for example as developed in the Human Behaviour-Change Project [5] . During the work of this project, it has become apparent that the way that papers are written presents a potentially insurmountable barrier to full automation of this process. There is just too much variation in the way that information is presented. This means that we are likely to need researchers to meet automated information extraction tools at least halfway and provide information in their papers in a much more structured form. The Paper Authoring Tool (PAT), developed by researchers at University College London in collaboration with the Human Behaviour-Change Project, the Society for the Study of Addiction and Silverback Information Services may be an important step in this direction [4] . It prompts authors of papers to provide the required information in a structured form, drafts the paper from them, and crucially creates a machine-readable version of all the information in the paper as a JSON file that authors can put online for anyone wishing to undertake evidence synthesis.
PAT includes a facility to link all the key features in a paper to relevant ontologies [6] , facilitating the structuring of knowledge and evidence integration.
The methodology used in this paper could be improved by including more databases, using a larger sample of papers to manually examine to establish whether they are in scope, and using a more comprehensive set of search terms, but the main conclusions of the paper are likely to remain sound: that the number of papers reporting on interventions of behavioural trials in health is too large for it to be feasible to manually extract all the key information from those papers for evidence synthesis. We are beginning to find ways to automate the process [5][7] but we will also need researchers to report their studies in a more structured way using tools such as PAT.