12th December 2022
Summary of experiences so far on RAPPORT….
Myself and my team were really delighted and excited to be funded for
this first ‘Discovery’ stage of the Wellcome Data Prize in Mental Health. Firstly, it was an opportunity to explore newer epidemiological methods,
that combine the predictive power of machine learning with the explanatory
capacity of traditional statistical methods. Specifically, this
was an opportunity to deploy these newer methods in the field of mental health
research. These are ‘targeted learning’, which is used increasingly in other
fields for causal modelling, and; ‘causal forests’ which can evaluate how
different groups of people respond differently to interventions, treatments or
‘exposures’.
Secondly, for some time I have become increasingly uncomfortable with
the lack of transparency and reproducibility of research using observational
health data, especially once machine learning gets involved! Machine learning
can make things more complex and difficult to report and reproduce as most of
the methods have a lot of ‘moving parts’, including stochastic (random)
processes. It increasingly strikes me as odd that even if you want to conduct
and publish a systematic review you are expected to make an a priori protocol
publicly available. In contrast, in observational data-based research,
protocols and plans of analyses are very rarely published before the study
begins! Moreover, the code used to clean, manage and analyse the data is rarely
made publicly available. This has contributed to a “crisis of reproducibility”
in science and other related problems such as the notorious ‘p-hacking’
(performing multiple analyses until you discover one that shows a statistically
significant result and selectively publish that!). Therefore, we saw this
project as an opportunity to practise and disseminate open science principles
in a way which helps encourage best practice in the field of mental health
research.
The members of our research team each bring different, but complementary
expertise to our project. We have an economist, a statistician, and two experts
experience-researchers as part of the team. In terms of clinical backgrounds,
I’m an adolescent psychiatrist and we also have Lina Gega, who is an academic
mental health nurse. In addition to our professional and academic backgrounds
we also have to remember that we also bring different lived experiences and
personal perspectives to the project. This has to be taken into account and
allowed to shape the language we use with each other. Also, three of us are
quite ‘technical’ so it is important that we don’t bamboozle other members with
obscure terminology and
acronyms!
As a researcher I always prefer to be involved in the whole “vertical
process” of deciding on which variables to collect, collecting the data and
then analysing and reporting on it. Thus, it can be frustrating working with
secondary data sources, even high quality ones such as the Millennium Cohort
Study. This is especially true in relation to mental health measures- those who
set up the cohorts don’t always choose the instruments I would have done.
Nevertheless, part of our work, mostly through the involvement of our lived
experience experts, is to identify other variables that could be collected in
the future. This will help with future causal modelling of the relationship
between physical activity (and other "active ingredients") and mental
health. On this note, the involvement of ‘experts by experience’ has been an
essential part of health research for many years. The effective coproduction of
research is almost a methodological specialism in its own right now. In this
particular project our experience of patient and public involvement has been
extremely positive. Our lived experience experts have been quick to raise
points and issues that we had not always considered. I’m also keen to see how
our lived experience experts can help make sense of our findings and also help
with communicating them to a broad audience. I must say though, my own personal
view is that it is not always helpful to have a very artificial divide between
‘experts by experience’ and the rest of the research team; most of us working
on the project, as with the wider population, have our own lived
experience in relation to mental health issues, both in ourselves and in close
family members.
As a research team we are made up of a Greek, a Hungarian and British
nationals. However, it has been interesting to hear about the perspectives of
those based in South Africa, and the particular challenges presented in that
context, in relation to the mental health of young people. Although our mental
health services in the UK are facing many difficulties, we still have far more
resources than those in lower and middle-income countries.
Moreover, we often forget how much progress many developed countries
have made in destigmatising and addressing mental health problems compared to
how these were perceived and treated in the last century. Still, perceptions
and care of mental illness have variably progressed - or not - across different
countries in the world
However, I’ve also had to recognise there is much we can learn from mental health services in other countries that have creatively adapted to a lack of professionals, such as psychiatrists. There are definitely some lessons we could learn in the UK, especially as our workforce shortages of both psychiatrists and registered mental health nurses becomes more severe.
We are conscious as a team, that the Wellcome Trust has adopted the
“active ingredients” approach to serve as a roadmap for an ambitious programme
to address common mental health problems in adolescents and young adults. This
provides a clear focus. Indeed, although this will hopefully change with new
cohorts, many of the large observational studies from relatively wealthy countries have
not collected any data that allow the direct measurement of some of the active
ingredients such as “sense of purpose”.
As we start to near the end of the ‘Discovery Phase’ of the Mental Health
Data Prize we are increasingly focused on what the next, ‘Prototyping’ stage
will look like. In this regard, almost from the start we have reached out to
international potential collaborators. In particular, we have been in contact
with global experts who will be able to help us create, scale up and implement
digital tools that will make these newer epidemiological methods accessible to
mental health researchers across the world, irrespective of economic status.
Paul Tiffin
(on behalf of 'teamRAPPORT')
Comments
Post a Comment