12th December 2022

Summary of experiences so far on RAPPORT….

Myself and my team were really delighted and excited to be funded for this first ‘Discovery’ stage of the Wellcome Data Prize in Mental Health. Firstly, it was an opportunity to explore newer epidemiological methods, that combine the predictive power of machine learning with the explanatory capacity of traditional statistical methods. Specifically, this was an opportunity to deploy these newer methods in the field of mental health research. These are ‘targeted learning’, which is used increasingly in other fields for causal modelling, and; ‘causal forests’ which can evaluate how different groups of people respond differently to interventions, treatments or ‘exposures’.

Secondly, for some time I have become increasingly uncomfortable with the lack of transparency and reproducibility of research using observational health data, especially once machine learning gets involved! Machine learning can make things more complex and difficult to report and reproduce as most of the methods have a lot of ‘moving parts’, including stochastic (random) processes. It increasingly strikes me as odd that even if you want to conduct and publish a systematic review you are expected to make an a priori protocol publicly available. In contrast, in observational data-based research, protocols and plans of analyses are very rarely published before the study begins! Moreover, the code used to clean, manage and analyse the data is rarely made publicly available. This has contributed to a “crisis of reproducibility” in science and other related problems such as the notorious ‘p-hacking’ (performing multiple analyses until you discover one that shows a statistically significant result and selectively publish that!). Therefore, we saw this project as an opportunity to practise and disseminate open science principles in a way which helps encourage best practice in the field of mental health research.

The members of our research team each bring different, but complementary expertise to our project. We have an economist, a statistician, and two experts experience-researchers as part of the team. In terms of clinical backgrounds, I’m an adolescent psychiatrist and we also have Lina Gega, who is an academic mental health nurse. In addition to our professional and academic backgrounds we also have to remember that we also bring different lived experiences and personal perspectives to the project. This has to be taken into account and allowed to shape the language we use with each other. Also, three of us are quite ‘technical’ so it is important that we don’t bamboozle other members with obscure terminology and acronyms!        

As a researcher I always prefer to be involved in the whole “vertical process” of deciding on which variables to collect, collecting the data and then analysing and reporting on it. Thus, it can be frustrating working with secondary data sources, even high quality ones such as the Millennium Cohort Study. This is especially true in relation to mental health measures- those who set up the cohorts don’t always choose the instruments I would have done. Nevertheless, part of our work, mostly through the involvement of our lived experience experts, is to identify other variables that could be collected in the future. This will help with future causal modelling of the relationship between physical activity (and other "active ingredients") and mental health. On this note, the involvement of ‘experts by experience’ has been an essential part of health research for many years. The effective coproduction of research is almost a methodological specialism in its own right now. In this particular project our experience of patient and public involvement has been extremely positive. Our lived experience experts have been quick to raise points and issues that we had not always considered. I’m also keen to see how our lived experience experts can help make sense of our findings and also help with communicating them to a broad audience. I must say though, my own personal view is that it is not always helpful to have a very artificial divide between ‘experts by experience’ and the rest of the research team; most of us working on the project, as with the wider population, have our own lived experience in relation to mental health issues, both in ourselves and in close family members.

As a research team we are made up of a Greek, a Hungarian and British nationals. However, it has been interesting to hear about the perspectives of those based in South Africa, and the particular challenges presented in that context, in relation to the mental health of young people. Although our mental health services in the UK are facing many difficulties, we still have far more resources than those in lower and middle-income countries.

Moreover, we often forget how much progress many developed countries have made in destigmatising and addressing mental health problems compared to how these were perceived and treated in the last century. Still, perceptions and care of mental illness have variably progressed - or not - across different countries in the world

However, I’ve also had to recognise there is much we can learn from mental health services in other countries that have creatively adapted to a lack of professionals, such as psychiatrists. There are definitely some lessons we could learn in the UK, especially as our workforce shortages of both psychiatrists and registered mental health nurses becomes more severe.

We are conscious as a team, that the Wellcome Trust has adopted the “active ingredients” approach to serve as a roadmap for an ambitious programme to address common mental health problems in adolescents and young adults. This provides a clear focus. Indeed, although this will hopefully change with new cohorts, many of the large observational studies from relatively wealthy countries have not collected any data that allow the direct measurement of some of the active ingredients such as “sense of purpose”.

As we start to near the end of the ‘Discovery Phase’ of the Mental Health Data Prize we are increasingly focused on what the next, ‘Prototyping’ stage will look like. In this regard, almost from the start we have reached out to international potential collaborators. In particular, we have been in contact with global experts who will be able to help us create, scale up and implement digital tools that will make these newer epidemiological methods accessible to mental health researchers across the world, irrespective of economic status.

Paul Tiffin

(on behalf of 'teamRAPPORT')


Comments

Post a Comment

Popular posts from this blog

Co-producing reporting guidelines for targeted learning studies

24th February 2023 There has been well publicised problems with reproducible and transparent reporting from observational data based research studies. This is a particular issue once machine learning gets involved, come due to the complexity. Many of the studies that are published would be difficult to reproduce, even by the research team themselves. Therefore today we had a workshop with two lived experience expert researchers, an additional lived experience expert and myself. Together we went through the draft reporting guidance we have been creating for targeted learning studies. This was a very useful exercise and helped harness lay researcher and lived experience into the guidance. This was important not just for making plans of analysis and protocols available publicly before analysis starts. It was also important in thinking about how results are reported and disseminated and the role of lived experience experts in this process. These guidelines were intended to build on s...
Read more
13 January 2023 Whole team event The whole team, including the lay researchers, met again to discuss a preliminary Directed Acyclic Graph (DAG). This is a diagram that describes how different variables are thought to be causally related to each. It was co-produced by the researchers and experts by experience. The DAG was intended to capture important variables that had been raised previously by the lay researchers and a young people’s advisory group. The meeting was very useful. In addition to discussing the DAG, the data scientist on the project ran through some preliminary results of the machine learning analysis to see if the findings resonated with the experts by experience. This led to some debate and the PPI lead is going to follow up with the lay researchers individually following the meeting. Finally, I delivered some training on the concept of machine learning in order to de-mystify this approach. All team members contributed during the meeting and it was great to hear a varie...
Read more
Introducing Paul: Hi, I’m Paul and I am the principal investigator (PI) for the RAPPORT project.   I have to give full credit to Lewis though, for encouraging me to apply for this award and for doing a lot of the preparation of the original grant application. By professional background I am an adolescent psychiatrist. I spent the first five years or so as a consultant psychiatrist working in an inpatient setting. I spent some time developing and evaluating early intervention services for young people affected by psychosis. In more recent years I’ve been working into our regional Forensic Child and Adolescent Mental Health Service for Yorkshire and Humber. Academically I’ve always been interested in applying quantitative methods to issues and problems in mental health services. In this regard there has been two main streams to my work. The first is around mental health services. The second stream is focused on medical and healthcare workforce issues, particularly selection and r...
Read more