Posts

Co-producing reporting guidelines for targeted learning studies

24th February 2023 There has been well publicised problems with reproducible and transparent reporting from observational data based research studies. This is a particular issue once machine learning gets involved, come due to the complexity. Many of the studies that are published would be difficult to reproduce, even by the research team themselves. Therefore today we had a workshop with two lived experience expert researchers, an additional lived experience expert and myself. Together we went through the draft reporting guidance we have been creating for targeted learning studies. This was a very useful exercise and helped harness lay researcher and lived experience into the guidance. This was important not just for making plans of analysis and protocols available publicly before analysis starts. It was also important in thinking about how results are reported and disseminated and the role of lived experience experts in this process. These guidelines were intended to build on s
Post a Comment
Read more

Show 'n Tell

 7th February 2023 Last week Wellcome/Social Finance hosted two afternoons of "Show and Tell". This gave teams participating in the Mental Health Data Prize to share their progress and experience of the "Discovery Phase". The presentations were very diverse and each brought interesting aspects and insights to the fore. Some of the presentations, including ours, focussed on the results of novel analyses performed. Others placed more emphasis and spent more timeshow casing what the eventual digital tool would look like. There was plenty of lively discussion, ranging from talking about technical issues, to broader discussions about how to most effectively incorporate lived experience into the projects. Many of the challenges mentioned were shared by most, if not all of the teams. The most common theme mentioned was the very short timescale for each stage of the project. This made staffing the projects, which is needed to convert funding into output, challenging. Howeve
Post a Comment
Read more
13 January 2023 Whole team event The whole team, including the lay researchers, met again to discuss a preliminary Directed Acyclic Graph (DAG). This is a diagram that describes how different variables are thought to be causally related to each. It was co-produced by the researchers and experts by experience. The DAG was intended to capture important variables that had been raised previously by the lay researchers and a young people’s advisory group. The meeting was very useful. In addition to discussing the DAG, the data scientist on the project ran through some preliminary results of the machine learning analysis to see if the findings resonated with the experts by experience. This led to some debate and the PPI lead is going to follow up with the lay researchers individually following the meeting. Finally, I delivered some training on the concept of machine learning in order to de-mystify this approach. All team members contributed during the meeting and it was great to hear a varie
Post a Comment
Read more
6th January 2023 Happy New Year!   Over the Christmas period I looked again at the mathematics behind targeted learning.  What is the 'magic' 🐇🎩behind this technique?   I was familiar with machine learning so the first part was clear to me. This is where propensity scores and predicted probabilities for the outcomes are generated, usually through ‘superlearning’.   However, previously I have never been quite sure how the ‘updating’ worked.   Thankfully I found this helpful video from Susan Gruber, which explains the maths behind the ‘updating’ stage in detail.   https://youtu.be/8Q9dfW3oOi4   There is also a helpful paper that explains the targeted learning process in detail  here . The key issue is that, once the machine learning predictions are derived then they can be further updated by essentially regressing the residuals on to the observed outcomes for a validation data set. This involves using the predictions as an ‘offset’ in a logistic regression. In this context an ‘
Post a Comment
Read more

The 'Table 2 fallacy'

 21st December 2022 I was asked a question the other day that got me reflecting on some issues in causal inference. The question was this: ‘…if a multivariable model includes all the relevant potential confounders then what is the advantage of using causal modelling methods ?’ Having reflected a bit on my own training in causal inference and after a quick Google I found this paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3626058/ The ‘Table 2 Fallacy’ refers to the assumption that all the effects of covariates estimated in a multivariable model are independent effect sizes. Indeed, the main exposure of interest is treated like any other covariate within a multivariable model.  It’s called the ‘Table 2 Fallacy’ because observational studies often use Table 1 to present the descriptives of the data used and Table 2 the results of a multivariable analysis. In my case it should probably be called the ‘Table 3 Fallacy’ as I usually use Table 2 to present my univariable results.  Interp
Post a Comment
Read more