Case studies

A provider running a group weight management programme for children across several hundred community sites asked whether attending more sessions related to a greater change in outcome, and from what level of attendance onwards. We fitted variance-component models to establish how much of the variation sat between groups rather than between children, then estimated the relationship with the between-group and within-group components separated. The shape was checked with a non-parametric fit before any linear form was assumed, and a specification allowing the relationship to change at a threshold was compared against one that did not.

The same provider had reason to think that the composition of a group mattered to the individual children in it. Everyone in a group shares a venue, a deliverer and a cohort, so ordinary regression cannot separate the influence of peers from the environment they have in common. We absorbed the shared component with group fixed effects and used an instrumental-variables strategy to identify the peer relationship, then carried the analysis through peer review with the provider as co-authors.

A provider delivering a diabetes prevention programme nationally knew that outcomes differed across age, deprivation and ethnicity, and needed to know whether those differences reflected a different response to the programme or a different amount of it received. We specified a mediation analysis with attendance as the candidate mediator, estimating total, direct and indirect paths with inference by bootstrap, and set out which parts of the pathway the design could speak to and which it could not.

An adult weight management provider wanted to know whether a relationship reported in the published literature held in their own records. We replicated the published selection procedure first, so that any difference could be attributed to the data rather than to the method, then re-estimated the same relationship on the full enrolled sample without conditioning on a variable measured after enrolment. Both specifications were reported side by side, with the difference between them traced to the sample each was estimated on.

An online weight management service was losing a substantial share of members who joined and never began, and wanted to distinguish them from the members who went on to use the service most. We fitted logistic models for each behaviour with quadratic terms and interactions throughout rather than assuming constant effects, flagged sparse cells rather than over-interpreting them, and built the report around predicted-probability figures rather than coefficient tables, with two missing-data treatments shown.

A commissioner running a lifestyle service across a large region wanted to understand why completion varied so much between localities. We modelled completion with spatially correlated random effects at small-area level, alongside deprivation, urban and rural classification and travel distance to the nearest delivery venue, and separated the variation attributable to place from the variation attributable to the people living there.

A provider was about to change how it encouraged enrolled participants to attend, and wanted the change evaluated rather than assumed. We designed a randomised comparison before the change was introduced: the unit of randomisation, the outcome definitions, the smallest difference worth detecting and the analysis plan were agreed in writing and dated before any data existed. The specification was written to be followed by the delivery staff who would be collecting the data.

A provider wanted to characterise when members stopped attending rather than only how many of them did. We treated drop-out as a time-to-event outcome, described it with Kaplan–Meier curves by cohort, and fitted proportional-hazards models with time-varying covariates for engagement, distinguishing members who left the service from those who completed it as competing events.

A commissioner needed to know how the people its service reached compared with the population it was funded to serve. We drew the comparison against a national survey, applying the survey design weights and accounting for its clustering and stratification, and calibrated the service records to the same population totals so that the two could be read on one basis. Non-response in both sources was assessed rather than assumed away.

An organisation asked for an independent read of an analysis it had already completed, ahead of a submission. We worked from the outputs and the code supplied, replicated the selection and modelling decisions where they could be reproduced, and checked the denominator, the missing-data treatment and anything conditioned on after enrolment. The written opinion was itemised and phrased so that it could be handed to the analyst who did the work.