Subgroup analyses

11 Dec

Heterogeneity in treatment effects

Newspaper advertisements seeking patients and ...

Subgroup analysis is the core of interpretation of random controlled trials. But it must respect some strictly defined rules otherwise it will lead the reader to dangerous misinterpretation. A recent article (2010) by Kent et al exposes a very useful checklist for authors as well as for readers.

A given treatment induces a 25% Relative Risk Reduction of a given disease; a subgroup analysis implemented on one hand in a low risk of disease group of subjects versus in an other hand in a high risk of disease group of subjects will lead to an Absolute Risk Reduction of 1% versus 5% and a number of subjects needed to treat to prevent one additional disease of 100 versus 20. What is at stakes for health policy decision makers, care providers and patients is no less than efficacy, efficiency and harms of treatments grounded on evidence based results.

In their open access article the authors, from the Institute for Clinical Research and Health Policy Studies (Boston, MA, USA) and from the Center for statistics in Medicine (University of Oxford, UK) and from the department of Hygiene and Epidemiology (Ioannini, Greece) clearly expose the advantages and limits of the subgroup analysis techniques, weather they aim at exploratory research without immediate clinical implication or they attempt to further confirm an already strong a priori pathophysiological or empirical knowledge.

According to those authors, for reporting on subgroup analysis and heterogeneity in treatment effects, health services authors should:

Evaluate the distribution of the risk of disease in the overall study population before any treatment using a pre-specified externally developed risk prediction model (eg: risk score).

Pre-specify the subgroups including the threshold values for continuous or ordinal variables (except for clearly labelled exploratory purposes which are potentially useful for hypothesis generation and informing future research but having little or no immediate relevance to patient care).

Report the statistical significance between subgroups using interaction terms (testing for the significance of a treatment effect within a subgroup is inappropriate due to poor statistical power).

Correct the statistical comparisons for the number of the number of primary subgroup analysis performed.

The full text of the entire article is available in open access here.

doi:10.1186/1745-6215-11-85
Cite this article as: Kent et al.: Assessing and reporting heterogeneity in
treatment effects in clinical trials: a proposal. Trials 2010 11:85

4 Responses to “Subgroup analyses”

  1. James P. Scanlan January 4, 2012 at 12:14 am #

    The referenced article by Kent et al., like virtually all other research and commentary on subgroup analyses, is premised on the view that absent a subgroup effect a factor that decreases or increases an outcome rate will do so to the same proportionate degree for any baseline rate. Such a view, however, is fundamentally illogical for the simple reason that a factor cannot cause equal proportionate changes in two different baseline rates while at the same time causing equal proportionate changes in the rates for the opposite outcomes. That is, for example, a factor that reduces baseline rates of 5% and 10% by equal proportionate amounts will necessarily increase the opposite outcome rates (95% and 90%) by different proportionate amounts. Since there is no more reason to regard it as normal that two groups will experience equal proportionate changes in one outcome than there is to regard it as normal that they will experience equal proportionate changes in the opposite outcome, there is no reason to regard it as normal that they will experience equal proportionate changes in either outcome. In fact there exist reasons, inherent in normal distributions, to expect that a factor that changes the prevalence of an outcome will have a larger proportionate effect on the group with the lower baseline rate and a larger proportionate effect on the opposite outcome on the other group. Recognition of the forces underlying these patterns is important for using a relative risk reduction observed as to one baseline rate in order to estimate the absolute risk reduction and number needed to treat in the case of other baseline rates (as reflected in Table 3 of reference 1).

    References:

    1. Subgroup Effects sub-page of Scanlan’s Rule page of jpscanlan.com: http://www.jpscanlan.com/scanlansrule/subgroupeffects.html

    2. Scanlan JP. Interpreting Differential Effects in Light of Fundamental Statistical Tendencies, presented at 2009 Joint Statistical Meetings of the American Statistical Association, International Biometric Society, Institute for Mathematical Statistics, and Canadian Statistical Society, Washington, DC, Aug. 1-6, 2009.PowerPointPresentation : http://www.jpscanlan.com/images/Scanlan_JSM_2009.ppt; Oral Presentation: http://www.jpscanlan.com/images/JSM_2009_ORAL.pdf

    3. Scanlan JP. Assessing heterogeneity of treatment effects in light of fundamental statistical tendencies. Trials May 26, 2011(responding to Kent DM, Rothwell PM, Ioannidis JPA, et al. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials 2010,11:85): http://www.trialsjournal.com/content/11/1/85/comments#498686

    • Ha-Vinh January 4, 2012 at 9:48 am #

      Dear Mr Scanlan

      I have just being reading with attention the content of your blog and it highly interested me. In France and in the medical field we more and more use the concept of number of patient being to treat to avoid one realization of the risk (e.g. one death) instead of the concept of relative risk for the reasons you very clearly exposed. Absolute difference of risk is also less misleading than relative risk, thought you seems disagree about that in your blog.

      Indeed, public policies stem too often from misleading statistical concepts which we, as physicians and scientists, fail to explain and that’s the reason why we are, in part, responsible for that.

      Thank you for your comments.

      Yours respectfully,

      Philippe Ha-Vinh

Trackbacks/Pingbacks

  1. Performance Measurement « Health Services Authors - December 19, 2011

    […] patient studied by the evidence-based medical research. In a precedent post I presented what the heterogeneity of treatment effect means. In the present post I will try to highlight where stands the fundamental difference between […]

  2. Performance Measurement – Converting Practice Guidelines Into Quality Measure « Health and Medical News and Resources - December 20, 2011

    […] patient studied by the evidence-based medical research. In a precedent post I presented what the heterogeneity of treatment effect means. In the present post I will try to highlight where stands the fundamental difference between […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: