Brazilian Journal of Pulmonology

ISSN (on-line): 1806-3756 | ISSN (printed): 1806-3713

SBPT

Publication continuous and bimonthly

SCImago Journal & Country Rank
Advanced Search

 

Current Issue: 2017 - Volume 43 - Number 3 (May/June)

CONTINUING EDUCATION : SCIENTIFIC METHODOLOGY

Subgroup analysis and interaction tests: why they are important and how to avoid common mistakes

Análise de subgrupos e testes de interação: por que são importantes e como evitar erros comuns

 

Juliana Carvalho Ferreira1; 2; Cecilia Maria Patino1; 3

 

1. Methods in Epidemiologic, Clinical, and Operations Research - MECOR - program, American Thoracic Society/Asociación Latinoamericana del Tórax, Montevideo, Uruguay.
2. Divisão de Pneumologia, Instituto do Coração - InCor - Hospital das Clínicas, Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brasil.
3. Department of Preventive Medicine, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.


 

 

 



A randomized clinical trial was conducted to compare the effect of vitamin C vs. placebo on improving pulmonary function in newborns of pregnant smokers; and to test if this effect differed by maternal genotype.(1) Vitamin C improved pulmonary function in newborns compared to placebo (TPTEF:TE ratio 0.383 vs 0.345; p = 0.006); and this effect was stronger in newborns with mothers with a specific genotype (p-interaction < 0.001).(1)

BACKGROUND

When conducting clinical trials, investigators examine the effect of interventions on outcomes in the study population and often in subgroups of patients defined by baseline characteristics (e.g., demographics, prognostic factors). The goal is to understand if the magnitude of the effect of the intervention differs within categories of a subgroup; in our example, genotype subgroups. If the effect is different within subgroups we call this effect modification of the intervention on the outcome due to the additional presence of the subgroup variable. We commonly conduct a test for interaction, using multivariable models, to evaluate for statistically significant subgroup differences. If the p value is significant, we conclude that the effect of the intervention on the outcome differs within subgroups, in our example, maternal genotype.

Understanding treatment effects across patient subgroups is important because it helps identify patient groups that respond better or worse to the intervention. However, subgroup analyses should be done with caution to avoid common mistakes that either lead to false negative or positive findings, especially when they are not pre-specified in the analysis plan before starting the study. A common mistake is to compare the effect of treatment on the outcome separately within each subgroup. For example, comparing the effect of vitamin C vs. placebo on pulmonary function in newborns among mothers with one genotype and then separately among the mothers with another genotype. This approach is incorrect because it leads to multiple testing, which means that instead of using only one calculation to test for differences in effect across subgroups (p for interaction across genotype-groups in our example), we use two or more different calculations for each subgroup analysis. Every time we add a calculation, we no longer can use the standard significant level of p < 0.05. In this case, since there are two calculations we would need to divide the p value by 2 and use p < 0.025 as the significance level.(2) Thus, we would overestimate subgroup differences if we kept the significance level at 0.05. Another challenge with subgroup analysis is that results may suggest that there are subgroup differences but the p-value is not statistically significant because the sample size within each subgroup is too small (Figure 1).

SUBGROUP ANALYSIS TIPS

1. Identify a few subgroups that seem highly relevant to your research question a priori and justify your choices.
2. Do not compare the effects of treatment vs. control in each subgroup. There are specific statistical tests to de-termine if there is an interaction between the treatment effect and the variables that define subgroups, which are best performed with the aid of a statistician.
3. Before making changes in clinical practice, subgroup results should be replicated in other studies.

REFERENCES

1. McEvoy CT, Schilling D, Clay N, Jackson K, Go MD, Spitale P, et al. Vitamin C supplementation for pregnant smoking women and pulmonary function in their newborn infants: a randomized clinical trial. JAMA. 2014;311(20):2074-82. https://doi.org/10.1001/jama.2014.5217
2. Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine--reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357(21):2189-94. https://doi.org/10.1056/NEJMsr077003

 

 


The Brazilian Journal of Pulmonology is indexed in:

Latindex Lilacs SciELO PubMed ISI Scopus Copernicus pmc

Support

CNPq, Capes, Ministério da Educação, Ministério da Ciência e Tecnologia, Governo Federal, Brasil, País Rico é País sem Pobreza
Secretariat of the Brazilian Journal of Pulmonology
SCS Quadra 01, Bloco K, Salas 203/204 Ed. Denasa. CEP: 70.398-900 - Brasília - DF
Fone/fax: 0800 61 6218/ (55) (61) 3245 1030/ (55) (61) 3245 6218
E-mails: jbp@jbp.org.br
jpneumo@jornaldepneumologia.com.br

Copyright 2019 - Brazilian Thoracic Association

Logo GN1