Logistic Regression: Limitations in the Estimation of Measures of Association with Binary Health Outcomes
DOI:
https://doi.org/10.20344/amp.21435Keywords:
Logistic Models, Models, Statistical, Odds Ratio, Outcome Assessment, Health Care, Poisson DistributionAbstract
Introduction: Logistic regression models are frequently used to estimate measures of association between an exposure, health determinant or intervention, and a binary outcome. However, when the outcome is frequent (> 10%), model estimates for relative risks and prevalence ratios might be biased. Despite the availability of several alternatives, many still rely on these models, and a consensus is yet to be reached. We aimed to compare the estimation and goodness-of-fit of logistic, log-binomial and robust Poisson regression models, in cross-sectional studies involving frequent binary outcomes.
Methods: Two cross-sectional studies were conducted. Study 1 was a nationally representative study on the impact of air pollution on mental health. Study 2 was a local study on immigrants’ access to urgent healthcare services. Odds ratios (OR) were obtained through logistic regression, and prevalence ratios (PR) through log-binomial and robust Poisson regression models. Confidence intervals (CI), their ranges, and standard-errors (SE) were also computed, along with models’ relative goodness-of-fit through Akaike Information Criterion (AIC), when applicable.
Results: In Study 1, the OR (95% CI) was 1.015 (0.970 - 1.063), while the PR (95% CI) obtained through the robust Poisson mode was 1.012 (0.979 - 1.045). The log-binomial regression model did not converge in this study. In Study 2, the OR (95% CI) was 1.584 (1.026 - 2.446), the PR (95% CI) for the log-binomial model was 1.217 (0.978 - 1.515), and 1.130 (1.013 - 1.261) for the robust Poisson model. The 95% CI, their ranges, and the SE of the OR were higher than those of the PR, in both studies. However, in Study 2, the AIC value was lower for the logistic regression model.
Conclusion: The odds ratio overestimated PR with wider 95% CI and higher SE. The overestimation was greater as the outcome of the study became more prevalent, in line with previous studies. In Study 2, the logistic regression was the model with the best fit, illustrating the need to consider multiple criteria when selecting the most appropriate statistical model for each study. Employing logistic regression models by default might lead to misinterpretations. Robust Poisson models are viable alternatives in cross-sectional studies with frequent binary outcomes, avoiding the non-convergence of log-binomial models.
Downloads
References
Hosmer Jr, Lemeshow S, Sturdivant RX. Applied logistic regression. 3rd ed. Hoboken: John Wiley & Sons; 2013.
Lash TL, VanderWeele TJ, Haneuse S, Rothman KJ. Modern epidemiology. Philadelphia: Lippincott Williams & Wilkins; 2020.
Celentano D, Szklo M. Gordis epidemiology. Amsterdam: Elsevier; 2020.
Tamhane AR, Westfall AO, Burkholder GA, Cutter GR. Prevalence odds ratio versus prevalence ratio: choice comes with consequences. Stat Med. 2016;35:5730-5.
Knol MJ, Le Cessie S, Algra A, Vandenbroucke JP, Groenwold RH. Overestimation of risk ratios by odds ratios in trials and cohort studies: alternatives to logistic regression. CMAJ. 2012;184:895-9.
Petersen MR, Deddens JA. A comparison of two methods for estimating prevalence ratios. BMC Med Res Methodol. 2008;8:9.
McCullagh P. Generalized linear models. New York: Routledge; 2019.
Barros AJ, Hirakata VN. Alternatives for logistic regression in crosssectional studies: an empirical comparison of models that directly estimate the prevalence ratio. BMC Med Res Methodol. 2003;3:21.
Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. New York: John Wiley & Sons; 2003.
Cavanaugh JE, Neath AA. The Akaike information criterion: background, derivation, properties, application, interpretation, and refinements. WIREs Comp Stats. 2019;11:e1460.
Manisalidis I, Stavropoulou E, Stavropoulos A, Bezirtzoglou E. Environmental and health impacts of air pollution: a review. Front Public Health. 2020;8:14.
World Health Organization. WHO global air quality guidelines: particulate matter (PM2.5 and PM10), ozone, nitrogen dioxide, sulfur dioxide and carbon monoxide. 2021.[cited 2023 Aug 06]. Available from: https://iris.who.int/bitstream/handle/10665/345329/9789240034228- eng.pdf?sequence=1&isAllowed=y.
Braithwaite I, Zhang S, Kirkbride JB, Osborn DPJ, Hayes JF. Air pollution (particulate matter) exposure and associations with depression, anxiety, bipolar, psychosis and suicide risk: a systematic review and metaanalysis. Environ Health Perspect. 2019;127:126002.
Borroni E, Pesatori AC, Bollati V, Buoli M, Carugno M. Air pollution exposure and depression: a comprehensive updated systematic review and meta-analysis. Environ Pollut. 2022;292:118245.
Zeng Y, Lin R, Liu L, Liu Y, Li Y. Ambient air pollution exposure and risk of depression: a systematic review and meta-analysis of observational studies. Psychiatry Res. 2019;276:69-78.
Liu Q, Wang W, Gu X, Deng F, Wang , Lin H, et al. Association between particulate matter air pollution and risk of depression and suicide: a systematic review and meta-analysis. Environ Sci Pollut Res Int. 2021;28:9029-49.
Trushna T, Dhiman V, Raj D, Tiwari RR. Effects of ambient air pollution on psychological stress and anxiety disorder: a systematic review and meta-analysis of epidemiological evidence. Rev Environ Health. 2021;36:501-21.
Cuijpers P, Miguel C, Ciharova M, Kumar M, Brander L, Kumar P, et al. Impact of climate events, pollution, and green spaces on mental health: an umbrella review of meta-analyses. Psychol Med. 2023;53:638-53.
Stewart AL, Greenfield S, Hays RD, Wells K, Rogers WH, Berry SD, et al. Functional status and well-being of patients with chronic conditions. Results from the Medical Outcomes Study. JAMA. 1989;262:907-13.
Stewart AL, Hays RD, Ware JE, Jr. The MOS short-form general health survey. Reliability and validity in a patient population. Med Care. 1988;26:724-35.
Berwick DM, Murphy JM, Goldman PA, Ware JE Jr., Barsky AJ, Weinstein MC. Performance of a five-item mental health screening test. Med Care. 1991;29:169-76.
Nunes B, Barreto M, Gil AP, Kislaya I, Namorado S, Antunes L, et al. The first Portuguese national health examination survey (2015): design, planning and implementation. J Public Health. 2019;41:511-7.
Gaio V, Dias CM. Association between ambient air pollution exposure and biomarkers of cardiovascular risk: link between the first Portuguese health examination survey and the air quality data. [cited 2023 Aug 12]. Available from: https://run.unl.pt/handle/10362/143849.
Gaio V, Roquette R, Monteiro A, Ferreira J, Rafael S, Dias CM, et al. Exposure to ambient particulate matter increases blood count parameters with potential to mediate a cardiovascular event: results from a population-based study in Portugal. Air Qual Atmos Hlth. 2021;14:1189-202.
Gaio V, Roquette R, Monteiro A, Ferreira J, Matias Dias C, Nunes B. Investigating the association between ambient particulate matter (PM(10)) exposure and blood pressure values: Results from the link between the Portuguese Health Examination Survey and air quality data. Rev Port Cardiol. 2023;42:251-8.
Gaio V, Roquette R, Monteiro A, Ferreira J, Lopes D, Dias CM, et al. PM10 exposure interacts with abdominal obesity to increase blood triglycerides: a cross-sectional linkage study. Eur J Public Health. 2022;32:281-8.
Ribeiro AI, Launay L, Guillaume E, Launoy G, Barros H. The Portuguese version of the European Deprivation Index: development and association with all-cause mortality. PLoS One. 2018;13:e0208320.
Instituto Nacional de Estatística. Censos 2021 dissemination platform (definitive results). [cited 2023 Sep 10]. Available from: https://censos. ine.pt/xportal/xmain?xpgid=censos21_produtos&xpid=CENSOS21&xlang=ptdb_censos_2021.html.
Buja A, Fusco M, Furlan P, Bertoncello C, Baldovin T, Casale P, et al. Characteristics, processes, management and outcome of accesses to accident and emergency departments by citizenship. Int J Public Health. 2014;59:167-74.
Muggli Z, Mertens T, Amado R, Teixeira AL, Vaz D, Pires M, et al. Cohort profile: health trajectories of immigrant children (CRIAS)-a prospective cohort study in the metropolitan area of Lisbon, Portugal. BMJ Open. 2022;12:e061919.
Graetz V, Rechel B, Groot W, Norredam M, Pavlova M. Utilization of health care services by migrants in Europe-a systematic literature review. Br Med Bull. 2017;121:5-18.
Norredam M, Krasnik A, Moller Sorensen T, Keiding N, Joost Michaelsen J, Sonne Nielsen A. Emergency room utilization in Copenhagen: a comparison of immigrant groups and Danish-born residents. Scand J Public Health. 2004;32:53-9.
Ruud SE, Hjortdahl P, Natvig B. Reasons for attending a general emergency outpatient clinic versus a regular general practitioner - a survey among immigrant and native walk-in patients in Oslo, Norway. Scand J Prim Health Care. 2017;35:35-45.
Gimeno-Feliu LA, Pastor-Sanz M, Poblador-Plou B, Calderon-Larranaga A, Diaz E, Prados-Torres A. Overuse or underuse? Use of healthcare services among irregular migrants in a north-eastern Spanish region. Int J Equity Health. 2021;20:41.
Schoevers MA, Loeffen MJ, van den Muijsenbergh ME, Lagro-Janssen AL. Health care utilisation and problems in accessing health care of female undocumented immigrants in the Netherlands. Int J Public Health. 2010;55:421-8.
Gimeno-Feliu LA, Calderon-Larranaga A, Diaz E, Poblador-Plou B, Macipe-Costa R, Prados-Torres A. Global healthcare use by immigrants in Spain according to morbidity burden, area of origin, and length of stay. BMC Public Health. 2016;16:450.
Dias S, Gama A, Cortes M, de Sousa B. Healthcare-seeking patterns among immigrants in Portugal. Health Soc Care Community. 2011;19:514-21.
Sauzet O, David M, Naghavi B, Borde T, Sehouli J, Razum O. Adequate utilization of emergency services in Germany: is there a differential by migration background? Front Public Health. 2020;8:613250.
Tzogiou C, Boes S, Brunner B. What explains the inequalities in health care utilization between immigrants and non-migrants in Switzerland? BMC Public Health. 2021;21:530.
Coutinho LM, Scazufca M, Menezes PR. Methods for estimating prevalence ratios in cross-sectional studies. Rev Saude Publica. 2008;42:992-8.
Lee J, Tan CS, Chia KS. A practical guide for multivariate analysis of dichotomous outcomes. Ann Acad Med Singap. 2009;38:714-9.
Behrens T, Taeger D, Wellmann J, Keil U. Different methods to calculate effect estimates in cross-sectional studies. A comparison between prevalence odds ratio and prevalence ratio. Methods Inf Med. 2004;43:505-9.
Martinez BA, Leotti VB, Silva GS, Nunes LN, Machado G, Corbellini LG. Odds ratio or prevalence ratio? An overview of reported statistical methods and appropriateness of interpretations in cross-sectional studies with dichotomous outcomes in veterinary medicine. original research. Front Vet Sci. 2017;4:193.
Gnardellis C, Notara V, Papadakaki M, Gialamas V, Chliaoutakis J. Overestimation of relative risk and prevalence ratio: misuse of logistic modeling. Diagnostics. 2022;12:2851.
Williamson T, Eliasziw M, Fick GH. Log-binomial models: exploring failed convergence. Emerg Themes Epidemiol. 2013;10:14.
Reichenheim ME, Coutinho ES. Measures and models for causal inference in cross-sectional studies: arguments for the appropriateness of the prevalence odds ratio and related logistic regression. BMC Med Res Methodol. 2010;10:66.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Acta Médica Portuguesa
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
All the articles published in the AMP are open access and comply with the requirements of funding agencies or academic institutions. The AMP is governed by the terms of the Creative Commons ‘Attribution – Non-Commercial Use - (CC-BY-NC)’ license, regarding the use by third parties.
It is the author’s responsibility to obtain approval for the reproduction of figures, tables, etc. from other publications.
Upon acceptance of an article for publication, the authors will be asked to complete the ICMJE “Copyright Liability and Copyright Sharing Statement “(http://www.actamedicaportuguesa.com/info/AMP-NormasPublicacao.pdf) and the “Declaration of Potential Conflicts of Interest” (http:// www.icmje.org/conflicts-of-interest). An e-mail will be sent to the corresponding author to acknowledge receipt of the manuscript.
After publication, the authors are authorised to make their articles available in repositories of their institutions of origin, as long as they always mention where they were published and according to the Creative Commons license.
