Believability of relative risks and odds ratios in abstracts: cross sectional study.
P. Gøtzsche. BMJ (Clinical research ed.), 333 (7561):
231-4(July 2006)4383<m:linebreak></m:linebreak>LR: 20071115; PUBM: Print-Electronic; DEP: 20060719; JID: 8900488; PMC1523498; 2006/07/19 aheadofprint; ppublish;<m:linebreak></m:linebreak>Presentació de dades.
DOI: 10.1136/bmj.38895.410451.79
Abstract
OBJECTIVE: To compare the distribution of P values in abstracts of randomised controlled trials with that in observational studies, and to check P values between 0.04 and 0.06. DESIGN: Cross sectional study of all 260 abstracts in PubMed of articles published in 2003 that contained "relative risk" or ödds ratio" and reported results from a randomised trial, and random samples of 130 abstracts from cohort studies and 130 from case-control studies. P values were noted or calculated if unreported. MAIN OUTCOME MEASURES: Prevalence of significant P values in abstracts and distribution of P values between 0.04 and 0.06. RESULTS: The first result in the abstract was statistically significant in 70% of the trials, 84% of cohort studies, and 84% of case-control studies. Although many of these results were derived from subgroup or secondary analyses, or biased selection of results, they were presented without reservations in 98% of the trials. P values were more extreme in observational studies (P < 0.001) and in cohort studies than in case-control studies (P = 0.04). The distribution of P values around P = 0.05 was extremely skewed. Only five trials had 0.05 < or = P < 0.06, whereas 29 trials had 0.04 < or = P < 0.05. I could check the calculations for 27 of these trials. One of four non-significant results was significant. Four of the 23 significant results were wrong, five were doubtful, and four could be discussed. Nine cohort studies and eight case-control studies reported P values between 0.04 and 0.06, but in all 17 cases P < 0.05. Because the analyses had been adjusted for confounders, these results could not be checked. CONCLUSIONS: Significant results in abstracts are common but should generally be disbelieved.
%0 Journal Article
%1 Goetzsche2006
%A Gøtzsche, Peter C
%D 2006
%J BMJ (Clinical research ed.)
%K AbstractingandIndexingasTopic AbstractingandIndexingasTopic:standards AbstractingandIndexingasTopic:statistics&nu Case-ControlStudies CohortStudies Cross-SectionalStudies OddsRatio RandomizedControlledTrialsasTopic:statistics RiskAssessment RCT
%N 7561
%P 231-4
%R 10.1136/bmj.38895.410451.79
%T Believability of relative risks and odds ratios in abstracts: cross sectional study.
%U http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1523498&tool=pmcentrez&rendertype=abstract
%V 333
%X OBJECTIVE: To compare the distribution of P values in abstracts of randomised controlled trials with that in observational studies, and to check P values between 0.04 and 0.06. DESIGN: Cross sectional study of all 260 abstracts in PubMed of articles published in 2003 that contained "relative risk" or ödds ratio" and reported results from a randomised trial, and random samples of 130 abstracts from cohort studies and 130 from case-control studies. P values were noted or calculated if unreported. MAIN OUTCOME MEASURES: Prevalence of significant P values in abstracts and distribution of P values between 0.04 and 0.06. RESULTS: The first result in the abstract was statistically significant in 70% of the trials, 84% of cohort studies, and 84% of case-control studies. Although many of these results were derived from subgroup or secondary analyses, or biased selection of results, they were presented without reservations in 98% of the trials. P values were more extreme in observational studies (P < 0.001) and in cohort studies than in case-control studies (P = 0.04). The distribution of P values around P = 0.05 was extremely skewed. Only five trials had 0.05 < or = P < 0.06, whereas 29 trials had 0.04 < or = P < 0.05. I could check the calculations for 27 of these trials. One of four non-significant results was significant. Four of the 23 significant results were wrong, five were doubtful, and four could be discussed. Nine cohort studies and eight case-control studies reported P values between 0.04 and 0.06, but in all 17 cases P < 0.05. Because the analyses had been adjusted for confounders, these results could not be checked. CONCLUSIONS: Significant results in abstracts are common but should generally be disbelieved.
%@ 1468-5833
@article{Goetzsche2006,
abstract = {OBJECTIVE: To compare the distribution of P values in abstracts of randomised controlled trials with that in observational studies, and to check P values between 0.04 and 0.06. DESIGN: Cross sectional study of all 260 abstracts in PubMed of articles published in 2003 that contained "relative risk" or "odds ratio" and reported results from a randomised trial, and random samples of 130 abstracts from cohort studies and 130 from case-control studies. P values were noted or calculated if unreported. MAIN OUTCOME MEASURES: Prevalence of significant P values in abstracts and distribution of P values between 0.04 and 0.06. RESULTS: The first result in the abstract was statistically significant in 70% of the trials, 84% of cohort studies, and 84% of case-control studies. Although many of these results were derived from subgroup or secondary analyses, or biased selection of results, they were presented without reservations in 98% of the trials. P values were more extreme in observational studies (P < 0.001) and in cohort studies than in case-control studies (P = 0.04). The distribution of P values around P = 0.05 was extremely skewed. Only five trials had 0.05 < or = P < 0.06, whereas 29 trials had 0.04 < or = P < 0.05. I could check the calculations for 27 of these trials. One of four non-significant results was significant. Four of the 23 significant results were wrong, five were doubtful, and four could be discussed. Nine cohort studies and eight case-control studies reported P values between 0.04 and 0.06, but in all 17 cases P < 0.05. Because the analyses had been adjusted for confounders, these results could not be checked. CONCLUSIONS: Significant results in abstracts are common but should generally be disbelieved.},
added-at = {2023-02-03T11:44:35.000+0100},
author = {Gøtzsche, Peter C},
biburl = {https://www.bibsonomy.org/bibtex/2fab0eb7d899574ce92135a44a9e46258/jepcastel},
city = {Nordic Cochrane Centre, H:S Rigshospitalet, DK-2100 Copenhagen O, Denmark,. pcg@cochrane.dk},
doi = {10.1136/bmj.38895.410451.79},
interhash = {a5e89dce58c411e7a4f7e125919b0d7f},
intrahash = {fab0eb7d899574ce92135a44a9e46258},
isbn = {1468-5833},
issn = {1756-1833},
journal = {BMJ (Clinical research ed.)},
keywords = {AbstractingandIndexingasTopic AbstractingandIndexingasTopic:standards AbstractingandIndexingasTopic:statistics&nu Case-ControlStudies CohortStudies Cross-SectionalStudies OddsRatio RandomizedControlledTrialsasTopic:statistics RiskAssessment RCT},
month = {7},
note = {4383<m:linebreak></m:linebreak>LR: 20071115; PUBM: Print-Electronic; DEP: 20060719; JID: 8900488; PMC1523498; 2006/07/19 [aheadofprint]; ppublish;<m:linebreak></m:linebreak>Presentació de dades},
number = 7561,
pages = {231-4},
pmid = {16854948},
timestamp = {2023-05-04T08:59:38.000+0200},
title = {Believability of relative risks and odds ratios in abstracts: cross sectional study.},
url = {http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1523498&tool=pmcentrez&rendertype=abstract},
volume = 333,
year = 2006
}