SwampNut Posted July 19, 2023 Share Posted July 19, 2023 https://www.nature.com/articles/d41586-023-02299-w?utm_source=tldrnewsletter How many clinical-trial studies in medical journals are fake or fatally flawed? In October 2020, John Carlisle reported a startling estimate1. Carlisle, an anaesthetist who works for England’s National Health Service, is renowned for his ability to spot dodgy data in medical trials. He is also an editor at the journal Anaesthesia, and in 2017, he decided to scour all the manuscripts he handled that reported a randomized controlled trial (RCT) — the gold standard of medical research. Over three years, he scrutinized more than 500 studies1. For more than 150 trials, Carlisle got access to anonymized individual participant data (IPD). By studying the IPD spreadsheets, he judged that 44% of these trials contained at least some flawed data: impossible statistics, incorrect calculations or duplicated numbers or figures, for instance. And 26% of the papers had problems that were so widespread that the trial was impossible to trust, he judged — either because the authors were incompetent, or because they had faked the data. Carlisle called these ‘zombie’ trials because they had the semblance of real research, but closer scrutiny showed they were actually hollow shells, masquerading as reliable information. Even he was surprised by their prevalence. “I anticipated maybe one in ten,” he says. 1 Quote Link to comment Share on other sites More sharing options...
RXX Posted August 25, 2023 Share Posted August 25, 2023 On 7/19/2023 at 2:16 PM, SwampNut said: https://www.nature.com/articles/d41586-023-02299-w?utm_source=tldrnewsletter How many clinical-trial studies in medical journals are fake or fatally flawed? In October 2020, John Carlisle reported a startling estimate1. Carlisle, an anaesthetist who works for England’s National Health Service, is renowned for his ability to spot dodgy data in medical trials. He is also an editor at the journal Anaesthesia, and in 2017, he decided to scour all the manuscripts he handled that reported a randomized controlled trial (RCT) — the gold standard of medical research. Over three years, he scrutinized more than 500 studies1. For more than 150 trials, Carlisle got access to anonymized individual participant data (IPD). By studying the IPD spreadsheets, he judged that 44% of these trials contained at least some flawed data: impossible statistics, incorrect calculations or duplicated numbers or figures, for instance. And 26% of the papers had problems that were so widespread that the trial was impossible to trust, he judged — either because the authors were incompetent, or because they had faked the data. Carlisle called these ‘zombie’ trials because they had the semblance of real research, but closer scrutiny showed they were actually hollow shells, masquerading as reliable information. Even he was surprised by their prevalence. “I anticipated maybe one in ten,” he says. Go figure. makes sense if you think of studies like other manufactured items. What is the recall rate on the average automobile? Or how many times have you gotten an item that doesn't work? Probably not 44%, but there are lots and lots and lots of parts to studies produced by lots and lots of people of drastically varying competencies. Do you think every medical research facility or faculty has some super power to employ only the best and brightest? A study is only as good as the weakest link in the weakest citation in the study, which is be based on data from even more obscure citations. If you go down the citation research rabbit hole (I have done it just a few times in my life) you can do exactly what this guy did. It is daunting work, but pretty damned satisfying when you find the "AHA" buried in a second-or-third tier footnote. And of course we have repeated examples of top-tier researchers caught actively falsifying data and finally getting busted. Quote Link to comment Share on other sites More sharing options...
Zero Knievel Posted August 25, 2023 Share Posted August 25, 2023 So long as there’s an incentive to lie/cheat without serious consequences, it will happen. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.