TY - JOUR A1 - Weibel, Stephanie A1 - Popp, Maria A1 - Reis, Stefanie A1 - Skoetz, Nicole A1 - Garner, Paul A1 - Sydenham, Emma T1 - Identifying and managing problematic trials: A research integrity assessment tool for randomized controlled trials in evidence synthesis JF - Research Synthesis Methods N2 - Evidence synthesis findings depend on the assumption that the included studies follow good clinical practice and results are not fabricated or false. Studies which are problematic due to scientific misconduct, poor research practice, or honest error may distort evidence synthesis findings. Authors of evidence synthesis need transparent mechanisms to identify and manage problematic studies to avoid misleading findings. As evidence synthesis authors of the Cochrane COVID-19 review on ivermectin, we identified many problematic studies in terms of research integrity and regulatory compliance. Through iterative discussion, we developed a research integrity assessment (RIA) tool for randomized controlled trials for the update of this Cochrane review. In this paper, we explain the rationale and application of the RIA tool in this case study. RIA assesses six study criteria: study retraction, prospective trial registration, adequate ethics approval, author group, plausibility of methods (e.g., randomization), and plausibility of study results. RIA was used in the Cochrane review as part of the eligibility check during screening of potentially eligible studies. Problematic studies were excluded and studies with open questions were held in awaiting classification until clarified. RIA decisions were made independently by two authors and reported transparently. Using the RIA tool resulted in the exclusion of >40% of studies in the first update of the review. RIA is a complementary tool prior to assessing “Risk of Bias” aiming to establish the integrity and authenticity of studies. RIA provides a platform for urgent development of a standard approach to identifying and managing problematic studies. KW - COVID-19 pandemic KW - systematic review KW - research integrity KW - randomized controlled trial KW - good clinical practice KW - evidence synthesis Y1 - 2023 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-318236 VL - 14 IS - 3 SP - 357 EP - 369 ER - TY - JOUR A1 - Schmid, Benedikt A1 - Griesel, Mirko A1 - Fischer, Anna-Lena A1 - Romero, Carolina S. A1 - Metzendorf, Maria-Inti A1 - Weibel, Stephanie A1 - Fichtner, Falk T1 - Awake prone positioning, high-flow nasal oxygen and non-invasive ventilation as non-invasive respiratory strategies in COVID-19 acute respiratory failure: a systematic review and meta-analysis JF - Journal of Clinical Medicine N2 - Background: Acute respiratory failure is the most important organ dysfunction of COVID-19 patients. While non-invasive ventilation (NIV) and high-flow nasal cannula (HFNC) oxygen are frequently used, efficacy and safety remain uncertain. Benefits and harms of awake prone positioning (APP) in COVID-19 patients are unknown. Methods: We searched for randomized controlled trials (RCTs) comparing HFNC vs. NIV and APP vs. standard care. We meta-analyzed data for mortality, intubation rate, and safety. Results: Five RCTs (2182 patients) were identified. While it remains uncertain whether HFNC compared to NIV alters mortality (RR: 0.92, 95% CI 0.65–1.33), HFNC may increase rate of intubation or death (composite endpoint; RR 1.22, 1.03–1.45). We do not know if HFNC alters risk for harm. APP compared to standard care probably decreases intubation rate (RR 0.83, 0.71–0.96) but may have little or no effect on mortality (RR: 1.08, 0.51–2.31). Conclusions: Certainty of evidence is moderate to very low. There is no compelling evidence for either HFNC or NIV, but both carry substantial risk for harm. The use of APP probably has benefits although mortality appears unaffected. KW - respiratory failure KW - non-invasive ventilation KW - high-flow nasal cannula KW - awake prone positioning KW - COVID-19 KW - systematic review Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-255225 SN - 2077-0383 VL - 11 IS - 2 ER - TY - JOUR A1 - Riemer, Manuel A1 - Kranke, Peter A1 - Helf, Antonia A1 - Mayer, Debora A1 - Popp, Maria A1 - Schlesinger, Tobias A1 - Meybohm, Patrick A1 - Weibel, Stephanie T1 - Trial registration and selective outcome reporting in 585 clinical trials investigating drugs for prevention of postoperative nausea and vomiting JF - BMC Anesthesiology N2 - Background: Selective outcome reporting in clinical trials introduces bias in the body of evidence distorting clinical decision making. Trial registration aims to prevent this bias and is suggested by the International Committee of Medical Journal Editors (ICMJE) since 2004. Methods: The 585 randomized controlled trials (RCTs) published between 1965 and 2017 that were included in a recently published Cochrane review on antiemetic drugs for prevention of postoperative nausea and vomiting were selected. In a retrospective study, we assessed trial registration and selective outcome reporting by comparing study publications with their registered protocols according to the ‘Cochrane Risk of bias’ assessment tool 1.0. Results: In the Cochrane review, the first study which referred to a registered trial protocol was published in 2004. Of all 585 trials included in the Cochrane review, 334 RCTs were published in 2004 or later, of which only 22% (75/334) were registered. Among the registered trials, 36% (27/75) were pro- and 64% (48/75) were retrospectively registered. 41% (11/27) of the prospectively registered trials were free of selective outcome reporting bias, 22% (6/27) were incompletely registered and assessed as unclear risk, and 37% (10/27) were assessed as high risk. Major outcome discrepancies between registered and published high risk trials were a change from the registered primary to a published secondary outcome (32%), a new primary outcome (26%), and different outcome assessment times (26%). Among trials with high risk of selective outcome reporting 80% favoured at least one statistically significant result. Registered trials were assessed more often as ‘overall low risk of bias’ compared to non-registered trials (64% vs 28%). Conclusions: In 2017, 13 years after the ICMJE declared prospective protocol registration a necessity for reliable clinical studies, the frequency and quality of trial registration in the field of PONV is very poor. Selective outcome reporting reduces trustworthiness in findings of clinical trials. Investigators and clinicians should be aware that only following a properly registered protocol and transparently reporting of predefined outcomes, regardless of the direction and significance of the result, will ultimately strengthen the body of evidence in the field of PONV research in the future. KW - clinical trial KW - postoperative nausea and vomiting KW - selective outcome reporting KW - systematic review KW - trial registration Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:20-opus-265518 VL - 21 ER -