by Michael Turlik, DPM1 
The Foot and Ankle Online Journal 2 (8): 5
The author discusses the influence of industry funding on randomized controlled trials as well as, conflict of interest and misleading claims in medical research. Proposed solutions are given to include clinical trial registration and how to critically analyze an industry sponsored publication for misleading claims.
Key words: Evidence-based medicine, industry sponsored trials
This is an Open Access article distributed under the terms of the Creative Commons Attribution License. It permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. ©The Foot and Ankle Online Journal (www.faoj.org)
Accepted: July, 2009
Published: August, 2009
ISSN 1941-6806
doi: 10.3827/faoj.2009.0208.0005
Do industry sponsored trials produce pro industry results?
Pharmaceutical and device manufacturers regularly fund clinical trials to determine treatment efficacy. Are these sponsored trials designed to satisfy the marketing needs of the industry or to answer important clinical questions which will improve patient outcomes in healthcare? There exist incentives to industry funders to report clinically important results and conceal results which are not statistically significant. Industry sponsored studies are usually well planned and executed to minimize the conventional aspects of bias seen in randomized controlled trials (RCTs) (randomization, concealment allocation, blinding, intention to treat analysis). It is important to consider the possibility of different causes of bias when interpreting the results of industry sponsored studies.
In a study published in the British Medical Journal (BMJ) in August 2002, Kjaergard, and AlsNielsen looked at the conclusions of the trial’s authors and competing interests in RCTs published in the BMJ from 1997 through 2001. [1]
They found that the author’s conclusions were more likely to favor the experimental intervention in trials funded by pharmaceutical companies and device manufacturers as compared to non-industry trials. This result was statistically significant and trial quality was not an explanation for this relationship. In addition, other competing interests such as: personal, academic or political considerations did not affect the authors conclusions. The authors of the study concluded that industry sponsored trials tend to draw pro industry conclusions (sponsorship bias). In a follow-up study Als-Nielsen, et al., [2] reviewed 370 randomized controlled drug trials randomly selected from the Cochran database. It was found that trials funded by ‘for-profit’ organizations were significantly more likely to recommend the experimental drug as a treatment of choice as compared to non-industry trials. This relationship did not appear to be associated with treatment efficacy or adverse effects described in the primary studies. The authors concluded that trials funded by for-profit drug manufacturers may be more positive due to biased interpretation of the results “spin”.
Lexchin and colleagues performed a systematic review of drug studies to see if studies sponsored by the pharmaceutical industry generate pro-industry conclusions and if the methodology differs from trials not funded by the pharmaceutical industry. [3] Consistent with the earlier studies referenced the authors found that pharmaceutical industry sponsored studies tended to report positive results for the sponsor’s drug. Their explanation is that this is due to an inadequate comparator in the trial design and research funded by drug companies was less likely to be published than research funded by other sources (publication bias). Regarding study quality, industry sponsored studies were as good as or better than nonindustry sponsored studies in controlling for common sources of bias.
Bekelman and colleagues performed a systematic review to look at financial conflicts of interest in medical research. [4] They were able to demonstrate a statistically significant relationship between industry funded studies and pro-industry results. Industry sponsored trials were associated with restrictions regarding publication and data sharing. In other words, industry sponsored studies were less transparent than non-industry sponsored trials.
Bhandari and colleagues looked at the association between industry funding and the statistical significance of results in medical and surgical trials. [5] They reviewed 332 randomized controlled trials and they concluded that industry-funded trials were more likely to be associated with statistically significant pro-industry findings in both the medical and surgical trials. Leopold and colleagues looked at the relationship between funding and orthopedic studies. [6] They reviewed the 315 studies published in three different orthopedic journals and found that industry sponsored studies were significantly more likely to report a positive outcome than studies which did not receive industry funding. Another study evaluated the abstracts of podium presentations from (2001 – 2002) at the annual meetings of the American Academy of Orthopedic Surgeons. [7] Forty percent of the presentations reported some conflict of interest; positive findings were more common in those studies which reported a conflict of interest.
Conflict of Interest
Conflict of interest can be defined as behaviors which potentially may interfere with the professional judgment concerning patient care, clinical practice and medical research. Based upon the preceding section industry funded research tends to produce pro-industry results in medical research. Therefore, it is important for the authors of any studies to disclose clearly any potential conflicts of interest arising from drug and device manufacturers in the planning, execution and reporting of clinical trials. Failure to disclose conflict of interest should be considered a form of scientific misconduct.
Campbell and colleagues looked at the relationship between academic institutions and industry. [8] Sixty percent of department chairs and 67% of their departments had some personal relationship with industry. The authors believe it is important that these relationships be disclosed during publication of research findings and actively managed at the institutional level. A study of funding of clinical biomedical research in the United States over a ten-year period (1994 – 2004) revealed that funding by the federal government was unchanged over this period of time. [9] However, funding from industry increased from $4 billion to $14.2 billion. Patsopoulos, et al., in a database analysis from 1994 through 2003 looked at the funding pattern of the most frequently cited papers in medicine. [10] At least 76% of the papers cited had one author with an academic affiliation from a University. The funding from industry increased significantly over this time frame. They conclude that academia may be losing control over the clinical research agenda.
Lurie, et al., in a study of conflict of interest of advisory panel members to the FDA it was found that at 73% of the meetings at least one member disclosed a conflict of interest. [11] Only 1% of the members were recused from voting. However, excluding voting members with a conflict of interest would not have changed the overall vote outcome at any meeting studied.
Ghost management of a clinical trial can be defined as: a research study which has been controlled and shaped by the pharmaceutical industry or device manufacturers at any of the following levels research, analysis, writing and publication of the trial. These trials are then published under the names of leading academics which have little to do with the study. [12] It is difficult to determine how prevalent this is in the medical literature. In a recent article published in Journal of American Medical Association [13] the authors allege that published studies which minimized the risks of Rofecoxib were ghost written by employees of Merk. This was based upon review of court documents obtained by the authors during litigation of the company by patients who experienced adverse effects from the use of Rofecoxib. After careful review of the documents the authors conclusions were that the trials were prepared by employees of the company but first authorship was granted to affiliated academic physicians who sometimes failed to disclose industry financial support.
Fraud in medical research is uncommon. In 2008 of the August edition of the Journal of Bone Joint Surgery (British) [14] an article was published evaluating the usefulness of infuse in the healing tibial fractures. Infuse is a product of Medtronic. The article was with withdrawn by the Journal [15] in March of the following year. It is alleged that the senior author fraudulently included co-authors and data in the development of the study. [16] The senior author was a paid consultant of Medtronics.
Misleading Claims in Medical Research
It is accepted that selection of primary and secondary outcomes as well subgroup analysis should be carried out by the clinical researchers a priori. Reviewing the data post hoc to find statistically significant results is termed data dredging. [17] Multiple analyses of the data results in false positives or type I errors and the reduction in power for analysis of secondary outcomes. Chan and colleagues studied the extent and nature of outcome reporting bias in a cohort of randomized controlled trials. [18]
They compared the protocols relative to the published articles and compared and contrasted the study outcomes. Fifty percent of the efficacy outcomes and 65% of the harm outcomes were incompletely reported. Statistically significant outcomes were more likely to be reported than non-significant outcomes. Sixty two percent of the studies had one primary outcome changed, introduced or omitted. Eighty six percent of the survey responders denied the existence of unreported outcomes despite clear evidence to the contrary. The authors suggest that trial registration listing the outcomes studied prior to the beginning of the study would help with transparency and avoid post hoc data dredging for statistically significant results.
Composite outcomes are defined as a single outcome measure which is used to capture multiple usually adverse endpoints. [19] They are used primarily in drug trials as primary outcome measures to provide increased statistical efficiency. The disadvantage is that the multiple endpoints may not be similar in terms of patient importance, treatment effect and event rate. The more dissimilar the multiple endpoints are the less valid this measure is in terms of determining treatment efficacy. The authors suggest that the reader carefully review composite endpoints to assure that this is a valid measure of treatment effect otherwise; the treatment effect may be overstated.
When RCTs detect a large treatment benefit from the intervention being studied early in the trial it may be stopped earlier than planned. A recent review regarding RCTs which have been stopped early revealed that this practice is becoming more common in the literature. [20] The problem is that few trials adequately report the reasons for stopping the trial early, exhibit implausibly large treatment effects some based upon very small event rates. Most of the trials in the study which were stopped early for benefit were sponsored by for-profit corporations. It is thought that stopping a trial early leads to an overestimate of the treatment effect.
Clinical Trial Registration
In an attempt to prevent reporting bias (publication bias, selective outcome reporting bias) it has been suggested that journals only publish RCTs which have been registered in a clinical trial registry prior to the onset of the study. Each study will have a unique identifier which can be used by interested parties to verify when the trial was started if or when the trial was finished, who were the principal investigators and if the outcomes have been changed since the beginning of the trial. [21] An area of current concern regarding the databases is the lack of results regarding treatment efficacy and harms available from registered trials. [22] It is thought that having information reported regarding the results of preliminary or unpublished trials would be helpful in preparing a systematic review/meta-analysis.
Detecting Misleading Claims in Clinical Trials
Industry funding of clinical trials has been increasing and is associated with many different specialties to include orthopedics. It has been shown that industry funded trials are associated with pro-industry results (sponsorship bias). The current remedies include financial disclosure and trial registration. As noted earlier trials sponsored by industry are usually well-planned and carefully executed to minimize the possibility of bias. They usually have high internal validity. Despite this they may produce misleading claims which will lead to an overestimation of treatment effect. Montori and colleagues report their recommendations regarding evaluating clinical trials for misleading claims. [23] A summary of their recommendations are the following:
1. Read the methods and results sections of the trial only, avoid reading the discussion section.
2. Read the abstract of the article published in pre-appraised journals.
3. Evaluate critically the comparator to determine if it was appropriate.
4. If the primary outcome is a composite outcome critically evaluate its appropriateness.
5. Cautiously evaluate small reported treatment effects to determine clinical significance.
6. Avoid placing too much significance in subgroup analysis.
Malay, et al., [24] in an industry sponsored RCT compared nail debridement with nail debridement and topical antifungal nail lacquer for the treatment of onychomycosis. The primary outcome was mycological cure. The intervention demonstrated a greater than 76% mycological cure, while nail debridement resulted in a 0% mycological cure rate.
The authors in the introduction state the following regarding nail debridement and onychomycosis “it is not generally undertaken with an eye to curing the infection”. The podiatric physician will need to decide whether comparing nail debridement and nail debridement with topical antifungal nail lacquer is a fair comparison when using mycological cure as the primary outcome. If the primary outcome was mycological cure would a better comparator be an oral antifungal medication using a double dummy method?
In addition, the podiatric physician will need to decide whether mycological cure is an important outcome for patients. The secondary outcome in the study was the Bristol Foot Score a validated patient reported outcome. Would this have made a better primary outcome for the study? It is important to note that at the beginning of the study the median Bristol Foot Score was worse for the control group than the intervention group despite similar ranges. This imbalance is either the result of a problem with the randomization of the groups or is due to chance. It is unclear if this difference is clinically important. [25] This difference should be taken into account when comparing the groups for statistical significance at the end of the trial. [26]
FIELD study [27] was an industry sponsored study which was designed to evaluate the long-term cardiovascular events of fenofibrate. The study was a randomized controlled trial comparing fenofibrate to a placebo in 9,795 participants with diabetes mellitus who were followed for five years. The authors concluded that fenofibrate did not significantly reduce the risk of the primary outcome.
Using the data from the FIELD study the authors published another study in 2009 comparing the amputation rates in diabetic participants between the two groups. [28] The authors state that the amputation rates were a pre-planned tertiary outcome of the original study. Reviewing the trial registration (ISRCTN 64783481) for the study, pre-planned primary outcomes could be found, secondary outcomes were not provide at the time of registration and no tertiary outcomes were mentioned. Secondary and tertiary were described in the original article. [27] The authors conclude that “treatment with fenofibrate was associated with a lower risk of amputations, particularly minor amputations without known large-vessel disease.”
Using the data provided in the study a 2X2 table can be constructed. (Table 1) Using the data from the table and an online calculator 29 the absolute risk reduction (ARR) and number needed to treat (NNT) with 95% confidence intervals can be computed. (Table 2)
Table 1 Amputations without known large vessel disease.
Table 2 Clinical results of amputation (CI – confidence interval).
The absolute risk reduction is less than 1% even with the most optimistic estimate. Three hundred and seven patients would need to be treated for five years to prevent one additional minor amputation and it may be as high as 2,600. The podiatric physician will need to decide if the decrease in minor amputations in the study is clinically significant. Is the decrease in amputations worth the cost of the medication and the risks of adverse effects from fenofibrate?
Using the information provided in the original article, the risk of [28] serious adverse effects from using fenofibrate can be assessed. A 2 x 2 table for the occurrence of pancreatitis and pulmonary embolus can be constructed. (Tables 3 and 4)
Table 3 Occurrence of pancreatitis.
Table 4 Occurrence of pulmonary embolus.
Using the data from the constructed tables and an online calculator 29 the absolute risk increase (ARI) and the number needed to harm (NNH) with 95% confidence intervals can be calculated (Tables 5 and 6).
The risk of developing either a pulmonary embolus or pancreatitis from fenofibrate is about the same as preventing a minor amputation. The podiatric physician will need to determine if the preventing one minor amputation is worth producing one episode of pancreatitis and /or pulmonary embolus.
Table 5 Pancreatitis.
Table 6 Pulmonary embolus.
References
1. Kjaergard L, Als-Nielsen B: Association between competing interests and authors’ conclusions: epidemiological study of randomized controlled trials published in the BMJ 325: 249, 2002.
2. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL: Association of funding and conclusions in randomized drug trials: A reflection of treatment effect or adverse events? JAMA 290: 921 – 928, 2003.
3. Lexchin J, Bero LA, Djulbegovic B, Clark O: Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326: 1167 – 1170, 2003.
4. Bekelman, JE, Li, Y, Gross, CP: Scope and Impact of Financial Conflicts of Interest in Biomedical Research a systematic review. JAMA 289: 454 – 465, 2003.
5. Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S, Mears D, Schemitsch EH, Heels-Ansdell D, Devereaux PJ: Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 170 (4): 477 – 480, 2004.
6. Leopold SS, Warme W, Fritz Braunlich E, Shott S: Association between funding source and study outcome in orthopaedic research. CORR 415: 293 – 301, 2003.
7. Okike K, Kocher MS, Mehlman CT, Heckman JD, Bhandari M: Conflict of interest in orthopedic research. JBJS 89A: 608 – 613, 2007.
8. Campbell EG, Weissman JS, Ehringhaus S, Rao SR, Moy B, Feibelmann S, Dorr Goold S: Institutional academic industry relationships. JAMA 298: 1779 – 1786, 2007.
9. Moses H, Dorsey ER, Matheson DHM, Their SO: Financial Anatomy of Biomedical Research. JAMA 294: 1333 – 1342, 2005.
10. Patsopoulos NA, Ioannidis JPA, Analatos A: Origin and funding of the most frequently cited papers in medicine: database analysis. BMJ 332: 1061 – 1064, 2006.
11. Lurie P, Almeida C, Stine N, Stine A, Wolfe S. Financial conflict of interest disclosure and voting patterns at food and drug administration drug advisory committee meetings. JAMA 295:1921 – 1928, 2006.
12. Sismondo SL: Ghost management: How much of the medical literature is shaped behind the scenes by the pharmaceutical industry? PLoS Medicine 4: 1429 – 1433, 2007.
13. Ross JS, Hill KP, Egilman DS, Krumholz HM:
Guest Authorship and Ghostwriting in Publications Related to Rofecoxib. JAMA 299: 1800 – 1812, 2008.
14. Kuklo TR, Groth AT, Anderson RC, Frisch HM, Islinger RB: Recombinant human bone morphogenetic protein-2 for grade III open segmental tibial fractures from combat injuries in Iraq. J Bone Joint Surg 90B: 1068 – 1072, 2008.
15. Scott J: Editorial – Withdrawal of a paper. J Bone Joint Surg 91B: 285 – 286, 2009.
16. Meier B, Wilson D: Discredited Research Study Stuns an Ex-Army Doctor’s Colleagues. NY Times June 5, 2009.
17. Lord S, Gebski V, Keech A: Multiple analysis in clinical trials: sound science or data dredging? MJA 181: 452 – 454, 2004.
18. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG: Empirical evidence for selective reporting of outcomes in randomized controlled trials. JAMA 291: 2457 2465, 2004.
19. Freemantle N, Calvert M, Wood J, Eastaugh J, Griffin C: Composite outcomes and randomized controlled trials. JAMA 289: 2554 – 2559, 2003.
20. Montori VM, Devereaux PJ, Adhikari NKJ, Burns KEA, Eggert CH, Briel M, Lacchetti C, Leung TW, Darling E, Bryant DM, Bucher HC, Schünemann HJ, Meade MO, Cook DJ, Erwin PJ, Sood A, Sood R, Lo B, Thompson CA, Zhou Q, Mills, E, Guyatt GH: Randomized controlled trials stopped early for benefit: A systematic review. JAMA 294: 2203 – 2209, 2005.
21. Dickerson K, Drummond R: Registering clinical trials. JAMA 290: 516 – 523, 2003.
22. Wood A: Progress and Deficiencies in the Registration of Clinical Trials. NEJM 360: 826 – 830, 2009.
23. Victor M Montori VM, Jaeschke R, Schünemann HJ, Bhandari M, Brozek JL, , Devereaux PJ: Guyatt GH: Users’ guide to detecting misleading claims in clinical research reports. BMJ 329: 1093 – 1096, 2004.
24. Malay D, Yi S, Borowsky P, Downey M, Mlodzienski A: Efficacy of debridement alone versus debridement combined with topical antifungal nail lacquer for the treatment of Pedal onychomycosis: A randomized controlled trial. J Foot Ankle Surg 48 (3): 294 – 308, 2009.
25. Barnett S, Campbell R, Harvey I: The Bristol foot score developing a patient-based foot-health measure. JAPMA 95: 264 – 272, 2005.
26. Vickers A, Altman D: Analyzing controlled trials with baseline and follow up measurements. BMJ 323: 1123 – 1124, 2001.
27. Keech A, FIELD study: Effects of long-term fenofibrate therapy on cardiovascular events in 9795 people with type 2 diabetes mellitus (the FIELD study): randomised controlled trial. Lancet 366: 1849 – 1861, 2005.
28. Rajamani K, Colman P, Li LP, Best J, Voysey M, D’Emden M, Laakso M, Baker J, Keech A: Effect of fenofibrate on amputation events in people with type 2 diabetes mellitus (FIELD study): a prespecified analysis of a randomised controlled trial. Lancet 373: 1780 – 1788, 2009.
29. NNT calculator http://araw.mede.uic.edu/cgi-bin/nntcalc.pl Accessed 6/2/09.
Address correspondence to: Michael Turlik, DPM
Email: mat@evidencebasedpodiatricmedicine.com
1 Private practice, Macedonia, Ohio.
© The Foot and Ankle Online Journal, 2009
