Evidence Based Medicine Review
Authors:
Blume P., Walters J., Payne W., Ayala J., and Lantis J.
Publication:
Diabetes Care 31:631-636, 2008
Reviewer:
Michael Turlik, DPM
The Foot & Ankle Journal 1 (12): 5
This is an Open Access article distributed under the terms of the Creative Commons Attribution License. It permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. ©The Foot & Ankle Journal (www.faoj.org)
Methods
This was a multi-center randomized controlled efficacy trial of 342 diabetic subjects who were followed for a minimum of 112 days. The study was performed across 37 diabetic foot clinics and hospitals principally in the United States. The primary outcome was complete ulcer closure. There was a thorough description of the method of randomization as well as, concealment allocation. Subjects and investigators were not blinded and it was unclear if data collectors and analysts were blinded. Safety and effectiveness analysis was conducted by the company sponsoring the study.
Results
A sample size calculation was carried out with the expectation of a 20% difference between groups (Absolute Risk Reduction). Enough subjects were enrolled in the study to satisfy the sample size calculation. Baseline data examination reveals no difference between groups. Data from the primary outcome were analyzed as intention to treat as well as, per protocol. Efficacy of the intervention was reported as statistically significant both for intention to treat and per protocol analysis favoring the vacuum assisted closure method. Point estimates were reported for the primary outcome but 95% confidence intervals were not reported.
Funding
The manufacturer of the wound closure vacuum assisted device (KCI) supported the study. It was not indicated if the study was registered. The primary author has received payments from the manufacturer for speaking engagements. No other disclosures were noted for other authors or investigators. The article has been marked as an advertisement by the publisher.
Comment
The study contains several well described methodological techniques to limit bias, randomization, concealment allocation, and intention to treat analysis. However, due to the nature of the study investigators and subjects were unable to be blinded.
Although unblinded studies are associated with an increased treatment effect this is less likely when the primary outcome is objective such as resolution of an ulcer as opposed to soft measurements such as patient reported outcomes. [1] However, no mention was made regarding blinding of data collectors and analyzers.
The data from the study was analyzed by the company funding the study. This may be perceived as a potential source of bias, it is more reassuring to the reader when the data is analyzed by a neutral third party. The results of the primary outcome were presented as intention to treat and per protocol. It appears the author chose to assign the worst case scenario for the data lost to follow-up for the ITT analysis. Both methods were statistically significant however differed in their point estimate. Was this study clinically significant? The authors expected a 20% difference (ARR) between groups when they calculated their sample size. If the 20% difference is to be accepted as a clinically significant result then the result of the primary outcome using the intention to treat analysis was not clinically significant but only statistically significant. The per protocol analysis was both clinically and statistically significant. Furthermore, it is difficult to analyze the results with only point estimates and not 95% confidence intervals (CI). Why 95% CI were reported for secondary measures and not the primary outcome was unclear.
There appears to be a fairly high loss to follow-up in both arms of the study (approximately 30% per treatment arm). The prognosis for subjects lost to follow-up is thought to be different than the patients who remain in the study. [2] This loss of data may compromise the randomization sequence. Did the loss of follow-up effect the results of the study? The strength of the inference drawn from the study is modified by the magnitude of the difference between the intention to treat and per protocol analysis. It would have been instructive for the reader if the authors addressed this point during their discussion of the results.
Interpretation the study’s results would be better understood with a clear clinically important difference stated by the authors and with 95% CI reported about the point estimate of the primary outcome.
Using the data from this study 95% CI can be calculated for the Absolute Risk Reduction (ARR) and Number Needed to Treat (NNT) for both the intention to treat and per protocol analysis. [3] (table 1)
Table 1
The ARR only exceeds 20% during the per protocol analysis. The lower end of the 95% CI for both ITT and PP is greater than 0 which is consistent with a statistically significant result. Although the point estimate (ARR) for the intention to treat analysis is less than 20% , a risk reduction of more than 20% cannot be ruled out by evaluating the upper end of the 95% CI and would suggest a larger study is necessary or less loss of follow-up.
The NNT is a more clinician friendly metric to access efficacy in studies with dichotomous outcomes. The NNT for both are similar 5 (PP) and 8 (ITT) however, the upper limit of the 95% CI or worse case scenario is 12 (PP) and 24 (ITT). This appears to be a large difference.
Although the use of the vacuum assisted closure appears to be more efficacious the magnitude of the effect is unclear and the inference reduced. It is up to the reader to determine if the loss to follow-up, lack of blinding and lack of clinical significance reduces the inference of the results of this study.
In addition, since the study was designed as an efficacy rather than an effectiveness study, generalizing the results to clinical practice should be undertaking with caution.
The safety data were presented as treatment related rates at six months. However, the trial evaluated treatment until day 112 or ulcer closure by any means. It would be informative to the reader to review the data on safety prior and post intervention termination. There have been two meta-analysis published this year for vacuum assisted closure and diabetic foot ulcers this year. [4,5]
References
1. Woods L, Egger M, Lotte Gluud L, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336 (March): 601 – 605, 2008.
2. Montori VM, Guyatt GH. Intention-to-treat principle. Can Med Assoc J 165 (10): 1339 – 1341, 2001.
3. Graphpad. http://www.graphpad.com/quickcalcs/NNT1.cfm Accessed 11/3/2008.
4. Gregor S, Maegle M, Sauerland S, Krahn JF, Peinemann F, Lange S. Negative pressure wound therapy a vacuum of evidence? Arch Surg 143 (2): 189 – 196, 2008.
5. Bell G, Forbes A. A systematic review of the effectiveness of negative pressure wound therapy in the management of diabetes foot ulcers. Int Wound J 5 (2): 233 – 242, 2008.
© The Foot & Ankle Journal, 2008
