Comparative trials on hybrid walking systems for people with paraplegia: An analysis of study methodology

A new orthosis (SEPRIX) which combines user friendliness with low energy cost of walking has been developed and will be subject to a clinical comparison with conventional hip-knee-ankle-foot orthoses. In designing such comparative trials it was considered it may be worthwhile to use previous clinical studies as practical examples. A literature search was conducted in order to select all comparative trials which have studied two walking systems (hip-knee-ankle-foot orthoses) for patients with a complete thoracic lesion. Study population, intervention, study design, outcome measurement and statistical analyses were examined. Statistical power was calculated where possible. Of 12 selected studies, 7 were simple A-B comparisons, 2 A-B comparisons with a replication, 2 cross-over trials and 1 non-randomised parallel group design, the last of which was considered internally invalid due to severe confounding by indication. All A-B comparisons were considered internally invalid as well, since they have not taken into account that a comparison of two orthoses requires a control for aspecific effects (like test effects) which may cause a difference. Statistical power could only be examined in 4 studies and the highest statistical power achieved in one study was 47 %. It is concluded that statistical power was too low to be able to detect differences. Even analysis through interval estimation showed that the estimation of the difference was too imprecise to be useful. Since the majority of the surveyed papers have reported small studies (of only 4-6 patients), it is assumed that lack of statistical power is a more general problem. Three possibilities are discussed in order to enhance statistical power in comparative trials, i.e. multicentre studies, statistical pooling of results and improving the efficiency of study design by means of interrupted time series designs.