Comparison of data mining methodologies using Japanese spontaneous reports

Purpose Five data mining methodologies for detecting a possible signal from spontaneous reports on adverse drug reactions (ADRs) were compared. Methods The five methodologies, the Bayesian method using the Gamma Poissson Shrinker (GPS), the method employed in the UK Medicines Control Agency (MCA), the Bayesian Confidence Propagation Neural Network (BCPNN), the method using the 95% confidence interval (CI) for the reporting odds ratio (RORCI) and that using the 95% CI of the proportional reporting ratio (PRRCI) were compared using Japanese data obtained between 1998 and 2000. Results There were all in all 38 731 drug–ADR combinations. The count of drug–ADR pairs was equal to 1 or 2 for 31 230 combinations and none of them were identified as a possible signal with the MCA or BCPNN. Similarly, the GPS detected a possible signal in none of the combinations where the count was equal to 1 but in 7.5% of the combinations where the count was equal to 2. The RORCI and PRRCI detected a possible signal in more than half of the combinations where the count was equal to 1 or 2. When the pairwise agreement on whether or not a drug–ADR combination satisfied the criteria for a possible signal was assessed for the 38 731 combinations, the concordance measure kappa was greater than 0.9 between the MCA and BCPNN and between the RORCI and PRRCI. Kappa was around 0.6 between the GPS and MCA and between the GPS and BCPNN. Otherwise, kappa was smaller than 0.2. Conclusions The drug–ADR combinations detected as a possible signal vary between different methodologies. Copyright © 2004 John Wiley & Sons, Ltd.