Understanding study design in comparing drug studies

For a variety of reasons, pharmaceutical manufacturers often do not conduct head-to-head studies of their drugs versus the competition. On the other hand, head-to-head studies are the ultimate choice for people who have to make comparative assessments of drugs.

In the absence of head-to-head studies, decision makers generally have to make do by trying to compare competing therapies by looking at studies that have the most similarities (patient population, disease characteristics, treatment plan, etc). An important consideration in comparing studies is study design, and in particular, the methods used for analysis of the primary end points.

A common method for analysis is the last observation carried forward (LOCF) method. For subjects who for whatever reason drop out of a clinical study, the last-measured value for a variable such as response to treatment is carried forward and assumed to be valid for the last scheduled measurement in the study.

An alternative is the non-responder imputation (NRI) method. In this method, which is used for dichotomous (“yes or no”) or categorical variables, if a subject drops out of a study, that subject is assumed to be a non-responder, regardless of whether or not the subject was responding to treatment at the time of dropout.

Why is the method of analysis important? It can significantly affect the apparent result of a study. In the LOCF method, for example, the last observation is often the best observed for subjects who drop out because they cannot tolerate the therapy. This biases the result in favour of the active treatment.

In contrast, NRI is a conservative method that avoids this bias. NRI may, in fact, underestimate the efficacy of a therapy, for example by categorising as a non-responder a subject who may have dropped out simply because he moved or couldn’t make scheduled study visits. Because NRI is conservative, it is gaining respect as an analytical method. In fact, according to Okamoto et al*, the US Food and Drug Administration sees LOCF as no longer acceptable for some analyses, favouring conservative methods of imputing missing values.

NRI is, in fact, one of several methods of imputation, including worst observation carried forward (WOCF), group mean imputation (GMI), placebo mean imputation (PMI), and imputation based on the reason for discontinuation (IDUR). Furthermore, conservative methods and LOCF can be used appropriately in the same study. (NRI is not appropriate for continuous variables.)

The point for those who evaluate drugs and therapies is this: when comparing studies, know which analytical methods are being used, and understand how the choice of analytical method can affect how study results look.

*Okamoto A, Wang J, Mohanty S. Rescue behavior and imputation strategies in analgesic studies (PowerPoint presentation). Accessed 01 October 2007.