This is an issue about small data, but it has become critical over time. It's not that statisticians are doing a bad job when assessing the efficiency of a new drug, it's not that the data is bad.
What is bad is the process used to recruit test patients in clinical trials. It has become so bad that lot's of people are willing to lie about their condition in order to be accepted (and paid) in these clinical trials.
As a result, many drugs with great potential were erroneously turned down (by the FDA who supervises these clinical trials) because of flaws in clinical trials. It also indirectly increases the costs of the drugs that are eventually accepted, and these drugs (that are accepted) might be so based on wrong results due to testing the wrong people.
For more on this subject, read "Studying Drugs in All the Wrong People" published in the August edition of Scientific American.
How should this issue be addressed? Do you think the situation is this bad?
Vincent, I work pharma. with all due respect. Get your facts correct and correct, your TYPOs
"efficiency of a new drug" its
1. Efficacy and Safety (please note your typo)
2. FDA who supervises these clinical trials - NO, the -Sponsor - (a pharmaceutical company) is the Sponsor. FDA REviews the sponsor work
3. "drugs (that are accepted) might be so based on wrong results due to testing the wrong people"
The root of the problem, I suspect, is that there are really two different kinds of trials with two different purposes. Mechanism trials seek to isolate one physiologic or psychological aspect of an intervention from any others (especially from the placebo effect). These trials require a very homogeneous subject population. They are interested in the drug-disease interaction, so they carefully control everything else. Effectiveness trials, on the other hand, seek to understand how the drug works in the population of interest - the drug-patient interaction. To do this, you need a variety of subjects that form a representative sample of the intended population. Effectiveness trials tend to require more subjects than mechanism trials because they deal with greater variability in the types of patients.
Many drug trials are designed, at least in part, to confirm the mechanism of action of the intervention. They tell you about safety and efficacy in the narrowly-defined homogeneous subpopulation that were enrolled, but they do not generalize to the broader range of patients who might be candidates for therapy. Effectiveness trials, on the other hand, tell you if the intervention works, but have difficulty explaining why. Although we need both types, my opinion is that we have too many mechanistic studies and not enough effectiveness trials.
I think that the problem may be tied to the use of monetary incentives. The incentives themselves are the target for the participants. Change the incentive to a non-monetary compensation and people will be less likely to try to spoof the system. You will have a drop in total participants, but you may find that the ones who do volunteer are more genuine. Perhaps offering discount vouchers for medications given at completion of the study would be a good incentive.