|
|
|
|
Welcome
A WORKED CRITICAL APPRAISAL EXAMPLE.
by
Dr Nireeja Pradhan, MRCPsych
Dr Vijaya Murali, DPM
Paper
Efficacy and safety of Acamprosate in the treatment of detoxified alcohol-dependent patients.
a 90-day placebo-controlled dose finding study.
I. Pelc, P. Vernbanck, O. Le Bon, M. Gavrilovic, K Lion and P. Lehert
British Journal of Psychiatry (1997), 171, 73-77
(Please find the journal and look at the paper)
1. Why did the authors feel the need to use a newer medication like Acamprosate? What is the proposed mechanism of this drug?
Medications like Disulfiram, Bromocriptine, Buspirone and other drugs have been found to show a
small decrease in drinking , but do not reduce craving for alcohol and therefore do not significantly
reduce the risk of relapse in detoxified alcoholics. The authors in this paper have tried to prove that
Acamprosate reduce craving and increases abstinence in detoxified alcoholics.
The proposed mechanism of action of Acamprosate is as a GABA agonist and Glutamate antagonist although its exact pharmacological actions are not fully understood.
2. What is the type of study used by the authors in this paper? What are the advantages and disadvantages of this study design?
The type of study used is a Randomised, double-blind, parallel, placebo-controlled trial.
Advantages of this study design:
Randomised Controlled Trial (RCT) is the best method of investigation of new medical interventions.
a) Randomisation ensures that selection bias is kept to a minimum i.e. subjects have an equal chance of being admitted to the control or the experimental group.
b) Double blind design is one which neither the subject nor the researcher knows the research status of the subject. This therefore reduces the assesment bias.
c) The use of placebo ensures that the subjects belonging to the control group receive a placebo which is pharmacologically inactive treatment, as if it were an active and therefore this reduces the response bias also called as placebo effect. The knowledge that the person is being treated may alter the patient’s response to treatment. This is called the response bias.
d) Parallel group comparisons as used in this study ensures that each group receives a different treatment, with both groups being entered at the same time, so as to make both groups more comparable with respect to important characteristics.
e) RCT can also be used for metaanalysis.
Disadvantages of this study design
a) RCT are usually time consuming and expensive and are therefore usually done for a short period of time on a fever patients.
b) By using randomised placebo controlled trial we are withholding a potentially beneficial treatment from patients which may be unethical. There is also a question of how long the control group should be denied the experimental treatment intervention.
c) In double blind studies it is not always possible to completely eliminate information that could give a clue to the nature of treatment that is given, therefore the term "Masking" is preferred to describe the procedure when the subject and the researcher are not completely unaware of the nature of the treatment being given.
d) As RCT are commonly funded by drug companies or research organisations this may influence the research.
3. What are the limitations of data collection in this study?
a) Only patients who have undergone 14 days of inpatient detoxification have been included in the trial. No mention has been made of patients who have undergone detoxification in the community or as outpatients.
b) Subjects chosen have short drinking history of 12 months. The study has not shown the use of Acamprosate in long standing alcohol dependence especially who have had frequent relapses in the past.
c) The study excludes women not on contraception and patients with dual diagnosis and somatic disease.
d) The authors have not clarified the gender composition of the sample.
4. Were patients treated equally apart from the medical interventions?
All patients received supportive counselling and social support. No specific psychotherapeutic intervention was used. However the authors didn’t control for attendance to self-help groups which may have contributed to the patients remaining abstinent.
5. Describe what is meant by an "intention to treat analysis". What is the other alternative method of analysis?
In intention to treat analysis, data is analysed according to the way in which we intended to treat subjects and not the way they were actually treated i.e. all patients are analyses in the groups to which they were randomised
If patients drop out due to side effects, their last scores before dropping out should be carried forward and will therefore lower the average response rate, as they have not been on treatment sufficiently long to benefit. This will therefore give a more accurate view of drugs effect.
The alternative analysis is "per protocol analysis" or "on treatment analysis" which gives results only for those patients who complete treatment and therefore provides a more optimistic view of the drug’s effect.
6. Authors have used the Clinical Global Impression Scale in this study. What two important measures of a rating scale are needed to be known? What is their importance?
The two important measures that need to be known are reliability and validity. Reliability is concerned with the level of agreement between sets of observations, while validity is a term used to describe whether an instrument measures what it is supposed to measure.
A scale needs to be both reliable and valid. A reliable scale can have poor validity likewise an unreliable scale certainly is not valid, therefore both validity and reliability of the scale should be known. A scale is said to be suitable for use if reliability and validity is at least + 0.70.
7. What conclusions can you draw from Table 1 regarding the reasons for discontinuation in the study between the placebo and the treatment group?
a) Table 1 shows that more patients dropped out of the placebo group than the treatment group.
b) Severe relapse was also higher in the placebo group than in the treatment group.
c) Higher number of patients in the placebo group as compared to the treatment group were lost to follow up, which could have been due to patients relapsing, not benefiting and therefore dropping out of treatment.
8. Explain the term p"< 0.05? What alternative measures of statistical significance can be used instead of the ‘p’ value? What are the limitations of using the ‘p’ value?
The ‘p’ value is an estimate of the probability that the difference observed in the study could have occurred if the null hypothesis is true. The ‘p’ value can be thought of as the probability that the observed difference could have occurred by chance.
P<0.05 is known as 5% significant level which is an arbitrary value which states that the probability of a particular outcome occurring by chance is less than 1 in 20, therefore p<0.05 is said to be statistically significant.
An alternative measure of statistical significance is the ‘confidence interval’ (CI) which is the interval which is likely to capture the population mean with a specified level of confidence. When the CI is 95% chances are estimated to be 95 in 100 that the true mean falls within that interval.
The limitations of the ‘p’ value are that
a) It tells us nothing about the magnitude of the difference between the two treatments.
b) The ‘p’ value is less informative when in the non-significant range is unable to tell us whether there is no difference between the groups or that there were too few subjects to demonstrate such a difference.
c) The ‘p’ value is used as a single cut off point to ascertain whether treatment is effective or not. It would be more appropriate to view the results in a continuum which would be able to better assess the strength of evidence as done by using the CI.
9. Comment on the results of the study.
a) The study shows that Acamprosate is superior to the placebo for maintaining abstinence in detoxified alcoholics.
b) Acamprosate at a higher dosage had a better outcome, however at this dosages the patients had a higher incidence of side effects.
c) The time to first relapse was significantly increased in both Acamprosate groups as compared to the placebo groups.
d) Psychological dependence scores were found to be significantly lower in the Acamprosate group as compared to the placebo group.
e) No significant effects were seen on the Hamilton Scales on Anxiety and Depression between the three groups showing that Acamprosate has no effect on anxiety or depression.
f) The CGI scale which measures the patient’s overall clinical status also showed improvement in patients on Acamprosate as compared to Placebo.
10. the authors have used the Chi-square test. When can this test be used? Which other non-parametric test is mentioned in this study and what is its parametric equivalent?
The Chi-Square test is a non parametric test, which compares the value of discrete variables belonging to one sample with those of another sample.
Conditions for using this test are:
a) The population from which the samples are drawn do not have to be normally distributed.
b) It can be used with either ordinal or nominal data, unlike parametric tests where the data needs to be interval.
The other non-parametric test used is the Mann-Whitney test. The parametric equivalent of this test is the "Two sample (unpaired) t-test.
11. Were all the patients who entered the trial accounted for at its conclusion? What is meant by the term "acceptable loss"?
The total of 188 patients were included in the trial. A total of 119 patients completed the study of 90 days.69 patients dropped out of the study due to various reasons which have been documented by the authors.
An "acceptable loss" is a loss of not more than 20% of the subjects entered in the study. A loss of more than 20% seriously threatens the validity of the study.
12. What is the main implication of the study for clinical practice?
Acamprosate does reduce craving and relapse rates and can be effectively used in clinical practice. However it should not be as a sole treatment and should be combined with psychosocial treatment and adequate follow up.
13. List the main limitations of this study?
a) The sample size is relatively small. No prestudy power calculation has been mentioned.
b) The sample only represented inpatients who had undergone detoxification, therefore resulting in sample bias.
c) High rate of exclusion of potentially eligible subjects especially women, patients with somatic disease and dual diagnosis.
d) The randomisation procedure has not been explained.
e) Patients have not been followed up for an adequate period of time. 90 days is a relatively short period to judge the relapse rate in detoxified patients.
f) No confirmation of patients abstinence or compliance with treatment has been verified from informants.
g) The authors didn’t control for the attendance of patients to self help groups which may have contributed to the patients remaining abstinent.
h) The mechanism by which Acamprosate increases abstinence in detoxified patients is unknown. In this study, the authors were unable to enumerate any baseline characteristics which predicted treatment response to Acamprosate.
References
1. Critical Appraisal of epidemiological studies and clinical trials – Mark Elwood.
2. An introduction to Medical Statistics – Martin Bland
3. How to read a paper : The basics of Evidence Based Medicine – Trisha Greenhaugh
Some statistical procedures and nomenclature
By K L Suranga Wickrama
(We thank Dr M Sayeed Haque, Applied Statistician, Queen Elizabeth Psychiatric Hospital, Birmingham, for his comments)
Nominal numbers
just means names, e.g. females = 1, males = 2
Ordinal numbers
means order, e.g. IQ
Interval numbers (or measures)
same interval between scores, e.g. inches
Ratio numbers (or measures)
interval scores, but there is a true zero point
Cardinal numbers
ratio and interval scores, they can be discrete (e.g. shops) or continuous (e.g. length)
Degree of freedom
Total number of items which have to be known from any given sample, when the overall total is known, before any missing ones can be filled in (n - 1)
Mean
measures should be at least of interval status for mean to be calculated.
also called 'x bar'
simply the average, (divide the sum of scores by total number of scores)
Median
measures should be at least ordinal
simply the middle number, if number of scores are odd
or if the number of scores are even add the two middle one and divide by two
(after arranging the scores in ascending order)
e.g. 5 7 8 13 31
4 12 13 19 21 26
- 13+19/2
Mean Deviation
measures should be at least interval
subtract the mean from each individual score, and add all (ignore
the sign, [+ve or -ve] and just add all)
divide this sum from the number of scores.
Skew distributions and bimodal distribution

in +ve skew, mode<median<mean
in -ve skew, mean<median<mode
Variance
scores has to be at least interval
square of standard deviation
Standard deviation
scores have to be at least interval
subtract mean from each individual score and square each of them (get
rid of the sign)
then add all those and find the square root of that sum
divide that by the number of scores (for population),
(for the sample divide by the number of scores minus one)
Ranking
|
1 |
2 |
2 |
3 |
4 |
6 |
6 |
6 |
9 |
10 |
|
1 |
2 ˝ |
2 ˝ |
4 |
5 |
7 |
7 |
7 |
9 |
10 |
ranks in red
2 ˝ = (2+3)/2
7 = (6+7+8)/3
In Gaussean curve (bell shaped curve)
mean = median =mode
between one standard deviation (-1to+1, SD) around the mean there are 68% of scores
between two SD, there are 95.49% scores. (Gauss was a famous mathematician. The
normal distribution
is his contribution to statistics. Gaussean curve is also called the normal curve)
Variance ratio (F test)
compares spread of different distributions by looking at variances
can be regarded as a another statistical test,
as it looks at whether samples came from the same population or from different
populations
|
scale of measurement |
two independent groups |
paired data |
bivariate data |
|
qualitative (nominal) |
chi square |
Mcnemar's test |
chi square |
|
ordinal |
Mann Whitney U test |
Wilcoxon |
Spearman Rho rank correlation coefficient |
|
quantitative (ratio or ordinal) |
t test |
paired t test |
Pearson product moment correlation coefficient |
sign test can be used when the scores are paired and nominal
if the number of samples are more than two, Kruskal Wallis (non parametric),
and ANOVA (parametric) tests can be used
when there are more than two paired or when more than two measures are taken from
the same sample, Friedman (non parametric) and repeated ANOVA (parametric)
tests can be used