What is research?
To most people, the idea of research is 'intimidating'--it conjures up a complex and secret activity done on a grand scale in large institutions. In fact, it can be very simple and may involve nothing more than a therapist asking questions of his/her patients. Moreover, the question asked can be very down-to earth, such as whether cot deaths are reduced when babies are laid to sleep on their sides rather than on their backs.

Formally, research is the activity of discovering things, supplying new information and increasing the understanding of the world. In medical or health services research, this includes a vast range of activities, from discovering ways to improve medical care and establishing new treatments, to monitoring the effect of changes to lifestyle and uncovering adverse effects of existing treatments. In essence the research method represents a systematic way of gathering information and reaching an answer based on that information. It is worth remembering that research is only worthwhile if it gives useful information-positive or negative- which answers the original question to the satisfaction of researchers at large. For this to happen, the question must have been sensible and capable of being answered, the study must have been appropriate and powerful enough to answer the question.

What do researchers do?
In essence, the researcher brings together, analyses and interprets information (data) in order to answer a research question. Thus, before starting an investigation a researcher has to be clear about the purpose of undertaking it. Table 1 shows the four basic areas of activity to which research is central: description, explanation, evidence-gathering and generalisation. It is worth noting that any one of these activities is not 'better' than the others: the one chosen depends on the question being asked. It is also worth saying what research cannot do: it cannot look for proof since it merely collects evidence for or against a hypothesis; and it cannot give an indubitable answer, merely the best evidence available at the time- future research will overturn the findings. Though the research method can be a superb way of moving medical knowledge forward, it is important to recognise that its just a method of investigation: like any other method it can be fraught with problems.

Table 2 shows the types of research study which can be carried out, and gives a brief indication of why each type is used.  

The differences between what each of these studies can do is perhaps best illustrated by an example. In the 18th century, a person with intermittent headache might take the ground up bark of a tree whenever a headache arrived. If the treatment seems to relieve headaches, she might recommend it to a friend: this story is an ANECDOTE. Before the 20th century almost all of medicine took the form of anecdotes, some of which had a basis in fact but most of which did not. The central problem is that, in a complex world, one report does not prove much.

Later, when the value of recording begins to be instituted, an interested observer such as a local doctor might begin ground up bark to a variety of patients with headache. The result will be a number of CASE REPORTS documented by an independent observer, a CASE SERIES, which is more convincing than an anecdote on a scale of evidence. When it is later realised that the case series should be more rigidly defined, a COHORT STUDY might be instituted in which a group of very similar patients are given a properly specified dose of the 'active' substance which the bark contains. In this way, a group of patients with closely similar symptoms could be followed over time after a predefined course of treatment.

Though conclusions can now be more precisely drawn, because improvements might be due to other factors (e. g., headaches might go away of their own accord) a CLINICAL TRIAL, with at least a control and a treatment group, are necessary to discover the specific effect of the substance on headaches over and above these other factors. The most refined form of this is the RANDOMISED CONTROLLED TRIAL in which patients are randomly allocated to groups and the experimenters are blinded. If the true effect of the substance is relatively small, the results of a large number of RCTs will need to be pooled in a SYSTEMATIC REVIEW before this fact becomes convincingly apparent.

Details of types of studies

• Case reports
  A case report is a detailed record of the symptoms of a disease or its response to a treatment in one patient (or several if it is a case series). In the past this type of relatively simple investigation was the major form of evidence but it has been superceded in the 20th century by more sophisticated study types. However, it still has a use as evidence of effectiveness when the illness is very rare, or to look into adverse effects of treatment. If a disease is new, a cluster of several such case reports can be very valuable, for example, case series were useful for documenting the early reports of AIDS in the USA. In complementary medicine there is no shortage of this kind of 'evidence': complementary therapy journals have many such reports, though often they are short on detail.
• Cohort studies
  This is really a more extensive, and more properly defined, series of cases in which a group of patients (a 'cohort') is followed over time. It can be retrospective i.e. reviewing case records, or better, prospective. It is best in such studies that data is collected in a standardised manner, specifically for the purposes of the study. An example of such a study would be an examination of the effects of daily garlic on levels of helicobacter pylori in the stomach: by giving volunteers garlic for 8 weeks and testing for helicobacter at start and end, it could be determined whether this therapy might be useful. In this type of study there is no way of knowing whether any changes seen in the condition is due to factors other than the effect of the treatment. These factors include expectation of effect, the natural change in the course of the disease, the effect of the interaction of the therapist and the patient and a host of other factors which collectively go under the heading of the 'placebo effect'. However, such studies are nevertheless useful when clinical trials cannot be done, or when the research question is different, for example, what are the effects of organisational changes within the NHS.
• Surveys
  A survey is a systematic method of gathering data on what is happening. It gives a snapshot of current views or activities and is commonly used to assess , for example, the amount of disease in the community, patients use of therapies or their opinions of their treatment. Ideally, a survey to give maximum information will be targeted at and designed for a particular group of people. If it is piloted first, if the sampling method is appropriate and if the response rate is high enough it can answer a descriptive question.
• Clinical trials
  After 1940, it began to be realised that the results of both anecdotes and case series were suspect as regards giving a valid indication of the effectiveness of a treatment. Today controlled trials, especially those that are randomised (RCTs), are the best known ways of determining whether or not a treatment is efficacious for a particular condition. Such trials are most often described in terms of drug evaluation but in fact they can be use for almost any therapy, and even to assess a health care service or intervention. Controlled trials answer the question, 'Does it work?', by comparing--under as similar conditions as possible-- a group of patients given a new treatment with one or more 'control' groups. The controls groups can consist similar patients who get no treatment ('waiting list' controls), who receive a current or 'standard' treatment (standard treatment controls) or who receive an inactive treatment indistinguishable from the treatment (placebo controls). The most 'perfect' trials are generally considered to be those which can most clearly minimise the risk of 'bias' which can cause differences between groups to appear. Such trials are 'randomised' i.e. patients are first enroled into the study and then randomly allocated to receive treatment or not; generally placebo-controlled so that treatment is administered to both groups in exactly the same way, and double 'blinded' so that neither the patients nor the trial evaluators know treatments they are receiving.
• Systematic reviews
  It is very often the case that one clinical trial by itself has limited information. Systematic reviews attempt to bring together all reports of controlled trials using a particular therapy or treatment, in order to evaluate the therapy over a range of settings and patients groups. Thus, a good systematic review based on good trials provides more convincing evidence than does one clinical trial alone. Sometimes, if different RCTs have very similar patient groups and common 'outcome' measures (such as death rates within a particular time), the data from individual studies can be pooled and re-analysed to give one 'global' outcome. This is a very special case and is called meta-analysis. The Cochrane Collaboration was established in 1992 to collect, review and then publish systematic reviews of studies already in print.

Factors involved in the success or failure of research studies
Why research is so often 'unsuccessful' or 'inconclusive' is most often because some fundamental rules have not been followed. Several of these are given below.

• The hypothesis must be valid
  There must be a reason for beginning a research study, however basic. It is sometimes the case that researchers either have no reason--they just gather data in the hope that an interesting finding will turn up--or their hypothesis is flawed. For example, because asprin is a specific therapy for acute headache we cannot hypothesize that it might also be useful to counteract baldness.
• Outcome measures must be valid and sensitive.
  A study following a group of patients has to measure a sign or symptom in each to tell whether there has been any change in the illness.Such outcomes have to be measurable and must be expected to change if the treatment effect is to be demonstrated.
• The study type chosen must be able to give an answer
  It is sometimes the case that the type of study chosen will not give the answer required. A cohort study, if positive, shows only that a group of patients when given a treatment and enroled in a study will have symptom relief. A further controlled study is required to determine the cause of the symptom relief.
• The number of patients in the study must be large enough
  Because of the background variability in patient symptoms too small groups run the risk of a 'genuine' effect of treatment being lost in background noise. There are formal statistical methods for calculating the optimum sample size for a patients population and these must be followed. In general the less effective the treatment the larger the sample size will need to be to detect a difference between treatment and control groups.
• Flaws within controlled trials
  Because controlled trials are intricate to perform, there can be flaws which limit their usefulness. Randomisation and blinding can fail for a variety of reasons (usually lack of care on the part of the trial investigators). Again patients may be lost to follow-up. Such flaws often limit the conclusions that can be drawn from a particular study. Since no study is perfect, one trial alone is rarely conclusive.

Types of research study
When people say, 'research has shown...' they sometimes assume that the question--whatever it was--is settled. In fact 'research' is a broad term covering a variety of types of study each with a different design. These have not been developed specifically to baffle the layman: they have evolved over the years as our understanding of the difficulties inherent in scientific research has grown. What we are really faced with is different kinds of evidence, ranging from slim anecdote which can hide all kinds of false views to the kind of 'hard' evidence obtained from controlled clinical trials which are more likely to satisfy the scientist. A list of these are shown in the Table.

Issues which complicate research

• Spontaneous improvement in disease
  The signs and symptoms of many diseases come and go, and vary in intensity from day to day. Also some conditions are self-limiting, that is they get better by themselves without treatment. Studies, especially those seeking to assess the effectiveness of a treatment, have to take these changes into account. This is one reason why clinical trials have a group of untreated controls against which patients in the treatment group can be compared. For example, without untreated controls, a study of a new treatment for hay fever could show a positive result, unrelated to treatment, simply because pollen counts fell during the investigation.
• Placebo effect
  It has long been recognised that factors other than the specific effect of the therapy can help patients to improve. The improvement seen is also called the placebo response, and factors influencing it are also known to include hope, expectation, trust in the therapist though there may be a plethora of presently unknown contributory factors. Some early therapies which we now know to be useless, such as bloodletting for a variety of conditions, where considered efficacious at the time, due to placebo effects of the apparent treatment. Investigators take this effect into account by including a 'placebo control' group as part of their trials. Only when a treatment shows an effect on symptoms over and above the placebo effect is the treatment considered to be efficacious for the illness.
• Effectiveness/efficacy difference
  Scientific research, like other specialist from of human activity, has its own jargon: terms such as 'treatment', 'control', 'subject' have specific meanings though most can be surmised by the general reader. One piece of jargon that has great capacity to confuse, however, is the difference between effectiveness and efficacy. Effectiveness relates to whether a 'treatment package' works in practice. For example, a study in which patients were given a herbal extract every week for 8 weeks, and which found that quality of life significantly improved would be justified in concluding that, at least on the basis of this study, 'the action of giving patients with a certain condition a herbal extract as part of a trial while all the hope, expectation and excitement involved' is effective in improving quality of life. What cannot be concluded is that the improvements in quality of life were caused specifically by the herbal extract i.e. that the specific efficacy of the extract had been shown. To show that, a randomised controlled trial, containing a control group taking a placebo extract, is required.

© RCCM June 1999