Trying to demonstrate clinical effectiveness of your product can be difficult. A clinical study can be expensive, and so, extensive planning needs to be done prior the study to make sure that you accomplish what you set out to evaluate during the study. This post outlines six things to consider when designing your clinical study.
1. Set Goals
There are several possible outcomes that can be evaluated, such as patient satisfaction, quality of life, patient activation, or disease specific outcomes. It is important to know the outcomes you are trying to measure, as this may influence the patient population you select.
2. Select A Study Design
An effective study demonstrates causality that your product has improved the patients’ clinical outcomes or quality of life.
Randomized Control Trial (RCT)
The gold standard in clinical trials is the randomized control trial (RCT). In the RCT, the users are randomly allocated to the treatment and control group (i.e. use of your product versus continued regular care).
After patients have been recruited in the study, these patients are randomly assigned to receive treatment. Other than the use of your product, all other care received should be the same to avoid any bias in outcomes. The RCT is often used, as it minimizes selection bias and to account for any observed or unobserved differences between the patients using your product and the control group.
For instance, if you only recruit IT savvy patients in your study, the engagement rate with your product measured at the end of the study may be exceptionally high and not reflective of what you’d expect from the general population.
Conversely, with a RCT, you would have a control group and treatment group, both of which appear similar in terms of the observed variables (e.g. age, sex, socioeconomic status, disease state, desire to use IT etc). As such, you can say with greater confidence that any difference in outcomes measured between the treatment and control group can be attributed to the use of your product.
The randomization process is a lot more complicated than a coin toss to determine who gets the treatment (product use) versus control (regular care), as you want to ensure that any potential confounders that could lead to differences in outcomes are equally spread between the two groups. However, RCTs are the most expensive, as it requires more users (1 set of users for control, and the same number for the treatment group).
One alternative is a pre/post study in which baseline measures are first collected from your patient population. Thereafter, the treatment (i.e. product use) is administered to all study participants. Outcomes are measured again during and at the termination of the study.
Essentially, in this study design, the users act as their own control during the study. However, this test method is inappropriate for contexts in which the users would see improvements in health outcomes over time anyway, regardless of the application use, as it will be difficult to attribute improvements in outcomes solely to your product use.
The pre/post study design is however, an acceptable method for a pilot study, or feasibility test. The outcomes measured should preferably be objective measures (e.g. weight or blood sugar level) instead of something self reported, as improvements in any self reported outcome could be a result of increased attention as a result of the treatment, as opposed to real changes in outcomes. It should also not be used for cases in which symptoms and conditions fluctuate over time.
3. Picking Participants
Running the study is an expensive event – it is critical to identify the confounders in your study population before running the study and find a way to account for these. For instance, you could choose to exclude patients with these confounders in your study by including only a certain subset of the population.
However, in doing so, you’ll have to acknowledge the limits to generalizability in your study, e.g. we only demonstrated that this worked for our sample of heart disease patients over the age of 65.
4. The Study Site
A clinical study site not only allows you to have access to their patients and resources to run this study but also needs to provide a sufficient sample size for your study. Patients may choose not to participate in the study for whatever reasons or may drop out of the study mid way. Additionally, you need a sample size that will be able to demonstrate statistical significance i.e. any difference in the outcome with product use is not a function of chance but a real change.
5. Patient Permission
Be sure to get a signed informed consent from the patients and providers – here’s some of the things you should cover in your informed consent documentation:
- The study involves research and participation is voluntary. Refusal to participate or deciding to discontinue will not penalize the participant in any way.
- An explanation of the purposes of the research.
- An explanation of the expected duration of the participant’s participation and the approximate number of participants expected to be involved in the study.
- A description of the procedures to be followed.
- Identification of any procedures which are experimental.
- A description of any reasonably foreseeable risks or discomforts or benefits to the participant.
- A statement describing the extent, if any, to which confidentiality of records identifying the participant will be maintained.
- An explanation of whom to contact for answers to pertinent questions about the research and research subjects’
It is also important to make sure that your data collection and storage methods are compliant with Health Insurance Portability and Accountability Act (HIPAA).
6. Collecting Data
Remember to collect baseline measures and measures during and at the end of the study. The following are some of the survey instruments you can use for measuring the outcomes for your study:
- SF-36: http://tinyurl.com/kry38g3
- Patient Activation Measure: http://tinyurl.com/kn5za2w
- Hospital Consumer Assessment of Healthcare Providers and Systems (HCAPHS): http://tinyurl.com/lq8wtga
Another tip is to look up clinical studies in the disease population studied on http://scholar.google.com to determine the validated surveys often used to measure the outcomes for these patient populations. Other possible data you could collect include interviews or focus groups with staff and patients (seek permission first!).