In this course, you’ll learn the foundational economic theories behind health care innovation and how to optimize your own health care practice or organization. Designed to help you gain a practical understanding of the theoretical frameworks of behavioral economics and operations management in the health care setting, this course will help you apply these frameworks to assess health care practices and apply innovation while managing risk. You’ll also explore the best practices for evaluating one’s innovative practices, using real-life examples of success to see the concepts in action. By the end of this course, you’ll have honed your skills in optimizing health care operations, and be able to develop the right set of evaluations and questions to achieve best innovative practices within your organization.
From the lesson
Module 3
In this module, you’ll examine the practice of evaluation and how it is applied to health policy and programs. You’ll gain a better understanding of the need for evaluations in an ever-changing health care environment, and the importance of control groups to combat selection bias that may skew the findings of an evaluation. You’ll explore different methods to conducting an evaluation, the types of questions an evaluation aims to answer, and the difference between effectiveness and efficacy. By the end of this module, you’ll understand the theoretical framework behind an evaluation and be able to employ an evaluation to better analyze the effectiveness of your health care organization.
Andrew M. Heller Professor at the Wharton School, Senior Fellow Leonard Davis Institute for Health Economics Co-Director, Mack Institute of Innovation Management The Wharton School
Amol S. Navathe, MD, PhD
Assistant Professor of Medical Ethics and Health Policy Department of Medical Ethics and Health Policy
David A. Asch, MD, MBA
Professor of Medicine and Professor of Medical Ethics and Health Policy Department of Medicine
Roy Rosin, MBA
Chief Innovation Officer Penn Medicine
Kevin Volpp, MD, PhD
Professor of Medicine, Division of Health Policy / Professor of Health Care Management Perelman School of Medicine / The Wharton School
What is evaluation? Two concepts,
correlation and causality, and an example,
called the important, RAND Health Insurance Experiment.
We'll also cover quantitative versus qualitative evaluation.
So what is evaluation?
Program or policy evaluation is the application of
systematic methods to address questions about policies or programs and their results.
It includes long term monitoring of a program or policy as it evolves,
as well as one shot studies of a single program impact at a given moment.
The approaches used are based on
social science research methodologies and professional standards.
In particular, the field provides processes and tools to obtain valid,
reliable, and credible data to address
a variety of questions about performance of programs.
The whole idea is to provide scientifically measured,
rigorous evidence that informs evidence based policy making or program development.
The central challenge in evaluation however,
is to identify the causal relationship
between the programmer policy and the outcomes of interest.
So, causality is the key.
But what is causality?
Have you heard of the phrase,
"correlation is not causation"?
Let's define correlation first.
Correlation is a relation existing between variables which
occur together in a way not expected on the basis of chance alone.
For example, the obvious correlation
between the month of January and snow in the Northeastern United States.
Causation is a demonstration that one variable directly influences another variable.
When a variable does have an effect on the other variable,
then you can say that you have identified causation.
So, causation is a subset of correlation.
Meaning, you must have correlation to have causation.
But correlation is not equivalent to causation.
Conversely, if you have causation,
you must have correlation.
So, let's give some examples to bring this to life.
Let's start with correlation.
Did you know there is a correlation between the number of
Nobel prizes won by a country and per capita chocolate consumption?
By the way, Switzerland is number one.
Doubtful that the chocolate is causing the Swiss to win all those Nobel Prizes.
Or the correlation between children eating breakfast and getting better grades.
Surprisingly, there is a lot of research that's been done on this.
It may be causal for hungry kids in famine because they can't concentrate.
But it hasn't held writ large in the United States.
A third example, a community's crime rate and its number of police.
Doubtful that the crime rate itself is causing the number of police to go up.
On the other hand, what are the examples of causation?
Smoking and lung cancer.
This one is well established.
Statins use and lowering heart attacks in patients with high cardiovascular risk.
Again, one with a lot of medical evidence.
But what about in health policy?
Let's spend some time going through a canonical example of
a health policy experiment that you should remember.
The RAND Health Insurance Experiment was
a large scale randomized experiment conducted between 1971 and 1982.
For the study, RAND recruited 2,750 families accounting for almost 8,000 individuals.
These individuals were randomized to insurance plans with different designs.
Basically each plan made the individual pay a different share of the costs.
So what did they find? Participants who paid for a share of their health care use
fewer health services than the comparison group given free care.
Cost sharing reduced the use of
both highly effective and less effective services and roughly equal proportions.
Importantly, cost sharing did not
significantly affect the quality of care received by participants.
Wow. Seems like people might actually be okay at figuring out what healthcare is worth.
Well, while cost sharing in general had no adverse effects on participant health,
there were some exceptions.
Free care led to improvements in blood pressure,
dental health and vision.
These improvements were concentrated among the sickest and the poorest patients.
So, then again, maybe cost does matter.
The reason I share this with you is twofold.
First, it illustrates the causal effect of
cost sharing on a number of outcomes like health and use of services,
both high value and low value.
Second, because it is an important health policy finding, that you should know,
it has been the basis for
most health insurance design nationwide for the past 30 plus years.
The main reason for this is that evidence is considered very sound and robust.
That's because randomized trials are the strongest way to build evidence of causation.
But even they have limitations,
more on those tradeoffs later.
Another important dimension of evaluation to highlight is that
not all evaluations are about hard quantitative outcomes like cost or income.
Much of our insight comes from the qualitative.
Qualitative data our key complement to
quantitative impact evaluations because they can
provide added perspectives on a program's performance.
Evaluations that integrate qualitative and quantitative analyses
are characterized as using mixed methods.
So, what are examples of qualitative approaches?
They include surveys, focus groups, interviews, case studies.
Importantly, qualitative evaluations can also be designed to address causality.
Today, we covered what is evaluation,
the concepts of correlation and causality,
examples including the ever important RAND Health Insurance Experiment
Christian TerwieschAndrew M. Heller Professor at the Wharton School, Senior Fellow Leonard Davis Institute for Health Economics Co-Director, Mack Institute of Innovation Management
Amol S. Navathe, MD, PhDAssistant Professor of Medical Ethics and Health Policy
David A. Asch, MD, MBAProfessor of Medicine and Professor of Medical Ethics and Health Policy
Roy Rosin, MBAChief Innovation Officer
Kevin Volpp, MD, PhDProfessor of Medicine, Division of Health Policy / Professor of Health Care Management
