Evaluation Proposal for Multilingual Eye
Health Public Education Videos
Program Description and Literature
Review
Background
According to CNIB statistics, (1) only one-third of working-age Canadians with vision loss are employed, (2)
seniors with vision loss are three
times more likely than those with good vision to experience clinical
depression, and (3) people with vision loss are at high risk of social isolation, reduced community participation,
falls and associated morbidity and mortality such as hip fractures and death (CNIB,
n.d.). Vision loss can lead to other major losses in life, including job, relationships, driver’s license, and independence.
Hence, preserving vision and eye health is not just important to each
individual, but to society overall.
Anecdotally, as a practicing
ophthalmologist, I have noticed many patients who speak limited English present
with end-stage blinding eye disease. I
am of Chinese ethnic background and am fluent in Cantonese and Mandarin. Through speaking to those Chinese patients
who present with end-stage eye disease, it seems that this is either because
they do not know how to navigate our healthcare system or do not know what
symptoms to look for. Sadly, some end-stage eye diseases are not
salvageable, and for others, treating advanced disease is often associated with
poor outcomes as a result of higher treatment risks and complication rates,
longer recovery time for the patient, and higher costs to our healthcare
system.
Objective evidence is also
consistent with these anecdotal findings.
Health Canada reports that language barrier is a major hindrance to healthcare
access in Canada, may provide limitation to the quality of care and patient
satisfaction, and is associated with higher costs to our healthcare system (Health
Canada, 2003).
The need
Canada
is highly multicultural. According
to the 2011 Census, nearly 6.6 million people or 20.0% of Canada's population
reported speaking a language other than English or French at home (Statistics
Canada, 2012). 9 out of 10 of these individuals reside in a
metropolitan area: Toronto, Montreal, Vancouver, Calgary, Edmonton and
Ottawa. 1.8 million of these people live
in the Greater Toronto Area (GTA) and one-third of them speak one of Cantonese,
Punjabi, Mandarin, Urdu and Tamil (Statistics
Canada, 2012).
The number of visible
minorities and those who speak limited English or French will continue to grow. The top 5 source countries of
permanent residents admitted in 2013 are: China, India, Philippines, Pakistan
and United States (Citizenship
and Immigration Canada, 2014). Statistics Canada projects by 2031, almost
half of the population over 15 years old will be foreign-born or have at least
one foreign-born parent and the number of visible minorities will likely double,
with majority of them living in one of the Canadian metropolitan area (Statistics
Canada, 2010).
Most of the
public education materials currently available in Canada are in one of our
official languages, English or French. With
this trend of increasing ethnic diversity in Canada, there will be a need to
provide education materials in multiple languages.
Economic analysis
According
to The Cost of Vision Loss in Canada report published in 2009 by the CNIB and
Canadian Ophthalmological Society,
·
The real financial cost of vision
loss in Canada is estimated to be $15.8 billion for 2007, or 1.19% of Canada’s gross
domestic product.
o
This breaks down to $500 for every
Canadian or $19,370 for every Canadian with vision loss in 2007.
o
The real financial cost consists of:
§ Indirect costs (lost productivity, care and rehabilitation, and
others) of vision loss of $7.2 billion
§ Direct costs (health-related) of $8.6 billion
·
The net cost of suffering (also
known as the burden of disease) due to vision loss, over and above financial
costs, is estimated to be a further $11.7 billion in 2007.
·
Vision loss has the highest health
care costs (direct costs) of any disease category in Canada, costing Canadians
much more than diabetes, all cancers or cardiovascular disease.
So who
pays for the costs of vision loss? The financial
costs breakdown is as follows (CNIB
& Canadian Ophthalmological Society, 2009):
·
Individuals with vision loss ($3.5
billion)
·
Family and friends ($474 million)
·
Federal government ($2.4 billion)
·
Provincial and territorial
governments ($6.3 billion)
·
Employers ($141 million)
·
Society and others ($3.0 billion)
In essence, the largest financial
costs come from taxpayers, as federal and provincial governments bear 55.3% of
the costs, and all of society bears a further 18.7%.
Based
on a rapidly aging population in Canada and the fact that the top three causes
of blindness are all age-related, in the next 25 years, the number of Canadians
with vision loss is projected to double (CNIB
& Canadian Ophthalmological Society, 2009). With increase in
immigration, any intervention will also need to be culturally sensitive and
offered in multiple languages. Hence, it
is more important than ever to place eye health as an important public health
issue with culturally-appropriate health promotion in multiple languages.
The proposed program
The
multilingual eye health public education videos (MEHPEV) project consists of
producing a series of eye health patient education videos in the top
non-official languages spoken in Canada, as well as in English and French. Videos will be hosted on the University of
Toronto Department of Ophthalmology and Vision Sciences (DOVS) YouTube channel
and will be available on the DOVS website as education tools for those who
speak limited English or French.
Awareness of the videos will be promoted through various ethnic media
(television (TV), radio, newspaper, websites), public and private immigrant
organizations, and through healthcare professionals.
Target population
The
target audience, to which the MEHPEV will be promoted, is a vulnerable
population consisting of those who speak one of the top non-official languages in
the GTA. The number of languages to be
included will depend on the funding amount raised for the program, starting
with the top spoken languages. The
MEHPEV promotion, and hence the target population, may be expanded to the rest
of Canada starting with the other metropolitan cities if the project is proven
to be successful locally and a transferability evaluation deems it to be
useful.
The
top 10 languages spoken in GTA are: Cantonese, Mandarin, Spanish, Urdu,
Punjabi, Tamil, Persian (Farsi), Russian, Italian, and Portuguese.
Video content
For
each language, the topics for the videos will start with top 3 causes of
blindness (cataracts, glaucoma, and age-related macular degeneration) and 2 other
common eye diseases (dry eyes and floaters).
Information on the basic workings of the eye, importance and frequency
of regular eye examinations will be incorporated as well as education on the various
eye health professionals and how to access eye care. If there is sufficient funding, additional
topics will be added in the future, including other eye diseases, strategies to
reduce risk of eye disease and injury, and peri-operative information and
instructions.
Format
Online videos were selected as
the media of choice for a number of reasons:
1)
Video was chosen over printed material as those with eye
diseases may not have good vision and can rely on the voice component of the
videos. This also has the added benefit
of reaching those who are illiterate as well.
2)
Online format was chosen over one time or intermittent TV or
radio broadcasts to allow for long term availability and ease of access.
3)
Online video format is more cost efficient compared to
broadcasting educational videos of several minute duration on TV or radio.
There are limitations to
online video format, as some elderly individuals may not be familiar with
internet usage, but it is hoped that some of them will have family or friends
who can assist them with access and the ethnic community organizations we are
partnering for the video dissemination will have human or technical resources
to facilitate access to the videos as well.
Hopefully these and other mitigating measures will help to allow the
online videos format to strike a balance between cost, program longevity and
ease of access.
Why is this proposed program of interest to me?
The
MEHPEV program started as a personal project.
Each time I see a patient who presents with end-stage eye disease, my
heart sinks. As many of these patients
are elderly and speak limited English, they remind me of my grandparents. I wish I could have helped them earlier,
prevented them from losing their vision or having to go through difficult
treatment course with higher risks, and I am sure their family would have
wished that for them too.
I
entered medicine not to just be a clinician.
I have always had a passion for education. I am currently actively involved in education
of medical trainees, ophthalmologists both locally and internationally, and
other healthcare professionals. I would
like to expand my education and volunteer interests to help the general
population. I think that international
outreach is highly needed, valuable and encouraged and I am involved with
international work. However, I see a
huge need locally right at home. Even
though we have one of the best healthcare systems in the world and we are so
fortunate to live in a first-world country, I see many cases of third-world
end-stage eye diseases in my practice in GTA.
Hence, I feel very strongly that we should help ourselves right at home
and started this program.
I envisioned this program to require
3 components/stages:
(1)
Obtaining funding
(2)
Video production
(3)
Video dissemination
Project goals
The goals of this project are to:
1)
Reduce the number of patients who
speak one of the major non-official languages in GTA presenting with end-stage eye
disease (secondary prevention)
2)
Promote healthy lifestyle and eye
disease risk reduction (primary prevention)
3)
Save healthcare spending costs
through prevention, and early detection and early treatment
How will this program be achieved?
•
Funding
•
Application to governments,
hospitals, and granting agencies submitted
•
Seeking individual donations ongoing
•
Seeking corporate sponsorships
ongoing
•
Dissemination
•
Media: multilingual TV, radio,
newspaper, websites
•
Toronto Public Health in current
collaboration
•
Community groups
•
Physician groups (ophthalmologists,
family physicians)
Below is a sample video series on
cataracts produced in English, Cantonese and Mandarin which was filmed
unscripted without any rehearsal. With
funding, the final videos will be more professional with standardized script,
rehearsal and computer graphics and animation of eye anatomy and surgery.
https://youtu.be/MapvrOXJ1Eg (English)
https://youtu.be/QFZJGbm-ooc (Mandarin)
https://youtu.be/gzoAG5g8C_A (Cantonese)
Similar health education programs
Health promotion is “the combination of educational and environmental
supports for actions and conditions of living conducive to the health” (Green
& Kreuter, 1991, p. 4)
and health education is “any combination
of learning experiences designed to facilitate voluntary behaviour conducive to
health” (Green
& Kreuter, 1991, p. 17). Kok, van den Borne,
and Mullen (1997) defined health promotion as a matrix of goals and means. The health promotion matrix shown in Figure
1, with the goals of health promotion across the top and the means to reach
these goals down the left hand column, provides a clear visual relationship between
health promotion, health education and prevention (Kok,
1997). The goals of health promotion are further
defined as:
(1)
Positive health – promoting healthy
lifestyle
(2)
Primary prevention – risk reduction
to prevent onset of disease
(3)
Secondary prevention – early
detection and early treatment of disease to control progression
(4)
Tertiary prevention – care of
patients with acute and chronic disease to minimize the impact of disease on
patient’s function, longevity and quality of life
Positive health
|
Primary prevention
|
Secondary prevention
|
Tertiary prevention
|
|
Education
|
X
|
X
|
X
|
X
|
Means for action
|
X
|
X
|
X
|
X
|
Pricing
|
X
|
X
|
X
|
X
|
Regulation
|
X
|
X
|
X
|
X
|
Figure 1. Matrix of health promotion, health education
and prevention
Health
education is thus the voluntary behaviour-focused component of health
promotion. Its impact is often only
successful when the other means are in place as well. In the case of the MEHPEV, means for action
exists as patients are able to access eye care by an ophthalmologist, through
referral from their family physician or optometrist. However, many patients are not aware of this
or the differences between the different healthcare professionals. Ophthalmologists are physicians and surgeons
specializing in the diagnosis and treatment of eye diseases. The training involves an undergraduate
degree, medical school (4 years), and ophthalmology residency training (5
years). As a specialist physician,
referrals are needed from a family physician or optometrist before a patient
can be seen by an ophthalmologist.
Consultations and visits with an ophthalmologist are covered by Ontario
Health Insurance Plan (OHIP).
Optometrists are not physicians and specialize in vision assessment,
prescribing and fitting glasses and contact lenses. No referral is required to see an
optometrist, and annual visits to optometrists are covered by OHIP for those 18
years old and under, 65 years old and older, and in individuals with certain
health conditions such as diabetes and high blood pressure. For others, a typical visit with an
optometrist in 2015 can range from $80 to $150.
For those with extended health plans, this cost may be partially or
fully covered. So in addition to
providing health education on eye diseases, educating patients about how (the
means of action, pricing and regulation) they can seek medical attention is
valuable as well.
“Vision
2020: The Right to Sight” is a global initiative launched by the World Health
Organization (WHO) and International Agency for the Prevention of Blindness in
1999 with the aim to eliminate avoidable blindness by the year 2020 (World
Health Organization, n.d.). The action plan for Vision 2020 has been
updated and replaced by the “WHO Global Action Plan for the Prevention of
Avoidable Blindness and Visual Impairment 2014-2019: Towards Universal Eye Health”
(World
Health Organization, 2013). Several countries, including Australia,
developed comprehensive national framework implementation plan that includes
government funding, educational programs, health programs and healthcare
professional training (Australian
Health Ministers’ Conference, 2005). However, there is
no publicly available information on the evaluation strategy utilized for this
national initiative. In fact, no
specific performance indicators or tracking mechanisms were introduced until
the 2014 implementation plan, which included information regarding funding, key
priority areas, indicators and other progress measures (Australian
Government Department of Health, 2014). Future progress
report to the Australian national framework may provide more information on the
program evaluation strategies utilized.
The evaluation strategy used to assess the public education component of
the Australian national framework may provide insights that can help with
refining the evaluation strategy used for the MEHPEV program.
To
date, no government-sponsored national eye health initiative has been started
in Canada and through extensive literature review I was unable to find program
evaluation information on similar multilingual public education program or eye
health education program.
Purpose of the Evaluation
In
general, a program is only useful if it achieves its intended goals. A program may be well-funded, have a wealth
of human resources, and even research-proven interventions, but may still be
unsuccessful in achieving its goals if it has the incorrect target population
or ecological context. In the real
world, there is limited funding and human resources, so it is even more crucial
to ensure programs are effective and efficient.
Hence, program evaluation and program development complement each other
and ideally evaluation should be incorporated from the program-planning stage.
The Australian national framework for action
to promote eye health and prevent avoidable blindness and vision loss (Australian
Health Ministers’ Conference, 2005) is an excellent example of this. The initial 2005 framework and the 2008
progress report show a highly admirable, comprehensive plan with funding and
multiple programs at local, regional and national levels (Eye
Health Working Group of the Australian Health Ministers Advisory Council, 2008). “Unfortunately, as
the national Framework did not contain specific performance indicators or
tracking mechanisms, it fell short of meeting its goals” (Vision
2020 Australia, n.d.). Evaluation strategies have been introduced in
the 2014 implementation plan and future reports will likely be able to provide
further details on evaluation type and outcomes.
The
MEHPEV project is currently in the program-planning stage. The purpose of evaluation of the MEHPEV is
two-fold:
(1)
Constructive – Given the MEHPEV
project is the first of its kind, it will be useful to know how to improve the
program to make it useful and effective as more languages and video topics are
included.
(2)
Conclusive – Since the MEHPEV
project was started by volunteers without any initial funding, knowing whether
it is useful is important to assist volunteers in deciding whether to continue
seeking funding and providing volunteer hours or to abandon the project and
devote financial and human resources to other tasks.
Chosen Evaluation Type & Outcome Evaluation Approach
The MEHPEV would benefit most from holistic
effectuality evaluation, which is a stakeholder-centered theory and
methodology designed to achieve real-world outcome evaluation (Chen,
2014, 266). In holistic effectuality evaluation, “evaluators
integrate the dynamic nature of the intervention program in a community
stakeholders’ views and practices with existing scientific methods to develop indigenous
concepts, theories, and methodologies for program evaluation” (Chen,
2014, 266). This form of hybrid evaluation allows
for initial constructive assessment
of the MEHPEV for its coherence.
Subsequently conclusive
assessment of the educational videos and its adjuvants can be performed. Throughout the evaluation process,
stakeholder involvement will be key.
Logic Model
Before
an evaluation of program is carried out, a clear description of the program is
valuable. Two tools that evaluators
often use in helping stakeholders to describe their program are: logic models
and action model/change model (Chen,
2014, p. 58). Each has its own merits and emphasis and
combined give a very comprehensive description of the program. Logic models are excellent at reducing a
complicated program to “a set of meaningful, manageable components” (Chen,
2014, p. 58). Action model/change model schema is more
complex, but “more useful when program planning or evaluation need to address
contextual factors and causal mechanisms” (Chen,
2014, p. 58).
A
logic model is a concept map that illustrates the various components of a
program, including theories and assumptions (Brown,
2012).
The basic
elements of a logic model are as follows (Brown, 2012; Chen,
2014, p. 59):
Inputs – resources (both
tangible and intangible), including contribution in kind, building space,
technology, professional expertise, course materials, and partnerships
Activities – how resources
are used with logical sequence of steps or processes
Outputs – the direct
evidence/end-products, including documents, change, job opportunities
Outcomes – elements of
success that results, including final reports, reflective writings, job
offers
The
logic model for the MEHPEV project is provided in Table 1.
Table 1. Logic Model for the Multilingual Eye Health
Patient Education Video Project
Inputs
|
Activities
|
Outputs
|
Outcomes – short-term
|
Outcomes – long-term
|
Funding
Volunteer ophthalmologists who
speak non-official languages
Video filming and edit team
YouTube/DOVS website manager
North York General Hospital eye
clinic and Kensington Eye Institute eye clinic as filming facility
|
Produce scripts for each video
topic
Translation of scripts into
multiple languages
Film videos
Edit videos
Seek media contacts for video
dissemination (TV, radio, newspaper, ethnic websites)
Seek video dissemination through
Toronto Public Health, community groups, physician groups
Seek additional funding to include
more languages and more video topics (government grant applications,
approaching individual and corporate donors)
|
Multilingual eye health patient
education videos
Video awareness promoted to the
target population of (those who speak one of the major non-official languages
in GTA)
Viewing of the online videos by
target population directly or through assistance by community newcomers
organizations
|
Increased awareness of eye disease
symptoms in the target population
Increased awareness of regular eye
examinations in the target population
|
Reduce number of patients from the
target population from presenting with end-stage eye disease
Cost savings to our healthcare tax
dollars through early prevention and early treatment
|
Action Model/Change Model
Change
models provide descriptive assumptions and deal with the causal processes that
are expected to happen to attain program goals (Chen,
2014, p. 66). Action models provide prescriptive
assumptions and deal with what actions must be taken to produce desirable
changes (Chen,
2014, p. 66)
Below
is a verbal description of the change model.
Intervention
-
Multilingual eye health patient
education videos
Determinants
-
Education of those who speak one of
the major non-official languages in GTA (the target population) about eye
disease symptoms and importance of regular eye examinations
Goals and outcomes
-
Goals:
1)
To reduce the number of patients who
speak one of the major non-official languages in GTA presenting with end-stage eye
disease (secondary prevention)
2)
To promote healthy lifestyle and eye
disease risk reduction (primary prevention)
3)
To save healthcare spending costs
through prevention, and early detection and early treatment
-
Outcomes:
·
Short-term
o
Increase awareness of eye disease
symptoms in the target population
o
Increase awareness of healthy
lifestyle and eye disease risk reduction in the target population
o
Increase awareness of importance of
regular eye examinations in the target population
·
Long-term
o
Reduce number of patients from the
target population presenting with end-stage eye disease
o
Improved eye health of the target
population through risk reduction and regular eye examinations
o
Cost savings to our healthcare tax
dollars through prevention, and early detection and early treatment
Stakeholder Involvement
A
stakeholder is a person with an interest or concern in something (“Stakeholder,”
n.d.). The MEHPEV project stakeholders include:
-
Individuals who speak limited
English or French living in GTA (the target population)
-
MEHPEV project volunteers
-
DOVS
-
Toronto Public Health
-
Public Health Ontario
-
Physician groups (ophthalmologists,
family physicians)
-
All taxpayers (through savings in
healthcare tax dollars via Ontario Ministry of Health and Long-term Care)
As
the MEHPEV is a small scale project, there are two key stakeholder groups:
target population and MEHPEV project volunteers. Representatives of each key stakeholder group
will be involved from the very beginning of program planning stage of the
project and the evaluation process. Intensive
interviews with representatives of each key stakeholder group would be carried
out initially. Questions to ask the key
stakeholder groups to facilitate in drafting the Action Model/Change Model
include:
-
What video topics are of interest?
-
How long should the videos be?
-
Would subtitles be useful?
-
How many and which languages should
the videos be produced in?
-
Given the human resources
(volunteers) and financial resources (donations), how many topics and languages
can we produce the videos in?
Involving
stakeholders in project planning stage is extremely valuable in gaining
insights regarding what the target population wants, what the volunteers are
able to produce, and what components to include in the project given the
funding and cost limitations. A
disadvantage of involving stakeholders is that sometimes it is not possible to
satisfy all the stakeholders or there may be conflicting interests between
stakeholders. For example, the target
population likely would want to have the videos produced in as many languages
as possible. However, for the MEHPEV
volunteer group, it is not possible to produce numerous video topics and in all
languages, as there is a limited pool of fund that we can work with and limited
number of volunteers. So judgement must
be exercised to balance all the wishes of the various stakeholders and still
work within the funding limits of the project.
Close
and regular communication between the evaluator and the stakeholders will occur
throughout the evaluation process, especially as the evaluation transition from
constructive to conclusive assessment.
References
Australian Government Department of
Health. (2014). Implementation plan under the National framework for action to
promote eye health and prevent avoidable blindness and vision loss. Australian
Government Department of Health. Retrieved from
http://www.health.gov.au/internet/main/publishing.nsf/Content/eyehealth-pubs-impl
Australian
Health Ministers’ Conference. (2005). National framework for action to promote
eye health and prevent avoidable blindness and vision loss. Australian
Government Department of Health and Ageing. Retrieved from
http://www.health.gov.au/internet/main/publishing.nsf/Content/eyehealth-pubs-frame
Brown, C.
A. (2012). Use of Logic Models to Plan and Assess Graduate Internship
Experiences. TechTrends, 56(6), 37–43.
http://doi.org/10.1007/s11528-012-0612-2
Chen, H.
T. (2014). Practical Program Evaluation: Theory-Driven Evaluation and the
Integrated Evaluation Perspective. SAGE Publications.
Citizenship
and Immigration Canada. (2014, October 31). 2014 Annual Report to Parliament on
Immigration. Retrieved December 10, 2015, from
http://www.cic.gc.ca/english/resources/publications/annual-report-2014/index.asp
CNIB.
(n.d.). Fast Facts about Vision Loss. Retrieved from
http://www.cnib.ca/en/about/media/vision-loss/pages/default.aspx
CNIB,
& Canadian Ophthalmological Society. (2009). The Cost of Vision Loss in
Canada: Summary Report 2009. Retrieved from
http://www.vision2020canada.ca/en/resources/Study/COVL%20Summary%20Report%20EN.PDF
Eye
Health Working Group of the Australian Health Ministers Advisory Council.
(2008). National framework for action to promote eye health and prevent
avoidable blindness and vision loss: progress report. Australian Health
Ministers’ Conference. Retrieved from
http://www.health.gov.au/internet/main/publishing.nsf/Content/eyehealth-pubs-progres1
Green, L.
W., & Kreuter, M. W. (1991). Health promotion planning: an educational
and environmental approach. Mayfield Pub. Co.
Health
Canada. (2003). Language Barriers in Access to Health Care [Health Canada,
2001] (report). Retrieved from http://www.hc-sc.gc.ca/hcs-sss/pubs/acces/2001-lang-acces/index-eng.php
Kok, G.
(1997). Effectiveness of health education and health promotion: meta-analyses
of effect studies and determinants of effectiveness. Patient Education and
Counseling, 30(1), 19–27.
Stakeholder.
(n.d.). [In Merriam-Webster’s online dictionary]. Retrieved December 21, 2015,
from http://www.merriam-webster.com/dictionary/stakeholder
Statistics
Canada. (2010, January 8). Projections of the Diversity of the Canadian
Population. Retrieved December 10, 2015, from
http://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=5126&lang=en&db=imdb&adm=8&dis=2
Statistics
Canada. (2012, October 24). Census: Language. Retrieved December 10, 2015, from
http://www5.statcan.gc.ca/olc-cel/olc.action?ObjId=98-314-X2011001&ObjType=46&lang=en&limit=0
Vision
2020 Australia. (n.d.). National Framework Implementation Plan. Retrieved
December 9, 2015, from
http://www.vision2020australia.org.au/our-work/advocacy-campaigns/national-framework-implementation-plan
World
Health Organization. (2013). Universal Eye Health: A global action plan
2014-2019. Retrieved from
http://www.who.int/blindness/AP2014_19_English.pdf?ua=1
World
Health Organization. (n.d.). What is VISION 2020? Retrieved December 9, 2015,
from http://www.who.int/blindness/partnerships/vision2020/en/
Comments
Post a Comment