Program Evaluation is important to ensure the quality and
effectiveness of a program. I chose to
look at the evaluation that was done on “Big Brothers and Big Sisters”. http://www.issuelab.org/fetch/publicprivate_ventures_104.pdf
Evaluation
Model
This program was evaluated using the Provus– Discrepancy evaluation
model.
The Provus method identifies four specific stages of all programs:
Design Content, Installation, Process, and Product. Within each of these stages there are
important questions to consider when evaluating a program.
Design Content – “Is the program adequately defined?”
Installation – “Is the program installed as described in
Design Content?”
Process – “Are the resources and techniques being used
congruent with the goals of the program?”
Product – “Are the program objectives achieved in the
implementation?”
“Making a
Difference – An Impact Study of Big Brothers and Big Sisters”
“Making
a Difference – An Impact Study of Big Brothers and Big Sisters” was written by Joseph
P. Tierney, Jean Baldwin Grossman, with Nancy L. Resch. This evaluation was completed to answer some
questions about the mentoring process.
The major purpose for this report was to produce evidence that mentoring
programs are meaningful and worthwhile.
It also attempts to explain what makes a mentoring program successful or
unsuccessful. The focus of this report
is on the “Big Brothers and Big Sisters“(BBBS) program. This organization was chosen because it has
the most success. This evaluation
completes an impact study of eight BBBS sites and analyzes the data to prove
that it is a worthwhile organization and in which areas they are effective and
why.
Pros
Design
Content – “Is the program adequately defined?”
The program and the purpose are clearly defined and include a
substantial amount of background information.
It discusses the criteria for participants and explains how the focus
group will be determined.
Installation
– “Is the program installed as described in Design Content?”
The program is laid out exactly how it is stated in the Design
Content area. They explain the intention
and what is going to happen and they follow through.
Process –
“Are the resources and techniques being used congruent with the goals of the
program?”
There is clear cut data showing that the effect of having a
mentorship program is important and it makes a difference in behaviour, academics,
and drugs and alcohol.
Product –
“Are the program objectives achieved in the implementation?”
The product is achieved.
They wanted data to support the importance of mentorship programs and
they were diligent and organized with the handling of information and were
successful.
Areas for Future Research allows the reader to know exactly
where this evaluation could grow and what questions were left after the impact
study.
Cons
There was only one study used for this evaluation. Also this evaluation was done 20 years ago
and it would be beneficial to re-evaluate and complete another impact study.
The study left many questions unanswered that could use an
additional impact study.
Some of the reports that were created to substantiate evidence
are difficult to understand and make sense of quickly. To understand them completely you would have
to go fairly in-depth into the impact study.
Overall
I think they did a really good job of evaluating and creating
an impact study. They were also very
organized with the information that they gathered. The table of contents made it easy to find
what I was looking for and where to find it.
Hi Janine.
ReplyDeleteAs a former Little Brother I liked your choice of program. I agree that it is dated but it is a good example of a well-planned and executed evaluation. The methodology and data analysis were strong. Based on the people involved and the supporters there was ample financial resources available for the PE. One thing that I felt they missed were concrete recommendations. I would like to see them use better language to make their message stronger in regards to supporting the program. It also had a bit of a summative feel to it. I like the way you applied Provus to the piece, breaking down each aspect and connecting it to the report.
Good work!
Jay