Evaluation Plan

Click here for Evaluation Plan

Summary of Assignment 5

       This survey's intent was to find out if the Literacy Place guided reading materials were effective and worth the expense in Springside school.
       Overall my survey was easy to understand.  There were a few changes that I made based on the results of the quiz.  I added in one question and changed the wording on a couple questions.  After the quiz I asked staff if they would add anything or change anything and they all replied that it was good.
       Based on the results of the survey, the majority of the staff are happy with the Literacy Place program.  They would like more resources and find that the teacher guide is helpful but they do not use all the sections within it.
      I would also like to create a quiz and give it to the students that are participating in Literacy Place guided reading.  It would be interesting to see if students think that Literacy Place is engaging and if they can see the connection between what they are doing and the intended outcomes.
     I found that because I know all the people who completed the survey and work in the school that I could anticipate most of their answers.  I think that if I was going to evaluate a program it would be better if I was not a stakeholder.
     I also found that when I was creating my questions I had a hard time creating open ended questions because I was trying to find out specific information.  When I did include open ended questions the answers were minimalist and didn't provide much insight.

Assignment 5 - Revised Quiz

Click here for Assignment 5 - revised quiz

Assignment 5 - Quiz before testing.

Click here to view.

Assignment 4

Click here for assignment 4.

Assignment 3

Click here for Assignment 3.

Assignment 2

I chose the CIPP model because it was the most effective model for this case study.  The CIPP model “provides a systematic way of looking at many different aspects of the curriculum development process.” (http://www.cglrc.cgiar.org/icraf/toolkit/The_CIPP_evaluation_model.htm) This is a new program and was continually being adjusted to suit the needs of the participants.  

Context

The context explains who will be targeted and the need for the course.  It explains that First Nations women are at risk during pregnancy of developing Gestational Diabetes Mellitus (GDM).  It continues to explain the critical and important external factors which include a lack of exercise during pregnancy. 

Inputs

Inputs discuss the feasibility of the program.  Are there supports in place?  What type of resources and equipment will be required?  This report discusses the hiring of staff to organize and target the intended audience.  A partnership with the YMCA was formed to provide services for the women.  There were a number of free services and supports put in place to entice the women into participating.

Process

The process describes how well the program was received and utilized by the intended audience.  With this being a pilot program, the program continually adapted to the audience to include many supports and services to the women.  This helped to maintain attendance and participation in the program. 

Product

Product addresses the question “Was the program successful?”. This program had a low attendance rate, only 7% of the potential target population.  They had to include a number of services for free just to maintain attendance and participation in the program.  This section should include feedback of the overall experience from the women who participated.

Assignment One

Program Evaluation is important to ensure the quality and effectiveness of a program.  I chose to look at the evaluation that was done on “Big Brothers and Big Sisters”. http://www.issuelab.org/fetch/publicprivate_ventures_104.pdf

Evaluation Model

This program was evaluated using the Provus– Discrepancy evaluation model. 

The Provus method identifies four specific stages of all programs: Design Content, Installation, Process, and Product.  Within each of these stages there are important questions to consider when evaluating a program.

Design Content – “Is the program adequately defined?”

Installation – “Is the program installed as described in Design Content?”

Process – “Are the resources and techniques being used congruent with the goals of the program?”

Product – “Are the program objectives achieved in the implementation?”

(http://www.slideshare.net/AgroSmekad/provuss-discrepency-evaluation-model)

“Making a Difference – An Impact Study of Big Brothers and Big Sisters”

“Making a Difference – An Impact Study of Big Brothers and Big Sisters” was written by Joseph P. Tierney, Jean Baldwin Grossman, with Nancy L. Resch.  This evaluation was completed to answer some questions about the mentoring process.  The major purpose for this report was to produce evidence that mentoring programs are meaningful and worthwhile.  It also attempts to explain what makes a mentoring program successful or unsuccessful.  The focus of this report is on the “Big Brothers and Big Sisters“(BBBS) program.  This organization was chosen because it has the most success.  This evaluation completes an impact study of eight BBBS sites and analyzes the data to prove that it is a worthwhile organization and in which areas they are effective and why. 

Pros

Design Content – “Is the program adequately defined?”

The program and the purpose are clearly defined and include a substantial amount of background information.  It discusses the criteria for participants and explains how the focus group will be determined.

Installation – “Is the program installed as described in Design Content?”

The program is laid out exactly how it is stated in the Design Content area.  They explain the intention and what is going to happen and they follow through.

Process – “Are the resources and techniques being used congruent with the goals of the program?”

There is clear cut data showing that the effect of having a mentorship program is important and it makes a difference in behaviour, academics, and drugs and alcohol.

Product – “Are the program objectives achieved in the implementation?”

The product is achieved.  They wanted data to support the importance of mentorship programs and they were diligent and organized with the handling of information and were successful.

Areas for Future Research allows the reader to know exactly where this evaluation could grow and what questions were left after the impact study.

Cons

There was only one study used for this evaluation.  Also this evaluation was done 20 years ago and it would be beneficial to re-evaluate and complete another impact study.

The study left many questions unanswered that could use an additional impact study.

Some of the reports that were created to substantiate evidence are difficult to understand and make sense of quickly.  To understand them completely you would have to go fairly in-depth into the impact study.

Overall

I think they did a really good job of evaluating and creating an impact study.  They were also very organized with the information that they gathered.   The table of contents made it easy to find what I was looking for and where to find it.