Assignment #3: Evaluation Assessment

Planning a Program Evaluation

MUSICAL FUTURES PROGRAM

Engage Stakeholders

Who should be involved?

Administrators, Teachers, Students, Parents, School Community

How might they be engaged?

All stakeholders will have a participatory role with the evaluation  to enhance future applications and implementation of the program.

Focus the Evaluation

What are you going to evaluate?  Describe program (logic model).

This program evaluation will focus on the Musical Futures Program.  This pedagogical model of music education is being used to expand and implement music programs in schools within the Prairie Spirit School Division.  The hope from the school division is that this framework/model will expand student engagement in meaningful music participation and provide a structure which is more cost effective than more traditional instrumental instruction which currently exists.

What is the purpose of the evaluation?

The intent of the evaluation is to assess whether or not the program delivers what it claims and if it is actually providing authentic quality of music education  for students.  Will the goals of the school division be met by using this program model?

Who will use the evaluation? How will they use it?

Who/users	How will they use the information?
School Division	They will be able to assess the effectiveness of the program by the ratio of student involvement and cost inputs, staff feedback, community feedback, positive results related to student engagement in music (academic/social)
Administrators/School Community	They will be able to assess the effectiveness of the program by the ratio of student involvement and cost inputs, staff feedback, community feedback, positive results related to student engagement in music (academic/social)
Students Parents	Students – will be able to have evidence to advocate for their interests or needs educationally Parents – will be able to make recommendations for future needs of students and families

What questions will the evaluation seek to answer?

Why has this program been selected as the model for music curriculum implementation?

Is this program providing an authentic music curriculum which allows for real life application of music skills?

Has the program increased student enrollment in music instruction?

Is this a successful model for supporting non-musically trained teachers to implement music instruction?

What information do you need to answer the questions?

What I wish to know	Indicators – How will I know it?
What is the program philosophy?	Referencing the program philosophy and identifying how the school division has implemented it.
What makes this a curriculum versus activity? Is the program assessed for successful long term outcomes?	Examination of the program curriculum goals and long term growth/potential.
Does this program meet the music needs of stakeholders?	Examination of the feedback from stakeholders – positive, negative, suggestions for future applications.
Has the program increased student music engagement?	Identification of student enrollment numbers and life span of program in schools, is there an identifiable change in value of music in schools?

When is the evaluation needed?

Since this is a potential NEW program being implemented in the Fall of 2013, it would be helpful to initial assessments now and follow it throughout the implementation during the 2013-14 school year.

What evaluation design will you use?

Scriven - Summative and Formative

Collect the information

What sources of information will you use?

Existing information:	Previous program evaluations
People:	School Administrators, Teachers, Students, Parents, School Community
Pictorial records and observations:	Music activities, lessons, events, school climate observations related to music program

What data collection method(s) will you use?

x	Survey		Document review
x	Interview		Testimonials
x	Observation		Expert panel
	Group techniques		Simulated problems or situations
	Case study		Journal, log, diary
	Tests		Unobtrusive measures
x	Photos, videos		Other (list)

Instrumentation: What is needed to record the information?

Individual Stakeholder Surveys, Interview Questionnaires, Observation Survey,

When will you collect data for each method you’ve chosen?

Method	Before program	During program	Immediately after	Later
Survey	x		x
Observation		x	x	x
Interview		x	x	x

Will a sample be used? 

No	x
Yes		If yes, describe the procedure you will use.

Analyze and Interpret

How will the data be analyzed?

Data analysis methods:	Quantitative and Qualitative date will be tabulated and sorted
Who responsible:	As the program evaluator I will be directly involved with this process

How will the information be interpreted—by whom?

It would be beneficial to bring the information back to the schools/division to hold a focus group to determine how

the information will be used for future application of the program.

What did you learn? What are the limitations?

The ultimate goal is to learn whether or not this type of pedagogical approach to music instruction is meeting the

needs/interests of students and whether or not it diminishes the value/undermines a traditional instrumental music

program.  Limitations to the program evaluation is that it demands long term observation of sustainability of the

Program.

Use the Information

How will the evaluation be communicated and shared?

To whom	When/where/how to present
All stakeholders	Focus group meeting, planning meeting

Next steps?

Does the success or failure of this program affect current music programs already in place which are more traditional in structure and philosophy?

Manage the evaluation                 Standards

x	Human subject’s protection	x	Utility
	Management chart	x	Feasibility
x	Timeline		Propriety
x	Responsibilities		Accuracy
	Budget