![Overview of Sessions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (14)
Ähnlich wie Program Evaluation In the Non-Profit Sector
Ähnlich wie Program Evaluation In the Non-Profit Sector (20)
Program Evaluation In the Non-Profit Sector
- 1. The American Nonprofit Sector and Philanthropy I Fall 2010 Program Evaluation Kelly Wishart, MA, CEC Administrator, Professional Development and Quality Children’s Friend Providence, RI
- 20. Deciding what to evaluate.
- 22. Deciding what to evaluate: Source: Measuring Program Outcomes: A Practical Approach, United Way of America (1996)
Hinweis der Redaktion
- (Are you implementing the services or training that you initially planned to implement? Are you reaching the intended target population? Are you reaching the intended number of participants? Are you developing the planned collaborative relationships?) (Are participants exhibiting the expected changes in knowledge, attitudes, behaviors, or awareness? Can these changes be attributed to the program?)
- process evaluation - An evaluation that examines the extent to which a program is operating as intended by assessing ongoing program operations and whether the targeted population is being served. A process evaluation involves collecting data that describes program operations in detail, including the types and levels of services provided, the location of service delivery, staffing; sociodemographic characteristics of participants; the community in which services are provided, and the linkages with collaborating agencies. A process evaluation helps program staff identify needed interventions and/or change program components to improve service delivery. It is also called formative or implementation evaluation. outcome evaluation - Evaluation designed to assess the extent to which a program or intervention affects participants according to specific variables or data elements. These results are expected to be caused by program activities and tested by comparison of results across sample groups in the target population. Also known as impact and summative evaluation. formative evaluation - A type of process evaluation of new programs or services that focuses on collecting data on program operations so that needed changes or modifications can be made to the program in its early stages. Formative evaluations are used to provide feedback to staff about the program components that are working and those that need to be changed. summative evaluation - A type of outcome evaluation that assesses the results or outcomes of a program. This type of evaluation is concerned with a program's overall effectiveness.
- qualitative data - Information that is difficult to measure, count, or express in numerical terms. For example, a participant's impression about the fairness of a program rule/requirement is qualitative data. quantitative data - Information that can be expressed in numerical terms, counted or compared on a scale. For example, improvement in a child's reading level as measured by a reading test.
- Step 1: Assemble an evaluation team. Planning and executing an evaluation should be a team effort. Even if you hire an outside evaluator or consultant to help, you and members of your staff must be full partners in the evaluation effort. If you plan to hire an outside evaluator or an evaluation consultant, Chapter 4 provides information on hiring procedures and managing an evaluation that involves an outside professional. Step 2: Prepare for the evaluation. Decide what to evaluate, building a program model, stating your objectives in measurable terms, and identifying the context for the evaluation. The more attention you give to planning the evaluation, the more effective it will be. Step 3: Develop an evaluation plan. An evaluation plan is a blueprint or a map for an evaluation. It details the design and the methods that will be used to conduct the evaluation and analyze the findings. You should not implement an evaluation until you have completed an evaluation plan. Step 4: Collect evaluation information. Once you complete an evaluation plan, you are ready to begin collecting information. This task will require selecting or developing information collection procedures and instruments. Step 5: Analyze your evaluation information. After evaluation information is collected, it must be organized in a way that allows you to analyze it. Information analysis should be conducted at various times during the course of the evaluation to allow you and your staff to obtain ongoing feedback about the program. This feedback will either validate what you are doing or identify areas where changes may be needed. Step 6: Prepare the evaluation report. The evaluation report should be a comprehensive document that describes the program and provides the results of the information analysis. The report should also include an interpretation of the results for understanding program effectiveness.
- Outside Eval. Advantages: do not have a stake in the evaluation's findings, the results may be perceived by current or potential funders as more objective. may have greater expertise and knowledge than agency staff about the technical aspects involved in conducting an evaluation. Outside evaluators may offer a new perspective to program operations The evaluation may be conducted more efficiently if the evaluator is experienced. Outside Eval Disadvantages: Hiring an outside evaluator can be expensive. Outside evaluators may not have an adequate understanding of the issues relevant to your program or target population. Internal staff Lead/Outside support Advantage : May be less expensive than hiring an outside evaluator (this is not always true). The use of an agency staff member as a team leader may increase the likelihood that the evaluation will be consistent with program objectives. Possible disadvantages: The greater time commitment required of staff may outweigh the cost reduction of using the outside professional as a consultant instead of a team leader. A professional evaluator used only for consulting purposes may not give as much attention to the evaluation tasks as may be needed. Like the Team 3 option, Team 2 may be perceived as less objective than using an outside evaluator. Total I-in House advantage: An in-house evaluation team may be the least expensive option, but this is not always true. An in-house staff evaluation team promotes maximum involvement and participation of program staff and can contribute to building staff expertise for future evaluation efforts. Possible disadvantages: An in-house team may not be sufficiently knowledgeable or experienced to design and implement the evaluation. Potential funders may not perceive evaluation results as objective.
- Whatever team you select, remember that you and your staff need to work with the evaluation team and be involved in all evaluation planning and activities. Your knowledge and experience working with program participants and the community are essential for an evaluation that will benefit the program, program participants, community, and funders. If your answer to all the resource questions is "no," you may want to consider postponing your evaluation until you can obtain funds to hire an outside evaluator, at least on a consultancy basis. You may also want to consider budgeting funds for evaluation purposes in your future program planning efforts. If your answer to question 1 is "yes," but you answer "no" to all other questions, you will need maximum assistance in conducting your evaluation and Team 1 (an outside evaluator with in-house support) is probably your best choice. If you answer "no" to question 1, but "yes" to most of the other resource questions, then Team 3 (in-house staff only) may be an appropriate choice for you. Keep in mind, however, that if you plan to use evaluation findings to seek program funding, you may want to consider using the Team 2 option (in-house evaluation team with outside consultant) instead and trying to obtain evaluation funds from other areas of your agency's budget. If your answer to question 1 is "yes" and the remainder of your answers are mixed (some "yes" and some "no") then either the Team 1 or Team 2 option should be effective.
- Tradition models have ended at the OUTPUTS box, now more than ever we need to show the OUTCOMES! Inputs These are materials that the organization or program takes in and then processes to produce the results desired by the organization. Types of inputs are people, money, equipment, facilities, supplies, people's ideas, people's time, etc. Inputs can also be major forces that influence the organization or programs. For example, the inputs to a nonprofit program that provides training to clients might include learners, training materials, teachers, classrooms, funding, paper and pencils, etc. Various laws and regulations effect how the program is conducted, for example, safety regulations, Equal Opportunity Employment guidelines, etc. Inputs are often associated with a cost to obtain and use the item -- budgets are listings of inputs and the costs to obtain and/or use them. Processes (or Activities or Strategies or Methods) Processes are used by the organization or program to manipulate and arrange items to produce the results desired by the organization or program. Processes can range from putting a piece of paper on a desk to manufacturing a space shuttle. However, logic models are usually only concerned with the major recurring processes associated with producing the results desired by the organization or program. For example, the major processes used by a nonprofit program that provides training to clients might include recruitment of learners, pretesting of learners, training, post-testing and certification. Outputs Outputs are usually the tangible results of the major processes in the organization. They are usually accounted for by their number, for example, the number of students who failed or passed a test, courses taught, tests taken, teachers used, etc. Outputs are frequently misunderstood to indicate success of an organization or program. However, if the outputs aren't directly associated with achieving the benefits desired for clients, then the outputs are poor indicators of the success of the organization and its programs. You can use many teachers, but that won't mean that many clients were successfully trained. Outcomes Outcomes are the (hopefully positive) impacts on those people whom the organization wanted to benefit with its programs. Outcomes are usually specified in terms of: a) learning, including enhancements to knowledge, understanding/perceptions/attitudes, and behaviors b) skills (behaviors to accomplish results, or capabilities) c) conditions (increased security, stability, pride, etc.)
- Used more often for program development and planning; measurement objectives may not have a time-bound statement in them.