TYPES OF EVALUATION

Four major types of evaluation are used in the case studies: formative. process, outcome, and, to a lesser extent, impact.

In 1992, the National Cancer Institute (NCI) described each of the four types of evaluation in its widely consulted publication Making Health Communication Programs Work: A Planner's Guide (Source: U.S. Department of Health and Human Services, Public Health Service, National Institutes of Health, Office of Cancer Communications, NCI, NIH Publication No. 92­1493).

Formative: Formative evaluation, including pretesting, is designed to assess the strengths and weaknesses of materials or campaign strategies before implementation. It permits necessary revisions before the full effort goes forward. Its basic purpose is to maximize the chance for program success before the communication activity starts.

The case studies reviewed by the subcommittee reveal that formative research and evaluation methods were used extensively by PHS agencies.

Process: Process evaluation is the examination of procedures and tasks involved in implementing a program. This type of evaluation can also include examination of administrative and organizational aspects of the program.

Outcome: Outcome evaluation is used to obtain descriptive data on a project and to document short-term results. Task­focused results are those that describe the output of the activity (e.g., the number of public inquiries received as a result of a public service announcement). Short-term results describe the immediate effects of the project on the target audience (e.g., percentage of the target audience showing increased awareness of the subject). Information from an outcome evaluation may include the following:

Impact: Impact evaluation is the most comprehensive of the four evaluation types. It is desirable because it focuses on the long-range results of the program, including changes or improvement in health status. Impact evaluations are rarely possible because they are usually costly, involve extended commitment, and may depend upon strategies besides communication. Also, the results often cannot be directly related to the effects of an activity or program because of other (external) influences on the target audience that occur over time. Information obtained from an impact study may include

Michael Ramah, vice president at Porter/Novelli, and Beverly Schwartz, director of social marketing at the Academy for Educational Development (AED), discussed the role of these four major types of evaluation in communication interventions. Appendix 2 provides an overview of evaluation strategies for assessing the efficiency and effectiveness of interventions. These strategies may include focus groups, observation studies, survey and baseline data, pretesting, and tracking.

Return to Table of Contents

Return to Committee Reports Page