Home / Project Showcase / Process Evaluations and Qualitative Research ProjectsView Printable Version
Project Showcase

Process Evaluations and Qualitative Research Projects

Process Evaluations and Qualitative Research Projects

Featured Project:  
Effective Health Care Program Governance & National Priority-Setting: An Appreciative Inquiry & Evaluation Study

Client:  Agency for Healthcare Research and Quality,
            U.S. Department of Health and Human Services
 
The purpose of this project is to develop a conceptual framework and formalized governance structure for the Agency for Healthcare Research and Quality’s (AHRQ) Effective Health Care (EHC) Program.  Specific tasks being performed include:
  • The conduct of interviews, case studies, and an Appreciative Inquiry Workshop with EHC program staff and stakeholders
  • The design and administration of a web-based survey to gather quantitative inputs from stakeholders
  • The conduct of a cross-organization comparison of approaches with international organizations engaged in a similar mission to the EHC program.
 
The final deliverable for this project will be a set of alternative roadmaps that AHRQ can use to develop a revised, formalized governance structure to lead the EHC program into the future.
 

Community Colleges as the Classroom for America’s Workforce: Best Practices and Lessons Learned from the Field in Working with the Public Workforce System

Client:  Center for Program Planning and Results of the Office of the Assistant Secretary for Administration and Management (OASAM), U.S. Department of Labor

The purpose of this project is to study a sample of community colleges to identify challenges, promising strategies and best practices regarding retraining America’s workforce for in-demand industries.  Specific tasks being performed include:

  • Identifying sample community college sites
  • Conducting site visits, which include in-depth interviews and focus groups with stakeholders
  • Collecting additional data such as web-based surveys, Workforce Investment Act Standardized Record Data (WIASRD), data from the Federal Research and Evaluation Database (FRED), evaluation data from the High Growth Job Training Initiative and Community-Based Job Training Grants, and data from the IMPAQ’s completed WIA Net-Impact Evaluation
  • Conducting qualitative and quantitative data analysis to provide a comprehensive understanding of the interaction between community colleges and local workforce development systems.

The final deliverable for this project is a Final Report summarizing the results of the analysis and providing appropriate recommendations to DOL.  

 

Longitudinal Program Evaluation of HHS Healthcare Associated Infections (HAI) National Action Plan (NAP)

Client: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services

In response to the preventable toll that healthcare-associated infections (HAIs) have had on human life and healthcare expenditures, the Department of Health and Human Services (HHS) released the HHS Draft National Action Plan (NAP) to Prevent Healthcare-Associated Infections in January 2009.  Congress responded to the Draft NAP and funded operative divisions of HHS to begin the implementation of the plan with funds included in the FY 09 appropriations  and funds from The American Recovery and Reinvestment Act of 2009 (ARRA).

During this project, IMPAQ will develop and implement an iterative longitudinal comprehensive program evaluation framework of the Healthcare Associated Infections National Action Plan (HAI NAP).  This evaluation will employ the Context-Input-Process-Product (CIPP) evaluation model to assess overall initiative impacts, outcomes, and the adoption and diffusion of HAI interventions using both qualitative and quantitative assessment approaches.  The results of the ongoing evaluations will be used to provide feedback that will guide improvements to the ongoing and future programs and initiatives of the HAI NAP. Specifically, we will look at the leadership and strategy of the HAI NAP, knowledge development, infrastructure development, HAI prevention practice adoption, and the effects on stakeholder. Thus, this project will involve interviews with agency leaders, key stakeholders, and NAP work group members; document and literature reviews; review of HHS data systems; online surveys; and focus groups.

Communication Dissemination Evaluation

Client: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services
 
The purpose of this project is to conduct a review and comprehensive evaluation of the current dissemination mechanisms employed by the various units of the Agency for Healthcare Research and Quality (AHRQ) to assess their types, methods, and levels of effectiveness.  IMPAQ conducted interviews with AHRQ Center Directors and Portfolio Leads to evaluate the effectiveness of the Office of Communication and Knowledge Transfer’s support of communication dissemination processes, investigated public and private sector use of “Web 2.0” communication mechanisms on health topics, and explored opportunities for AHRQ to establish new and innovative approaches to the dissemination of health-related research and best practices information.
 

Child Care Services Program Assessment and Benchmarking Study

Client:  National Institutes of Health, U.S. Department of Health and Human Services
 
IMPAQ conducted an assessment of the National Instutitues of Health’s childcare service program to determine the effectiveness of the existing program, as well as document areas of improvement for ensuring its continued competitiveness with comparable organizations.  As part of this project, IMPAQ conducted interviews with NIH senior officials, performed a benchmarking study of ten government and private sector organizations recognized as being “best in class,” and administering a web-based survey to NIH employees to assess their satisfaction with the current programs and services being offered.  The results of the study were used to develop program goals and recommendations for future improvements to NIH’s childcare services program.
 
Read the Final Report:
 
 

Process Evaluation for the National Reemployment Technical Assistance Initiative

Client: Employment and Training Administration, U.S. Department of Labor 
 
As part of the overall National Reemployment Technical Assistance Initiative, IMPAQ is conducting a process evaluation of the reemployment practices currently being used by local workforce investment programs at six selected sites across the U.S. that have been recipients of either a Workforce Innovation in Regional Economic Development (WIRED) or Regional Innovation (RIG) grant from the U.S. Department of Labor.  IMPAQ is conducting comprehensive site visits, including the use of structured interviews and focus groups, to evaluate current approaches to program integration, use of labor market information and assessment tools, and the strategies employed by the sites with the goal of identifying best practices which can be shared with other workforce investment agencies throughout the U.S.
 

Evaluation Synthesis

Client: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services 
 
The purpose of this project is to conduct a synthesis of selected evaluations completed for AHRQ since 2004 to determine the organizational and management elements common across the evaluated programs, initiatives, and products that affected or contributed to different levels of success in achieving their goals.  These factors may include, among others, the Agency’s level of support, clarity of goals and objectives, the design and implementation of the evaluation, appropriateness and level of involvement of AHRQ staff, level of demand for the information and products produced, ability to disseminate and spread uptake of positive findings and products, as well as external factors beyond the Agency’s control.  
 
To conduct this study, IMPAQ staff is developing a set of criteria (success factors) which will be applied in reviewing 25 evaluation and research reports conducted by AHRQ and its contractors.  We will also be conducting individual interviews with senior AHRQ staff and contractor staff to identify the factors that contributed to the success or lack of success of the evaluation effort.  IMPAQ will then prepare a synthesis report which identifies the key success factors common across the evaluations which will be presented to AHRQ senior management for use in designing and implementing future research projects.

Applying Lessons Learned in Community Collaboration to Health IT

Client: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services

To augment AHRQ’s existing health IT research, this project focuses on stakeholder collaboration and how models of community planning and development, both in healthcare and other industries, can be used for establishing sustainable Regional Health Information Organizations (RHIO) and health information exchange (HIE) activities. In particular, the project seeks to identify characteristics that distinguish successful and unsuccessful community efforts to collaborate on shared benefits, which can then be replicated to support RHIO and HIE development.  For this project, IMPAQ is:

  • Conducting a comprehensive environmental scan of the literature related to community planning and development, and RHIO and HIE initiatives
  • Convening an “Innovation Meeting” comprised of RHIO and HIE leaders, and national and international community planning experts who will assess the applicability of community collaboration models to sustainable RHIOs and HIE, including identifying best practices, illustrative models, and case studies
  • Preparing a final report synthesizing results and findings from the environmental scan and “Innovation Meeting”, as well as presenting recommendations for ongoing research, implementation, and policy work in the RHIO and HIE fields
  • Developing, executing, and evaluating a communication/dissemination plan that will drive awareness and integration of project findings and recommendations into practice at national, state, and local levels