INNOVATIVE THINKING
RIGOROUS APPROACHES
REAL WORLD IMPACT

 

Program Evaluations

An industry leader in public program evaluations, IMPAQ designs, develops, and implements leading-edge approaches to impact and program evaluations.

What gives IMPAQ the edge is its pool of talented researchers who are able to quickly adapt to clients’ changing needs. We consider problems from many perspectives, as our teams bring together economists, statisticians, and other social scientists with experts and programmers experienced in cleaning, organizing, and displaying large, disparate data sets.

IMPAQ teams apply a broad range of statistical techniques, developing simulations, modeling complex systems, and performing econometric analyses. While many firms perform parallel qualitative and quantitative analyses, IMPAQ combines these methods so that they inform each other to broaden and deepen insights. The results provide not just the “what” but the “why,” with easy-to-understand evidence-based summary reports that include practical recommendations for improving programs and policy.

IMPAQ has evaluated some of the world’s largest and most complex public programs, including those of the Department of Labor (DOL), the Department of Health and Human Services, the Agency for International Development, the World Bank, the Department of Education, the USDA Food & Nutrition Service, foreign governments, colleges, and county and state governments.

What sets us apart are our talented researchers who can adapt to our clients' changing needs. Our diversity allows us to consider problems from many perspectives, and our teams bring together economists, statisticians, and other social scientists with data experts and programmers experienced in cleaning, organizing, and displaying large, disparate data sets.

IMPAQ started as a program evaluation firm. In fact, the IMPAQ name is derived from our unique expertise and experience conducting random assignment and quasi-experimental program evaluation studies. Today, impact evaluations remain a core service offering.​

IMPAQ’s program evaluation services include:

Click on any of the links below to learn more about our evaluation services.


Impact Evaluations

Impact evaluations can assess intended and unintended changes that may be attributed to a particular intervention. IMPAQ helps clients answer key questions for evidence-based policy making, such as: what works, for what subgroup, and at what cost?

For example, IMPAQ successfully evaluated the impact of entrepreneurship training on labor market outcomes for Project GATE, sponsored by the DOL. The firm designed and implemented an evaluation to address two questions:

  1. Could Project GATE be replicated?  
  2. Was Project GATE effective at increasing business ownership, employment, and self-sufficiency?

You can read the final report here.

IMPAQ’s impact evaluations have become integral to global efforts to improve the effectiveness of public spending, aid delivery, and raise living standards. Click here for examples of impact evaluations IMPAQ has implemented.

Process Evaluations

By examining a program’s planning, implementation, monitoring, quality, and improvement activities, IMPAQ determines relationships between components and outcomes and can provide recommendations for better achieving goals, maximizing resources, and increasing impact. These evaluations are often used to determine whether or not a program should be continued, expanded, refined, or eliminated.

For example, for the Department of Education, Institute of Education Sciences, IMPAQ is conducting a 5-year national evaluation of Comprehensive Technical Assistance Centers. This system of 22 centers was designed to expand the capacity of state education agencies to improve instruction and education outcomes and to close achievement gaps. The IMPAQ evaluation team is collecting activity data, conducting site visits and telephone interviews, and surveying staff and constituents to evaluate the Centers’ designs, implementations, and production outcomes of their technical assistance services.

Outcome Evaluations

Outcome evaluations can measure the change that has occurred as a result of a program implementation or intervention. An outcome evaluation may confirm, for example, the number of people who received HIV prevention education or the resulting increase or decrease in HIV infections as a result of that education.    

An example is IMPAQ’s evaluation of the U.S. Department of Labor’s (DOL) Trade Adjustment Act Community College and Career Training (TAACCCT) initiative. This initiative provides funding to community colleges to develop, improve, and expand adult educational training pathways for careers in the energy and mining industries. IMPAQ has performed or is performing outcome evaluations for single college and consortium TAACCCT awardees, at schools such as Mesa Community College (Arizona), Greenville Technical College (South Carolina), and the Community College of Baltimore County (Maryland). Through the use of rigorous methodologies, social network analysis, online survey focus groups, and other cutting edge approaches, IMPAQ is working to identify mismatches between industry needs and training capabilities and is helping the schools align courses with employer and industry needs to better prepare students for in-demand, high-wage jobs. Because IMPAQ conducts these evaluations across multiple states, it can also help the DOL and the industry with lessons learned and program-level recommendations.

Mixed Methods Evaluations

Mixed-method evaluations can integrate quantitative and qualitative approaches, to strengthen data reliability, validate findings and recommendations, and broaden and deepen understanding of how or why specific outcomes were achieved.

While mixed-method evaluations are widely used in program evaluations, many evaluation firms do not utilize them to their full potential. IMPAQ is an industry leader in this arena, having designed and implemented novel and highly successful approaches such as rapid cycle evaluations, where evaluations are conducted in real-time -- to help clients implement incremental improvements that can make a program more successful.   

For example, IMPAQ is providing The Center for Medicare & Medicaid Services (CMS) with an independent, mixed-method approach structure, process, and outcome evaluation to inform the Report to Congress of the Graduate Nurse Education (GNE) demonstration. The goal of the GNE is to increase the number of Advance Practice Registered Nurses by expanding the amount of clinical training opportunities. For this program, IMPAQ is ascertaining the appropriate growth in the number of registered nurses, the growth for specialties (Clinical Nurse Specialist, Nurse Practitioner, Certified Registered Nurse Anesthetist, and Certified Nurse Midwife), and the costs to the Medicare program. Primary data collection includes in-depth interviews, focus groups, check-in calls, and a quantitative data collection. Secondary analysis involves data from the American Association of Colleges of Nursing, the National Center for Education Statistics, and other sources. 

Learn More

Complete our contact form or call us at (443) 259-5500 to discuss how we can help you collect, analyze, and interpret data.