Quantitative and Qualitative Evaluation
SHADAC staff have experience designing and conducting both program- and initiative-level evaluations. In addition, we provide technical assistance related to assessing, monitoring, continuous improvement, performance measurement, and evaluation of health policies and programs, delivery system and payment reforms, and innovation in the Medicaid program. Our more recent program evaluation expertise is formative in nature and relies on both qualitative and quantitative data collection and analysis methods, including document reviews, key informant interviews, focus groups, case studies, and secondary data analyses.
Related SHADAC work:
Click on any title below to learn more about the project.
Opportunities to Learn More About Serving Justice-Involved Individuals Through 1115 Demonstration Evaluations
This issue brief from SHADAC researchers, funded by the California Health Care Foundation (CHCF), identifies the unique opportunities states should consider when designing evaluation plans specific to their justice-involved populations, provides an overview of justice-involved 1115 demonstration initiatives, and summarizes what is known from existing evaluations of these activities. The brief also identifies a set of opportunities to design robust and equity-focused 1115 demonstration evaluation plans specific to justice-involved populations. Given the complex dynamics, unique obstacles, and varied experiences justice-involved individuals face, a new, more equity-focused evaluation approach is needed.
Minnesota Study of Telehealth Expansion and Payment Parity
SHADAC staff served as an evaluation consultant to the Minnesota Electronic Health Records Consortium (MNEHRC). On behalf of the Minnesota Department of Health (MDH), MNEHRC led one component of a comprehensive evaluation to assess the impact of telehealth expansion and payment parity on access to healthcare, quality and outcomes, patient satisfaction, equity, and costs. Using electronic health records (EHR) data, the MNEHRC conducted descriptive and difference-in-difference analyses to evaluate how outpatient use of telehealth changed post-telehealth expansion, the prevalence of audio-only (telephone) and audio-visual (video) telehealth use, and how expansion of telehealth affected quality of care. SHADAC advised on the evaluation design, participated in team meetings to review preliminary findings, and provided feedback on the evaluation report.
CMMI Vermont All-Payer Accountable Care Organization Model Evaluation
SHADAC serves as subcontractor to NORC at the University of Chicago on its evaluation of the Vermont All-Payer Accountable Care Organization (ACO) model, supported by the Center for Medicare and Medicaid Innovation (CMMI). During the evaluation design phase, SHADAC reviewed drafts and advised on measures related to opioid use disorder. During the evaluation implementation phase, SHADAC provided substantive feedback to the NORC evaluation team on versions of their annual reports to CMMI.
Past Work
Medicaid Section 1115 Waiver Demonstration Evaluation Technical Assistance
SHADAC staff provided technical assistance (TA) and expertise to Alaska, Colorado, Illinois, and New Hampshire as these states participated in the Building State Capacity to Evaluate Innovative Medicaid Policies project through a contract with the National Governors Association (NGA). SHADAC monitored federal evaluation guidance and helped states develop data-driven approaches to evaluating approved and proposed Medicaid policies under Section 1115 waiver authority. Activities included: reviewing draft request for proposals and planning documents; participating in convenings with state and federal officials; drafting resources to respond to state-specific evaluation needs; and producing a brief of key considerations and promising tactics for states to use in evaluating new Medicaid policies.
Study of the Impact of the Affordable Care Act Implementation in Kentucky
SHADAC served as the lead evaluator studying the impact of the Patient Protection and Affordable Care Act (ACA) in Kentucky, a project supported by the Foundation for a Healthy Kentucky. This mixed methods study examined the implementation and outcomes of health reform in Kentucky and tracked a range of indicators related to coverage, access, cost, and quality. SHADAC also conducted an assessment of Section 1115 waiver options for advancing Medicaid reforms in the state and wrote an issue brief on the potential of ACA provisions to affect access to SUD treatment, including some basic data on prevalence, access, and more.
Evaluation Design of CentraCare’s Vitality Wellness in Rural Minnesota
The CentraCare Foundation, in cooperation with CentraCare system and CentraCare-Long Prairie leadership, contracted with SHADAC to develop a plan to evaluate the outcomes of a community fitness center, Vitality Wellness, located adjacent to its Long Prairie hospital location. SHADAC completed document review, key informant interviews, and an evaluation design in less than six months. The design included proposed evaluation domains and questions, target groups, measures, qualitative and quantitative data sources, product ideas, and budget considerations.
Evaluation of Minnesota State Employee Group Insurance Program (SEGIP) High Value Diabetes Care Program
SHADAC collaborated with the Minnesota State Employee Group Insurance Program (SEGIP) to evaluate a pilot program for high-value diabetes care. The Advantage Value for Diabetes (AVD) program aims to improve health through effective management of diabetes. The project focused on developing measures and a baseline analysis with two years of administrative claims data, before and after the program was implemented (2017-2018).
SHADAC Brief Summarizes Study of Effects of Medicaid Expansion on Physician Participation
A new study led by Dr. Hannah Neprash is the first to provide direct answers about how clinicians responded to the Medicaid expansion. This brief summarizes key findings from this study, including the effect of expansion on the number of Medicaid appointments and number of Medicaid patients seen by primary care clinicians (i.e., Medicaid participation), the total number of appointments provided (i.e., their labor supply) and the share of those appointments paid for by Medicaid versus private coverage or other payers (i.e., payer mix).