Validation of PIPs is one of the activities that the External Quality Review Organization (EQRO) conducts on an annual basis. Medicaid Managed Care Rules state that PIPs underway during previous 12 months be validated. The purpose of a PIP is to achieve, through ongoing measurements and interventions, significant improvement sustained over time in clinical or nonclinical areas.
As part of the Florida Agency for Health Care Administration’s (Agency’s) procurement of the Statewide Medicaid Managed Care (SMMC) health plan contracts in SFY 2018–2019, the Agency required the health plans to conduct four PIPs and the the dental plans to conduct three PIPs.
*These state-mandated PIP topics were not initiated by Children’s Medical Services-S because the PIP topics were not applicable to the population served by the health plan. Children’s Medical Services-S instead submitted two additional PIPs for validation.
^ This PIP topic was not submitted by Florida Community Care-L because the PIP topic is not applicable to the Long-Term Care (LTC) Plus population served by the health plan.
The primary objective of PIP validation is to determine the MCO’s compliance with the requirements of 42 CFR §438.330(d)(2)(i-v), including:
- Measuring performance using objective quality indicators.
- Implementing system interventions to achieve improvement in quality.
- Evaluating effectiveness of the interventions.
- Planning and initiating of activities for increasing or sustaining improvement.
In its PIP evaluation and validation, HSAG used the Department of Health and Human Services, Centers for Medicare & Medicaid Services (CMS) publication, Protocol 1. Validation of Performance Improvement Projects: A Mandatory EQR-Related Activity (CMS Protocol 1), February 2023.1 HSAG evaluation of the PIPs includes two key components of the quality improvement (QI) process:
- The technical structure of the PIPs to ensure the MCOs designed, conducted, and reported PIPs using sound methodology consistent with the CMS protocol for conducting PIPs. HSAG’s review determined whether the PIP design (e.g., PIP Aim statement, population, sampling methods, performance indicator, and data collection methodology) is based on sound methodological principles and could reliably measure outcomes. Successful execution of this component ensures that reported PIP results are accurate and capable of measuring sustained improvement.
- HSAG evaluated the implementation of a PIP and once designed, a MCOs effectiveness in improving outcomes depends on the systematic data collection process, analysis of data, and the identification of barriers and subsequent development of relevant interventions. Through this component, HSAG evaluated how well a MCO improves its rates through implantation of effective processes (i.e., barrier analyses, interventions and evaluation of results).
The goal of HSAG’s PIP validation is to ensure that the Agency and key stakeholders can have confidence that the MCO executed a methodologically sound improvement project, and any reported improvement is related to and can be reasonably linked to the QI strategies and activities conducted by the MCO during the PIP.
The following are steps used by HSAG to validate each PIP:
Step 1. Review the Selected PIP Topic
Step 2. Review the PIP AIM Statement
Step 3. Review the Identified PIP Population
Step 4. Review the Sampling Method
Step 5. Review the Selected Performance Indicator(s)
Step 6. Review the Data Collection Procedures
Step 7. Review the Data Analysis and Interpretation of PIP Results
Step 8. Assess the Improvement Strategies
Step 9. Assess the Likelihood that Significant and Sustained Improvement Occurred
HSAG’s PIP validation process is a desk review process; therefore, it is important for the MCOs to submit as much detail as possible at the time of submission. All attachments should be submitted separately and not embedded into the PIP submission form.
For any questions related to the PIP process, please contact:
Christi Melendez, RN, CPHQ
Executive Director
Performance Improvement Projects
Telephone: 602.801.6875
Email: cmelendez@hsag.com
To request a technical assistance call or to receive copies of the PIP templates, please contact:
Jen Montano
Project Manager
Performance Improvement Projects
Telephone: 602.801.6851
Email: jmontano@hsag.com
1 Department of Health and Human Services, Centers for Medicare & Medicaid Services. Protocol 1. Validation of Performance Improvement Projects (PIPs): A Mandatory EQR-Related Activity, February 2023. Available at: https://www.medicaid.gov/sites/default/files/2023-03/2023-eqr-protocols.pdf.
Performance Measure Validation (PMV) is one of the activities that the External Quality Review Organization (EQRO) conducts on an annual basis. Medicaid Managed Care Rules require validation of state-required performance measure data for managed care organizations (MCOs), prepaid inpatient health plans (PIHPs), prepaid ambulatory health plans (PAHPs), and primary care case management (PCCM) entities with financial incentives.
The purpose of the validation process is to:
- Evaluate the accuracy and validity of the performance measures reported by or calculated on behalf of the organizations;
- Determine the extent to which the performance measures were calculated according to specifications required by the state.
Health Services Advisory Group, Inc., (HSAG) conducts the validation activities as outlined in the Centers for Medicare & Medicaid Services (CMS) Performance Measure Validation Protocol. In some states, such as Florida, external auditors conduct HEDIS Compliance Audits™. Florida HMOs, PAHPs, and PIHPs submit their audited data and audit reports to the Agency for Health Care Administration (AHCA). The Florida Medicaid EQRO obtains the audit reports and final performance measure data from AHCA each year for the validation process.
These are steps taken by HSAG to complete the validation process:
Step 1. Request information from MCOs
Step 2. Review MCO information and compliance audit reports
Step 3. Perform independent review of reported rates
Step 4. Aggregate and categorize validation findings
For questions related to the validation of performance measures process, please contact:
Elisabeth Hunt, MHA, CPCS, CHCA
Executive Director: Data Science & Advanced Analytics-Audits
Health Services Advisory Group, Inc.
EHunt@hsag.com
HEDIS® is a registered trademark of the National Committee for Quality Assurance (NCQA)
HEDIS Compliance Audit™ is a trademark of the National Committee for Quality Assurance (NCQA)
Medicaid Managed Care Rules require that either the state, its designated agent, or the external quality review organization (EQRO) must conduct a review at least every three years to monitor compliance. The Agency for Health Care Administration (AHCA) has elected to perform compliance review activities for its managed care plans.
For questions related to the review of compliance, please contact:
Kim Elliott, PhD, CPHQ, CHCA
Executive Director, State & Corporate Services
602.801.6759
KElliott@hsag.com
Accurate and complete encounter data are critical to the success of any managed care program. State Medicaid agencies rely on the quality of encounter data submissions from its contracted health plans in order to monitor and improve the quality of care; establish performance measure rates; generate accurate and reliable reports; and obtain utilization and cost information. The completeness and accuracy of these data are essential in the state's overall management and oversight of its Medicaid managed care program and in demonstrating its responsibility and stewardship.
HSAG’s approach to evaluating encounter data for completeness and accuracy is consistent with the current Centers for Medicare & Medicaid Services (CMS) External Quality Review (EQR) Protocol 5, Validation of Encounter Data Reported by the Medicaid and CHIP Managed Care Plan. In alignment with the protocols, HSAG incorporates one or more of the following core evaluation activities into its encounter data validation (EDV) studies:
- Information systems (IS) review—assessment of State’s and MCOs’ information systems and processes
- Administrative profile—analysis of State’s electronic encounter data completeness and accuracy
- Comparative analysis—analysis of State’s electronic encounter data completeness and accuracy through a comparative analysis between State’s electronic encounter data and the data extracted from the MCOs’ data systems
- Medical records review—analysis of State’s electronic encounter data completeness and accuracy through a comparative analysis between State’s electronic encounter data and the medical records
HSAG’s approach provides an effective way to identify and confirm the quality of encounter data collected and maintained by states while providing a clear roadmap toward improvement.
For questions related to the review of encounter data validation, please contact:
Amy Kearney
Director, Data Science and Advanced Analytics
602.801.6886
akearney@hsag.com
Focused studies can provide states with valuable information to assist them in evaluating managed care organization (MCO) performance on a variety of topics. Comparisons of MCO performance against specified standards, national benchmarks, or with each other will provide the Agency for Health Care Administration (AHCA) with information on which to base quality improvement efforts.
Examples of focused studies Health Services Advisory Group (HSAG) have conducted are described below:
- "Adolescent Well-Child Care". HSAG developed the study methodology, including the study questions, study indicators, data collection process, and analysis plan.
- The "Adolescent Well-Child Care" focused study involved data collection from medical records. MCOs were responsible for collecting the necessary medical records after HSAG identified the selected sample cases.
- The “Identification of Special Health Care Needs” focused study relied entirely on administrative data obtained from specified state departments and the MCOs.
HSAG will continue to conduct focused studies as requested by AHCA.
For questions related to the review of focused studies, please contact:
Kim Elliott, PhD, CPHQ, CHCA
Executive Director, State & Corporate Services
602.801.6759
KElliott@hsag.com
Key information related to all external quality review (EQR) activities must be made available to all interested parties, including managed care organizations, various divisions within the Agency for Health Care Administration, and key stakeholders. In order to meet this objective, Health Services Advisory Group (HSAG) uses several mediums to communicate with the key groups, including a public website, a secured File Transfer Protocol (FTP) site for the exchange of sensitive data or information, electronic mail, conference calls, quarterly meetings with key stakeholders, quarterly meetings, and written communications.
In addition, HSAG has established internal mechanisms to continually evaluate the effectiveness of each approach and ensure that information is being communicated in the most effective, timely, and direct manner possible.
HSAG also provides educational opportunities during quarterly meetings, the answers to frequently asked questions on the website, and links to other key resources related to EQR services.
For questions related to the dissemination and education activities, please contact:
Kim Elliott, PhD, CPHQ, CHCA
Executive Director, State & Corporate Services
602.801.6759
KElliott@hsag.com
State Medicaid agencies are required by the Social Security Act, section 1932(c), to provide for an annual external, independent evaluation of quality outcomes and the timeliness of, and access to, services provided by the state’s Medicaid managed care organizations (MCOs). Medicaid Managed Care Rules require state Medicaid agencies to provide an annual report of this evaluation using information from mandatory quality review activities.
Health Services Advisory Group (HSAG) compiles an annual technical report each year that includes the following information for each quality review activity conducted:
- Objectives
- Technical methods of data collection and analysis
- Descriptions of the data obtained
- Conclusions from the data, including improvement opportunities
- Assessment of the MCOs' strengths and weaknesses with respect to quality, access, and timeliness
- Recommendations
The Florida Agency for Health Care Administration’s annual technical reports can be viewed on the Resources page of this website.
For questions related to the technical report, please contact:
Kim M. Elliott, PhD, CPHQ, CHCA
Executive Director, State & Corporate Services
602.801.6759
KElliott@hsag.com
