Case Studies

From multi-year task orders with a team of subcontractors to smaller-scale individual research efforts, L&M has led dozens of research projects from government agencies, as well as private clients.

External Quality Review Organization

District of Columbia Health Care Finance (under subcontract to Delmarva Medical Foundation), February 2007 through January 2019

Objective:

Assist the District of Columbia’s Department of Health Care Finance (DHCF) in evaluating the quality of care provided to Medicaid recipients enrolled in a managed care organization (MCO) by serving (with Delmarva Foundation) as the External Quality Review Organization (EQRO).

Challenges:

Because of the number of standards and the short timeframe for conducting the compliance reviews, the project requires a thorough understanding of health care regulations, as well as an efficient, organized, and detail-oriented approach that helped the MCOs prepare the required information for the on-site reviews.

Approach:

Prior to the most recent review in 2015, the team assisted with the development of comprehensive guidance for the MCOs, detailing the specific items required to document compliance with Medicaid regulations and DHCF contract standards, in addition to outlining the objectives, timelines, and process for the annual review.

The team annually conducts desk and on-site reviews with each MCO operating a Medicaid plan in the District of Columbia, assessing the plans’ compliance with a set of performance standards based on the Centers for Medicare & Medicaid Services (CMS) document, “A Health Care Quality Improvement System (HCQIS) for Medicaid Managed Care,” the Code of Federal Regulations, and DHCF requirements. In its review findings, the team documented the level of compliance for each element and component and provided detailed recommendations to assist DHCF and MCOs in addressing any compliance issues.

During the 2015 contract year, in light of a court case around the Early Periodic Screening Diagnosis and Treatment (EPSDT) Program - Salazar Salazar v. District of Columbia, the EQR review focused on validating EPSDT performance measures reported by the MCOs. L&M reviewed medical records and supplemental data provided by MCOs to document their compliance with requirements of the EPSDT Program and to ensure that elements such as blood lead screenings occur. The team provided detailed recommendations to assist DHCF and MCOs in addressing any compliance issues.


Results

Our Team
  • Improved the transparency of the review process by working with DHCF to identify key objectives for the 2015 review and assisting with the development of written guidance to communicate these objectives to the MCOs.
  • Optimized review efficiency by developing and implementing a pre-site visit survey, standardizing the process for collecting and cataloging plan documentation, and drafting and employing detailed checklists to ensure thorough, consistent, and fair reviews
  • Provided actionable recommendations to assist DHCF and Medicaid MCOs in improving the quality of care for more than 130,000 Medicaid managed care enrollees in the District of Columbia.

Medicare Advantage Plan and Benefit Trends and MA-FFS Cohort Analyses

Centers for Medicare & Medicaid Services (CMS), September 2008 through September 2009 (Base Year)

Objective:

Assist CMS in understanding trends in benefits and cost-sharing arrangements in Medicare Advantage (MA) plans and how these impact beneficiaries, particularly those with chronic conditions.

Challenges:

The team was tasked with analyzing and then summarizing hundreds of dimensions of MA plans in a meaningful way. This required both the technical expertise to build and work with a data set incorporating several years of data and multiple data sources, as well as the ability to highlight the most important findings and translate complex statistical analyses for a policy-making audience. Another challenge involved pricing out-of-pocket costs for individuals with chronic conditions when no claims data for these individuals were available, requiring a sophisticated matching analysis, as well as a side analysis to test the validity of the approach.

Approach:

L&M analyzed Health Plan Management System (HPMS) data for each MA plan to understand how premiums, cost sharing, and supplementary benefits had been changing over the period 2006 to 2010; how these changes impacted the average consumer; and how the impacts varied by geographic location and plan type. For the impact analysis, the team focused on variations in out-of-pocket costs for Medicare beneficiaries with chronic conditions under fee-for-service (FFS) compared to those enrolled in MA plans. The team created a matched sample of FFS and MA-enrolled beneficiaries and developed a complex algorithm using Medicare claims data to assess utilization patterns across FFS chronic condition cohorts. These utilization patterns were then mapped to the MA plan benefits structures to calculate out-of-pocket costs for MA-enrolled beneficiaries with different chronic conditions at different severity levels and identify drivers of higher out-of-pocket costs. A side analysis using Medicare Expenditure Panel Survey data, which includes utilization for individuals in FFS and MA, was conducted to validate the appropriateness of applying FFS utilization patterns to an MA population.


Results

Our Team
  • Organized, analyzed, and deciphered a vast amount of data to provide a snapshot of changes in MA plans and how these changes impacted beneficiaries
  • Helped CMS better understand the out-of-pocket costs of beneficiaries with chronic conditions, elucidating the plan features that most impacted these costs for different conditions
  • dentified areas of focus to improve monitoring and oversight of MA plans

CMS has exercised the first three option years under this task order contract; the team is continuing longitudinal analyses of plan and benefit trends and MA-FFS cohort analyses through the 2013 contract year.

Adding Hospital Readmission Data to the Hospital Compare Web Tool

Centers for Medicare & Medicaid Services (CMS), September 2008 – February 2009

Objective:

Assist CMS in integrating quality information related to hospital readmission rates into the Hospital Compare Web tool.

Challenges:

The team was tasked with developing concise, consumer-friendly explanations and displays of highly technical information. Health care quality measures are often difficult for consumers to understand and use, but the complexity of the readmission measures, which rely on a complicated risk adjustment methodology, heightened the challenge.

Approach:

With input from CMS and a panel of experts, L&M drew on its technical expertise in health care quality and plain language communications to develop language and paper-based mockups of hospital readmission information. The mockups were consumer tested using one-on-one cognitive interviews to assess participants’ comprehension of and reaction to the information. During the second phase, L&M oversaw the development of a Web-based prototype incorporating revised language and displays, which was tested with both consumers and clinicians.

Based on the research, L&M developed screen-by-screen recommendations for integrating the readmission information into the Web tool. The team then hosted a call with CMS staff and contractors to discuss how to fine-tune the recommendations to balance CMS’s policy, communications, and technology needs for the integration.


Results

Our Team
  • Optimized consumer understanding of the readmissions measures while ensuring the technical accuracy of the information throughout the textual descriptions and graphical displays
  • Enhanced consumer interpretation by developing measure descriptions and supporting information based on participant reactions to the information
  • Assisted CMS in appropriately layering the information on the site so that users are introduced to increasingly detailed explanations of the measures as they seek additional information within the Web tool

Evaluation of the Medicare Accountable Care Organizations (ACOs)

Centers for Medicare & Medicaid Services (CMS), July 2012 through December 2016

Objective:

Evaluate the extent to which the Center for Medicare & Medicaid Innovation’s (CMMI’s) Pioneer and Advance Payment Medicare Shared Savings Program (AP MSSP) ACOs improved population health, improved the quality of care, and controlled the cost of care and determined which organizational characteristics, strategic decisions, and operational practices contributed to achieving program goals.

Challenges:

The L&M team and our partner organizations were tasked with determining the effect that CMMI-supported ACOs had on population health, cost, and quality. Given that participation in the program was voluntary and non-random, the team was challenged to devise comparison groups to determine the effect of the ACO initiative. This required building and working with a multi-year data set from numerous sources to create comparison groups that shared key attributes with each of the ACO (intervention) market.

In addition, another challenge this analysis presented was the intervention was not specified; meaning organizations could design their own approach to meeting the three goals of the program (improve population health, quality of care and control cost of care). In order to understand the many approaches to responding to the intervention, the team collected and synthesized information about these complex organizations across a wide range of topics, e.g., leadership and governance, provider network development and engagement, marketplace/environment, organizational performance evaluation, financial planning and management, care management, information management/ health information technology (HIT) physician engagement / performance evaluation and access to care.

Approach:

The L&M team employed a mixed methods approach that included collecting and analyzing primary data collection, as well as secondary data analyses. In the final analysis, these activities were complementary. The team collected primary data through quarterly telephone assessments and site visits with a cohort of 80 Medicare ACOs comprising both intervention and comparison ACOs. These data were recorded in Dedoose (a qualitative analysis software) and coded to facilitate analyses of the interviews. Secondary data were drawn from the CMS Chronic Condition Warehouse (CCW), which includes administrative data, such as the Master Beneficiary Summary Files (MBSF) as well as claims data. The team prepared a master analytic file that linked MBSF, Hierarchical Condition Categories (HCC), and claims files. Using this file, the team identified a baseline and performance year treatment group of beneficiaries assigned/aligned to the CMMI ACOs as well as baseline and performance year comparison groups of Medicare beneficiaries.


Results

Our Team
  • Collected and analyzed a vast amount of data from over 700 telephone interviews in addition to nearly 80 site visits with ACOs
  • Adapted analytical approaches to accommodate the organization-designed interventions the 80 ACOs used to meet program goals
  • Helped CMS better understand the Pioneer and AP MSSP ACO models and how their organizational characteristics, strategic decisions, and operational practices contribute to performance

Reports

  • Evaluation of CMMI Accountable Care Organization Initiatives—Evaluation of Skilled Nursing Facility 3-Day Pioneer ACO Waiver Final Report; December 5, 2016
  • Evaluation of CMMI Accountable Care Organization Initiatives—Pioneer ACO Final Report; December 2, 2016.
  • Evaluation of CMMI Accountable Care Organization Initiatives—Advance Payment ACO Final Report; November 25, 2016.
  • Evaluation of CMMI Accountable Care Organization Initiatives—Pioneer ACO Evaluation Findings from Performance Years One and Two; March 10, 2015.
  • Evaluation of CMMI Accountable Care Organization Initiatives—Effect of Pioneer ACOs on Medicare Spending in the First Year; November 3, 2013.

Registry of Patient Registries (RoPR)

Agency for Healthcare Research and Quality (AHRQ) (With partners Academy Health and Quintiles), April 2014 through April 2017

Objective:

Assist the Agency for Healthcare Research and Quality (AHRQ)’s efforts to maintain and promote registration to the RoPR¹ system; establish a web-based interactive forum for discussing topics related to the methodology of registries; implement and refine the Outcome Measures Framework (OMF) for use in the RoPR, and; develop a series of white papers that will be incorporated in the fourth edition of the registry handbook, Registries for Evaluating Patient Outcomes: A User’s Guide.

Challenges:

The need for a system such as RoPR arises from the variation in outcome and quality measures used across studies. These variations can have a substantial impact on study findings and introduce challenges in the aggregation and comparability of results across studies. While the goals of increased collaboration, reduced redundancy, and improved transparency through the RoPR system are widely supported by stakeholders, the team faces a number of logistical and theoretical challenges in refining the OMF informational model and increasing awareness and registration in the RoPR system.

Approach:

With the assistance of a Technical Expert Panel (TEP) and key informants, the L&M team continually built on the existing OMF conceptual model to develop an information model that could be easily implemented within the RoPR. First, the team conducted a literature review of existing systems used to standardize language and definitions for outcomes measures and other data elements, including systems for registries, clinical trials, and quality reporting. Based on the literature review, the team then designed an information model that would further develop the outcomes measure framework and inform the development of an Internet-based system that would collect and display information on outcome measures used in patient registries in a standardized way. This system has been termed the Outcome Measures Repository (OMR). Through a qualitative analysis of four condition areas (asthma, depression, rheumatoid arthritis, and cardiac surgery), the team evaluated the level of clinical and analytical detail included in specific measures and determined how outcome measures across different registries can be categorized.

The L&M team is also organizing a web-based collaborative registries forum, which will serve as an interactive forum to discuss rapidly evolving areas of registry methodology and best practices. Similarly, the team has engaged in dissemination activities including webinar series, development of manuscripts about the RoPR for peer-reviewed journal submission, and creation of white papers addressing emerging issues in registry science. These papers may be adapted as chapters or may be incorporated into new editions of the User’s Guide, published yearly.


Results

Our Team
  • Completed a literature review and TEP activities to inform the development and refinement of the OMF. In 2016, the team presented the information model to operationalize the OMR
  • Planned and held six webinars, focusing on the design, development, launch, purpose, and capabilities of the RoPR system
  • Developed ten white papers that will be adapted as chapters or will be incorporated into the Fourth edition of the Registries for Evaluating Patient Outcomes: A User’s Guide handbook (expected publication in 2019)
  • Presented data-harmonization related topics at conferences such as the European League Against Rheumatism (EULAR), the AHRQ Research Meeting, and DIA (Develop, Advance, Innovate)
  • Prepared abstracts and manuscripts for publications in Health Affairs and Bioscience Technology

¹ The Registry of Patient Registries (RoPR), established in 2012, is a database of existing patient registries that was designed with extensive stakeholder participation to promote collaboration, reduce redundancy, and improve transparency in registry-based research. It contains information on existing registries, such as the registry description, classification, and purpose, as well as the registry sponsor’s interest in collaboration opportunities.