From multi-year task orders with a team of subcontractors to smaller-scale individual research efforts, L&M has led dozens of research projects from government agencies, as well as private clients.
Agency for Healthcare Research and Quality (AHRQ), March 2019 through March 2021
Provide scientific, analytical, technical assistance, dissemination, and logistical support to AHRQ and the multi-stakeholder grantee teams engaged in the Pediatric Quality Measures Program (PQMP) learning collaborative. The goal of PQMP is to improve the quality of care delivered to children covered by Medicaid and the Children’s Health Insurance Program (CHIP) through the implementation of newly developed measures across state, health plan, and provider levels. During the first year of the contract, L&M provided analytical support to the grantee teams to answer key research questions identified by AHRQ. Ongoing activities include developing QI toolkits with the grantee teams, documenting contextual factors related to QI efforts using an Implementation Science framework, providing general dissemination support and leading the effort to publish research articles and commentaries in an Academic Pediatrics journal supplement.
The heterogeneity of measures, settings of implementation, and approaches across grantee projects pose a challenge to identifying shared learnings, best practices and generalizable findings about implementation and quality improvement efforts. For grantees to focus their time and resources on carrying out measure implementation and assessment of usability at multiple-levels, AHRQ recognized that grantees would benefit from external support to coordinate activities and identify common findings.
Leveraging its deep under L&M provides wide-ranging support to facilitate collaboration among grantee teams as well as dissemination of findings to end-users. L&M coordinates opportunities for grantees to share their work on measure implementation and assists with the synthesis and delivery of findings across projects. The team engages stakeholders at multiple levels to develop a comprehensive view of the challenges of implementing measures in real-world situations. L&M is also supporting the development of QI toolkits and the production of a journal supplement in Academic Pediatrics to advance the body of knowledge on effective pediatric measure implementation and quality improvement strategies.
AHRQ has exercised the option year for L&M to support activities designed to promote the dissemination and adoption of the PQMP measures.
Centers for Medicare & Medicaid Services (CMS), June 2019 through June 2021 (with an additional option year)
Provide support services and technical assistance to CMS’s Hospital and Ambulatory Policy Group (HAPG) and Medicare Plan Payment Group (MPPG) during the Medicare Physician Fee Schedule (PFS) and Part C/Part D rulemaking cycles. In particular, L&M provides support services by triaging, reviewing, and summarizing public comments received in response to Notices of Proposed Rulemaking (NPRMs). Further activities under the contract include managing CMS’s resource mailbox for processing, tracking, and responding to inquiries throughout the year, providing technical assistance in the review of the preamble and regulations text, and updating existing CMS publications and sub-regulatory guidance.
Extremely tight deadlines for modifying and finalizing proposed policies coupled with a recent surge in the number of public comment submissions has made it increasingly difficult for CMS to manage, review, and respond to the important input provided through public comment. For instance, for the CY 2020 PFS Proposed Rule, CMS received over 40,000 public comments, with more than 18,000 submitted in the final 48 hours of the comment period. Managing this volume of comments in such a short timeframe gives CMS little time to carefully consider public input and make the regulatory and policy decisions needed to finalize and publish the final rule.
Leveraging its deep understanding of the Medicare program, including expertise related to Medicare Advantage, Part D, and the PFS, as well as our past experience reviewing public comments, the L&M team developed a customized web-based Triage Tool to manage the public comments received, facilitate comment review and triaging, store comment review information, and generate reports for CMS. For each of the three public comment periods L&M supported during the base year, the team worked with subject matter experts at CMS to adjust the review process and design report templates to meet the specific needs of each CMS policy team. L&M reviewed every comment received and provided CMS with descriptive information about the comment and submitting individual/organization, identified the topics addressed, and indicated whether the comment supported or opposed the policies in the proposed rule. This review information was packaged for CMS in customized reports and delivered to CMS throughout the comment period.
CMS has exercised the first of the two option periods and tasked the team to develop and implement automation to further enhance operational efficiency.
Centers for Medicare & Medicaid Services (CMS), July 2012 through December 2016
Evaluate the extent to which the Center for Medicare & Medicaid Innovation’s (CMMI’s) Pioneer and Advance Payment Medicare Shared Savings Program (AP MSSP) ACOs improved population health, improved the quality of care, and controlled the cost of care and determined which organizational characteristics, strategic decisions, and operational practices contributed to achieving program goals.
The L&M team and our partner organizations were tasked with determining the effect that CMMI-supported ACOs had on population health, cost, and quality. Given that participation in the program was voluntary and non-random, the team was challenged to devise comparison groups to determine the effect of the ACO initiative. This required building and working with a multi-year data set from numerous sources to create comparison groups that shared key attributes with each of the ACO (intervention) market.
In addition, another challenge this analysis presented was the intervention was not specified; meaning organizations could design their own approach to meeting the three goals of the program (improve population health, quality of care and control cost of care). In order to understand the many approaches to responding to the intervention, the team collected and synthesized information about these complex organizations across a wide range of topics, e.g., leadership and governance, provider network development and engagement, marketplace/environment, organizational performance evaluation, financial planning and management, care management, information management/ health information technology (HIT) physician engagement / performance evaluation and access to care.
The L&M team employed a mixed methods approach that included collecting and analyzing primary data collection, as well as secondary data analyses. In the final analysis, these activities were complementary. The team collected primary data through quarterly telephone assessments and site visits with a cohort of 80 Medicare ACOs comprising both intervention and comparison ACOs. These data were recorded in Dedoose (a qualitative analysis software) and coded to facilitate analyses of the interviews. Secondary data were drawn from the CMS Chronic Condition Warehouse (CCW), which includes administrative data, such as the Master Beneficiary Summary Files (MBSF) as well as claims data. The team prepared a master analytic file that linked MBSF, Hierarchical Condition Categories (HCC), and claims files. Using this file, the team identified a baseline and performance year treatment group of beneficiaries assigned/aligned to the CMMI ACOs as well as baseline and performance year comparison groups of Medicare beneficiaries.
Centers for Medicare & Medicaid Services (CMS), August 2018 through August 2020 (with 3 additional option years)
Assist the State Demonstrations Group (SDG) in the Center for Medicaid and CHIP Services (CMCS) by providing technical and analytical support for evaluation and monitoring of Medicaid Section 1115 demonstrations. Section 1115 of the Social Security Act authorizes the Secretary of the Department of Health and Human Services (HHS) to approve experimental, pilot, or demonstration projects to allow states flexibility to test unique approaches to program design and administration that are aimed at enhancing beneficiary access to quality services. L&M provides assistance with monitoring and evaluation activities by review and assessment of state demonstration program monitoring reports, evaluation designs, evaluation reports, implementation, monitoring, and payment protocols, and other deliverables to assess and provide recommendations concerning comprehensiveness, scientific and technical merits, and adherence to CMS requirements.
There are currently over 50 active Section 1115 demonstrations, with others approved on an ongoing basis. Each demonstration has deliverables outlined in CMS’s Special Terms & Conditions that require review, assessment, and rapid cycle feedback to the state. CMS relies on the project team to conduct reviews and ad hoc requests quickly (usually within two weeks); the team works on multiple assignments simultaneously. L&M provides the CMCS team with technical expertise in research methods and program evaluation, knowledge of process and outcome measures and data sources, as well as deep subject matter expertise in a range of topics (e.g., family planning, substance use disorders, and behavioral health) and populations (such as aged and disabled, pediatric, homeless) relevant to current and future Section 1115 Medicaid demonstrations.
The team routinely provides rapid turnaround guidance to support CMS oversight of state’s Section 1115 Medicaid demonstrations. Among other activities completed in the base year of our project, L&M and our partner organization have reviewed states’ proposed evaluation designs, providing thorough assessments of the designs’ technical merits to CMS with unique demonstration goals and concerns, histories and available data sources in mind. To support CMS’ ongoing monitoring of Section 1115 demonstrations, L&M reviews states’ semi-annual and annual progress reports, including those related to Delivery System Reform Incentive Payment (DSRIP) and Medicaid managed care programs. We have analyzed stakeholder responses to public comment periods for new demonstrations and have compiled metrics that would allow CMS to better monitor these demonstrations.
Agency for Healthcare Research and Quality (AHRQ) (With partners Academy Health and Truven Health Analytics), April 2014 through April 2019
Assist the Agency for Healthcare Research and Quality (AHRQ)’s efforts to maintain and promote registration to the RoPR¹ system; establish a web-based interactive forum for discussing topics related to the methodology of registries; implement and refine the Outcome Measures Framework (OMF) for use in the RoPR, and; develop a series of white papers that will be incorporated in the fourth edition of the registry handbook, Registries for Evaluating Patient Outcomes: A User’s Guide.
The need for a system such as RoPR arises from the variation in outcome and quality measures used across studies. These variations can have a substantial impact on study findings and introduce challenges in the aggregation and comparability of results across studies. While the goals of increased collaboration, reduced redundancy, and improved transparency through the RoPR system are widely supported by stakeholders, the team faces a number of logistical and theoretical challenges in refining the OMF informational model and increasing awareness and registration in the RoPR system.
With the assistance of a Technical Expert Panel (TEP) and key informants, the L&M team continually built on the existing OMF conceptual model to develop an information model that could be easily implemented within the RoPR. First, the team conducted a literature review of existing systems used to standardize language and definitions for outcomes measures and other data elements, including systems for registries, clinical trials, and quality reporting. Based on the literature review, the team then designed an information model that would further develop the outcomes measure framework and inform the development of an Internet-based system that would collect and display information on outcome measures used in patient registries in a standardized way. This system has been termed the Outcome Measures Repository (OMR). Through a qualitative analysis of four condition areas (asthma, depression, rheumatoid arthritis, and cardiac surgery), the team evaluated the level of clinical and analytical detail included in specific measures and determined how outcome measures across different registries can be categorized.
The L&M team is also organizing a web-based collaborative registries forum, which will serve as an interactive forum to discuss rapidly evolving areas of registry methodology and best practices. Similarly, the team has engaged in dissemination activities including webinar series, development of manuscripts about the RoPR for peer-reviewed journal submission, and creation of white papers addressing emerging issues in registry science. These papers may be adapted as chapters or may be incorporated into new editions of the User’s Guide, published yearly.
Centers for Medicare & Medicaid Services (CMS), September 2017 through September 2022
Support CMS outreach and education efforts related to the “Health Insurance Marketplace” – a consumer-facing website that helps individuals shop for and enroll in affordable individual or family health insurance – by providing actionable insights on the information needs of “Marketplace-eligible” consumers.
After a highly-publicized national debate about proposed changes to the Marketplace, enrollment in Marketplace plans has gradually declined year to year. CMS needed to assess current attitudes and perceptions of the program and gauge awareness of the changes that have been implemented as well as proposed. In addition to measuring current attitudes and perceptions, CMS wanted to understand how consumers’ information diets and communication contexts may have shaped attitudes about the Marketplace. Finally, CMS sought to understand how these factors may have differed among two broad segments of the “Marketplace-eligible” populations: uninsured individuals and Marketplace enrollees who recently purchased a Marketplace plan. All of these insights would help inform message development for future enrollment efforts.
L&M first conducted an environmental scan, reviewing relevant grey literature to understand current perceptions of the Marketplace, gauge public awareness of and reaction to changes to the Marketplace (real or proposed), and summarize factors that might explain the decline in the most recent open enrollment period.
L&M then conducted seven focus groups in two different cities to assess overall attitudes about and knowledge of the Marketplace. In each research location, the team conducted two focus groups with uninsured individuals (one with younger individuals age 19 to 34 and one with older individuals age 35 to 64). The team conducted three focus groups with Marketplace enrollees age 19 to 64. One of the three groups consisted only of “new enrollees” who had signed up for a Marketplace for the first time in the most recent enrollment period. The other two included a mix of new and reenrollees. These 90-minute discussions focused on consumer perspectives on the Marketplace and knowledge of recent changes to the Marketplace, barriers to enrollment for uninsured participants, and the “communication context” of participants with regard to health insurance and health care.
Centers for Medicare & Medicaid Services (CMS), September 2008 through January 2014
Assist CMS in understanding trends in benefits and cost-sharing arrangements in Medicare Advantage (MA) plans and how these impact beneficiaries, particularly those with chronic conditions.
The team was tasked with analyzing and then summarizing hundreds of dimensions of MA plans in a meaningful way. This required both the technical expertise to build and work with a data set incorporating several years of data and multiple data sources, as well as the ability to highlight the most important findings and translate complex statistical analyses for a policy-making audience. Another challenge involved pricing out-of-pocket costs for individuals with chronic conditions when no claims data for these individuals were available, requiring a sophisticated matching analysis, as well as a side analysis to test the validity of the approach.
L&M analyzed Health Plan Management System (HPMS) data for each MA plan to understand how premiums, cost sharing, and supplementary benefits had been changing over the period 2006 to 2010; how these changes impacted the average consumer; and how the impacts varied by geographic location and plan type. For the impact analysis, the team focused on variations in out-of-pocket costs for Medicare beneficiaries with chronic conditions under fee-for-service (FFS) compared to those enrolled in MA plans. The team created a matched sample of FFS and MA-enrolled beneficiaries and developed a complex algorithm using Medicare claims data to assess utilization patterns across FFS chronic condition cohorts. These utilization patterns were then mapped to the MA plan benefits structures to calculate out-of-pocket costs for MA-enrolled beneficiaries with different chronic conditions at different severity levels and identify drivers of higher out-of-pocket costs. A side analysis using Medicare Expenditure Panel Survey data, which includes utilization for individuals in FFS and MA, was conducted to validate the appropriateness of applying FFS utilization patterns to an MA population.
CMS has exercised the first three option years under this task order contract; the team is continuing longitudinal analyses of plan and benefit trends and MA-FFS cohort analyses through the 2013 contract year.
District of Columbia Health Care Finance (under subcontract to Qlarant Quality Solutions), March 2009 through June 2017
Assist the District of Columbia’s Department of Health Care Finance (DHCF) in evaluating the quality of care provided to Medicaid recipients enrolled in a managed care organization (MCO) by serving (with Qlarant) as the External Quality Review Organization (EQRO).
Because of the number of standards and the short timeframe for conducting the compliance reviews, the project requires a thorough understanding of health care regulations, as well as an efficient, organized, and detail-oriented approach that helped the MCOs prepare the required information for the on-site reviews.
Prior to the most recent review in 2015, the team assisted with the development of comprehensive guidance for the MCOs, detailing the specific items required to document compliance with Medicaid regulations and DHCF contract standards, in addition to outlining the objectives, timelines, and process for the annual review.
The team annually conducts desk and on-site reviews with each MCO operating a Medicaid plan in the District of Columbia, assessing the plans’ compliance with a set of performance standards based on the Centers for Medicare & Medicaid Services (CMS) document, “A Health Care Quality Improvement System (HCQIS) for Medicaid Managed Care,” the Code of Federal Regulations, and DHCF requirements. In its review findings, the team documented the level of compliance for each element and component and provided detailed recommendations to assist DHCF and MCOs in addressing any compliance issues.
During the 2015 contract year, in light of a court case around the Early Periodic Screening Diagnosis and Treatment (EPSDT) Program - Salazar Salazar v. District of Columbia, the EQR review focused on validating EPSDT performance measures reported by the MCOs. L&M reviewed medical records and supplemental data provided by MCOs to document their compliance with requirements of the EPSDT Program and to ensure that elements such as blood lead screenings occur. The team provided detailed recommendations to assist DHCF and MCOs in addressing any compliance issues.
Centers for Medicare & Medicaid Services (CMS), September 2008 – February 2009
Assist CMS in integrating quality information related to hospital readmission rates into the Hospital Compare Web tool.
The team was tasked with developing concise, consumer-friendly explanations and displays of highly technical information. Health care quality measures are often difficult for consumers to understand and use, but the complexity of the readmission measures, which rely on a complicated risk adjustment methodology, heightened the challenge.
With input from CMS and a panel of experts, L&M drew on its technical expertise in health care quality and plain language communications to develop language and paper-based mockups of hospital readmission information. The mockups were consumer tested using one-on-one cognitive interviews to assess participants’ comprehension of and reaction to the information. During the second phase, L&M oversaw the development of a Web-based prototype incorporating revised language and displays, which was tested with both consumers and clinicians.
Based on the research, L&M developed screen-by-screen recommendations for integrating the readmission information into the Web tool. The team then hosted a call with CMS staff and contractors to discuss how to fine-tune the recommendations to balance CMS’s policy, communications, and technology needs for the integration.