Individual ACO experiences vary

In September 2016, the Center for Medicare and Medicaid Innovation (CMMI) selected NORC at the University of Chicago to conduct an independent evaluation of the Next Generation Accountable Care Organization (NGACO) Model program. On August 27, 2018, CMMI released NORC’s first report¹ on the findings of its evaluation, which included NORC’s estimate of the impact of the NGACO program on Medicare Part A and Part B spending (gross impact) in 2016. NORC defined the net impact of the program by combining this gross impact with the results of the NGACO program’s shared savings and losses. These results were published by CMMI in October 2017² (i.e., shared savings and losses). This Milliman white paper combines the aggregate gross impact of each of the NGACOs shown in the NORC report with the shared savings/(loss) results of each NGACO to calculate the net impact of each individual NGACO.

Comparison of results

NORC estimates that the gross impact of the NGACO program in 2016 across all NGACOs was a reduction in Medicare Part A and Part B expenditures of $100.1 million while the net impact was $62.1 million.³ In Figure 1, we show the 2016 gross impact, the shared savings/(loss) results, the net impact, and whether the NGACO is still in the NGACO program in 2018 for each of the 18 NGACOs (the list is sorted from greatest to least net impact). The Shared Savings/(Loss) column in Figure 1 shows the NGACO’s actual payment between it and Centers for Medicare and Medicaid (CMS) in 2016.

The top five NGACOs in Figure 1 had a combined positive net impact of $69.0 million (i.e., they reduced Medicare Part A and Part B spending) but only two of the five remained in the NGACO program in 2018. In addition, the top two NGACOs with the greatest net impact also had the greatest shared losses and left the NGACO program.

The bottom five NGACOs in Figure 1 had a combined negative net impact of ($32.6) million (i.e., they increased Medicare Part A and Part B spending) and four of the five remained in the NGACO program in 2018. The bottom two NGACOs with the lowest net impact also had the greatest and third-greatest shared savings. Both are still in the NGACO program.

Figure 2 groups ACOs separately into those that had shared savings and those that had shared losses as calculated under the NGACO program’s parameters in 2016. Contrary to expectations, those NGACOs with shared losses had a positive net impact as calculated by NORC and this positive net impact was significantly greater than the positive net impact of those with shared savings.

Different methodologies

The NGACO program’s methodology for developing benchmarks and shared savings/(losses) results is significantly different from NORC’s methodology for assessing the impact of the NGACO program on Medicare Part A and Part B spending.

The NGACO program’s methodology for developing the 2016 benchmark:

An NGACO’s shared savings/(losses) are the difference between this benchmark and its actual 2016 expenditures, applying the NGACO-selected risk-sharing percentage (i.e., 80% or 100%).

Conversely, NORC’s methodology for estimating the gross impact of the NGACO program uses a “difference in differences” design, comparing changes in expenditures for NGACO-aligned beneficiaries and a “comparison group” receiving usual care before a baseline period from 2013 to 2015 and after 2016, the onset of the NGACO model’s incentives. The comparison beneficiaries were propensity-score-weighted to be similar to those in the NGACO group on domains of:

Any difference in the change in costs from the baseline period to 2016 for the NGACO population versus the comparison population is assumed to be the result of the NGACO’s actions.

Unlike the NGACO methodology, the NORC methodology does not have a standard discount of 2.25% nor regional and national efficiency adjustments. The gross impact is calculated by comparing the NGACO-aligned beneficiaries’ expenditures to the comparison group’s expenditures (not the comparison group’s expenditures less the 2.25% and regional and national adjustments). The gross impact of $100.1 million across all NGACOs is a 1.7% reduction in expenditures.

Discussion of results

Clearly, the NGACO program’s savings/(loss) calculation leads to financial results for the NGACOs that are significantly different from the “difference in differences” methodology used by NORC.

If we assume that the NORC methodology is a more credible way of measuring an NGACO’s impact on reducing expenditures than the NGACO program methodology, then the existing methodology may be providing perverse incentives to NGACOs. In this case, the existing methodology may be encouraging NGACOs that are reducing Medicare spending (i.e., with a positive net impact) to leave the program while encouraging those that are increasing Medicare spending (i.e., with a negative net impact) to stay in the program.

Conversely, if we assume the existing NGACO program methodology is a more credible way of measuring shared savings/(losses), then the NORC report may lead NGACOs to incorrectly conclude that the NGACO program shared savings/(losses) methodology is flawed. If an NGACO believes CMS is paying money to NGACOs that are driving up Part A and B expenses and collecting money from NGACOs that are decreasing Part A and B expenses, it could decide to leave the program based on this inaccurate conclusion.

Either way, NGACOs may want to examine whether their care management efforts are producing the type of savings as measured by the program or as measured by NORC in order to determine whether they should consider modifying those care management efforts or discontinuing participation in the program.

Caveats and limitations

In preparing this white paper, we relied on the publicly available data from CMS. We have not audited or verified this data or the validity of the methodologies used. If the underlying data or information is inaccurate or incomplete, the results of our analysis may likewise be inaccurate or incomplete. In that event, the results of our analysis may not be suitable for the intended purpose.