Hest, Robert
Senior Research Fellow

p 612-624-5507
e hestx005@umn.edu

Changing Population Estimates: Implications for Data Users

.

The National Center for Health Statistics (NCHS) at the Centers for Disease Control (CDC) recently changed the population estimates they use to calculate population-level rates in their National Vital Statistics System (NVSS), such as birth rates and mortality rates that are commonly accessed through tools such as CDC WONDER. These population estimates allow for the production of rates by providing a population denominator against which to compare the number of births or deaths.

Beginning with the 2021 data year, CDC transitioned from using "bridged-race" population estimates based on the race and ethnicity categories specified in the 1977 OMB Race and Ethnic Standards for Federal Statistics and Administrative Reporting to using "single-race" population estimates from the U.S. Census Bureau (“Census”) based on the race and Hispanic ethnicity categories specified in the 1997 OMB standards for the collection of data on race and ethnicity. The bridged-race estimates had been created to better align the population estimates from Census that used the 1997 OMB categories with the birth and death records from vital statistics systems that in many cases still used the 1977 OMB categories. This transition to the single-race population estimates appears to be precipitated by Census' recent improvement to their collection of race and ethnicity data beginning with Census 2020 data collection and continuing with subsequent years of the American Community Survey.

This move to single-race population estimates has several implications for data users:

• Statistics based on the single-race population estimates are only made available via CDC WONDER for data years 2018 forward, and statistics based on bridged-race population estimates are only available through data year 2020. Data users should not make comparisons between statistics based on bridged-race population estimates and statistics based on single-race population estimates. Further, because of changes made in 2020 to how Census measures race and ethnicity, caution should be used when comparing statistics by race and ethnicity from 2020 and later to data from 2019 and earlier.
   
• CDC WONDER has expanded the number of race and ethnicity categories available to data users. Previously, only five race categories and two Hispanic ethnicity categories were available. This has been expanded to up to 31 race categories and two Hispanic ethnicity categories.
   
• The single-race population estimates produced by Census suppress the number of persons less than five years of age at the county level to comply with Census' privacy policies. This prevents users from calculating age-adjusted rates at the county level or by level of urbanization using the single-race data, though crude rates can still be calculated at these levels of geography.

This change also affects SHADAC’s Suicide Deaths measure on State Health Compare. That measure’s Race and Ethnicity breakdown is now available in separate series for data years 1999–2020 and 2018–2021; estimates should not be compared between these series. Furthermore, SHADAC is unable to update the Metropolitan Status breakdown for this measure past data year 2020 due to the previously explained county-level suppressions.

Going forward, data users should keep changes in the denominator in mind when comparing annual estimates based on CDC vital statistics data. SHADAC strives to account for these types of considerations on State Health Compare by making clear when data years aren’t comparable and providing data users with relevant context around changes in methodology and data collection.

SHADAC Staff

p 612.624.4802
e shadac@umn.edu

Race/Ethnicity Data in CMS Medicaid (T-MSIS) Analytic Files: 2020 Data Assessment

The Transformed Medicaid Statistical Information System (T-MSIS) is the largest national database of current Medicaid and Children’s Health Insurance Program (CHIP) beneficiary information collected from U.S. states, territories, and the District of Columbia (DC).¹ T-MSIS data are critical for monitoring and evaluating the utilization of Medicaid and CHIP, which together provide health insurance coverage to more than 90 million people.²

T-MSIS data files are challenging to use directly for research and analytic purposes due to their size and complexity. To optimize these files for health services research, CMS repackages them into a user-friendly, research-ready format called T-MSIS Analytic Files (TAF) Research Identifiable Files (RIF). One such file, the Annual Demographic and Eligibility (DE) file, contains race and ethnicity information for Medicaid and CHIP beneficiaries. This information is vital for assessing enrollment, access to services, and quality of care across racial and ethnic subgroups in the Medicaid/CHIP population, whose members are particularly vulnerable due to limited income, physical and cognitive disabilities, old age, complex medical conditions, housing insecurity, and other social, economic, behavioral, and health needs.    

Completeness of race and ethnicity data reported to CMS remains inconsistent among the states, territories, and DC. To guide researchers and other consumers in their use of T-MSIS data, CMS produces data quality assessments of the race and ethnicity data along with other data such as enrollment, claims, expenditures, and service use. The Data Quality (DQ) assessments for race and ethnicity data have been posted for data years 2014 through 2020 and indicate varying levels of “concern” regarding race and ethnicity data completeness. Some data years have multiple data versions (e.g., Preliminary, Release 1, Release 2), each with its own DQ assessment. This blog explores 2020 Data Release 1, the most recent T-MSIS race and ethnicity data for which a DQ assessment is available.

Evaluation of T-MSIS Race and Ethnicity Data
DQ assessments for each year and release of T-MSIS data are housed in the Data Quality Atlas (DQ Atlas), an online evaluation tool developed as a companion to T-MSIS data.³ The DQ Atlas assesses T-MSIS race and ethnicity data using two criteria: the percentage of beneficiaries with missing race and/or ethnicity values in the TAF; and the number of race/ethnicity categories (out of five) that differ by more than ten percentage points between the TAF and American Community Survey (ACS) data. Taken together, these two criteria indicate the level of “concern” (i.e., reliability) for states’ T-MSIS race/ethnicity data. Five “concern” categories appear in the DQ Atlas: Low Concern, Medium Concern, High Concern, Unusable, and Unclassified. States with substantial missing race/ethnicity data or race/ethnicity data that are inconsistent with the ACS – a premier source of demographic data – are grouped into either the High Concern or Unusable categories, whereas states with relatively complete race/ethnicity data or race/ethnicity data that align with ACS estimates are grouped into either the Low Concern or Medium Concern categories. The Unclassified category includes states for which benchmark data are incomplete or unavailable for a given data year and version.  

To construct the external ACS benchmark for evaluating T-MSIS data, creators of the DQ Atlas combine race and ethnicity categories in the ACS to mirror race and ethnicity categories reported in the TAF (see Table 1). More information about the evaluation of T-MSIS race and ethnicity data is available in the DQ Atlas’ Background and Methods Resource.

Table 1. Crosswalk of Race and Ethnicity Variables between the TAF and ACS    

Table 2. Race and Ethnicity Data Quality Assessment, 2020 T-MSIS Analytic File (TAF) Data Release 1

Notes: *T-MSIS includes all 50 states, the District of Columbia (DC), and the U.S. territories of Puerto Rico (PR) and the Virgin Islands (VI). A DQ assessment is not available for VI in the 2020 TAF (Data Version: Release 1) due to incomplete/unavailable data. VI is therefore the only state/territory categorized as “Unclassified” in the 2020 TAF (Data Version: Release 1), and does not appear in Table 2.

Visualizing T-MSIS Data in the DQ Atlas
The DQ Atlas enables users to generate maps that compare the quality of T-MSIS data between states across different topics, such as race/ethnicity, age, income, and gender (see Figure 1). Visualizing T-MSIS data in this manner can help researchers quickly assess the completeness of a single variable as well as the relative completeness (or incompleteness) of certain variables compared to others. For example, in the 2020 TAF Data Release 1, all states and territories received a “low concern” rating for age data, whereas only 29 states and territories received a “low concern” rating for family income.

Figure 1. Data Quality Assessments of Beneficiary Information by U.S. State/Territory

Notes: Green = low concern; yellow = medium concern; orange = high concern; red = unusable; grey = unclassified.
Source: Medicaid.gov. (n.d.). DQ Atlas: Race and Ethnicity [2020 Data set: Version: Release 1]. Available from https://www.medicaid.gov/dq-atlas/landing/topics/single/map?topic=g3m16&tafVersionId=25. Accessed January 5, 2023.

Looking Ahead
Increasingly, a wide diversity of voices from non-profits, health insurers, state-based marketplaces, and policymakers have called for improving the collection of race, ethnicity, and language data, often with the goal of advancing health equity. CMS’s efforts to improve the quality and availability of T-MSIS data reflect this nationwide movement toward data collection practices that more accurately capture the diversity of the U.S. population.

In June 2022, CMS released updated technical instructions for reporting beneficiary race, including clarification on how to report race information for beneficiaries who report multiple races. That same month, the Biden Administration announced its intent to revise the federal government’s standards for surveying race and ethnicity – a policy change that is expected to result in disaggregated categories for Hispanic individuals and people of Middle Eastern or North African descent.⁴

California and New York have enacted historic policies to disaggregate race/ethnicity data within the past year: California now requires state agencies to list a separate category for Black descendants of enslaved people when collecting state employee data; and New York now requires state agencies to disaggregate Asian, Native Hawaiian, and Pacific Islander data into more granular collection categories (e.g., Chinese, Filipino, Hawaiian, Samoan).^5,6 It is likely that other states will follow these data disaggregation practices in the years ahead as public awareness of issues related to diversity, equity, inclusion, and racial justice continues to grow.  

Sources
¹ Medicaid.gov. Transformed Medicaid Statistical Information System (T-MSIS). Retrieved October 20, 2022, from https://www.medicaid.gov/medicaid/data-systems/macbis/transformed-medicaid-statistical-information-system-t-msis/index.html#

² Medicaid.gov. September 2022 Medicaid & CHIP Enrollment Data Highlights. Retrieved on January 5, 2023, from https://www.medicaid.gov/medicaid/program-information/medicaid-and-chip-enrollment-data/report-highlights/index.html

³ Saunders, H., & Chidambaram, P. (April 28, 2022). Medicaid Administrative Data: Challenges with Race, Ethnicity, and Other Demographic Variables. Kaiser Family Foundation. Retrieved October 31, 2022, from https://www.kff.org/medicaid/issue-brief/medicaid-administrative-data-challenges-with-race-ethnicity-and-other-demographic-variables/

⁴ Wang, H.L. (June 15, 2022). Biden officials may change how the U.S. defines racial and ethnic groups by 2024. NPR. Retrieved November 1, 2022, from https://www.npr.org/2022/06/15/1105104863/racial-ethnic-categories-omb-directive-15

⁵ Diaz, J. (August 16, 2022). California becomes the first state to break down Black employee data by lineage. NPR. Retrieved November 1, 2022, from https://www.npr.org/2022/08/16/1117631210/california-becomes-the-first-state-to-break-down-black-employee-data-by-lineage

⁶ The New York State Senate. (December 22, 2021). Assembly Bill A6896A. Retrieved November 2, 2022, from https://www.nysenate.gov/legislation/bills/2021/A689