Policymakers, journalists and the public turn to data released annually by the Federal Bureau of Investigation (FBI) to understand national and state crime trends. Recently, the FBI rolled out a plan to modernize its criminal data reporting. Here’s what these changes mean for our understanding of recent crime trends.

What data does the FBI collect and why is it important?

For nearly a century, the FBI has collected data on known law enforcement offenses from state and local agencies. Through its Uniform Crime Reporting program, the FBI aggregates these reports, applies quality control standards, uses estimates to fill gaps in their information, and then releases both the raw data and trend analysis.

Researchers and policy makers have come to rely in particular on an annual publication Uniform Crime Reporting, Crime in the United States, to understand crime trends from the previous year. This report typically contains high-level tables that track important statistics such as national, regional, and national violent crime rates. Each year’s annual report tends to come out about nine months after the end of the year it describes, although the 2021 crime data summary comes out a bit later.

How has the FBI historically collected and reported this data?

The FBI historically used two systems to collect crime data. The older of the two, the Summary Reporting System, tracked “the monthly number of crimes known to law enforcement.” Although easy to use, the system tended to overlook important nuances. Among other things, it only counted the most serious offense in an incident, applying a “hierarchy rule” to determine which offenses were more serious than others. The system also focused on a limited number of crime types and did not capture details such as the number of people involved.

More recently, the FBI and news agencies have begun using the National Incident-Based Reporting System. This system tracks information in much more detail than the summary report system and covers many other types of crimes. The National Incident Based Reporting System also drops the hierarchy rule, allowing law enforcement to report multiple offenses in a single incident. Consider a hypothetical robbery where the victim, injured in the attack, dies from his injuries. Under the summary reporting system, this incident would be recorded only as a murder – the robbery would disappear from FBI records. Under the National Incident Reporting System, robbery and murder would be counted with considerable detail on both offences.

What is changing in the collection and reporting of crime data?

For years, the FBI accepted data in both formats. On January 1, 2021, however, the FBI stopped accepting data through the Summary Reporting System.

Unfortunately, despite the benefits of the new national incident-based reporting system, many state and local law enforcement agencies have yet to make the switch. Law enforcement agencies covering just over half the population reported a full year of data to the FBI in 2021. In comparison, recent FBI reports are based on data from agencies covering more of 95% of the population. The time and expense of updating outdated government computer systems is likely partly responsible for the late adoption of the new system.

The FBI recently began publishing crime data on a quarterly basis. Police service participation has been relatively low so far, but is expected to increase as more agencies make the switch. These quarterly reports, available via an online data dashboard, should eventually become a regular and more reliable source of crime data.

How will these changes affect reported crime data for 2021?

With so many agencies not reporting a full year of data for 2021, this year’s annual crime data release will have significant blind spots.

We know that the agency’s annual report Crime in the United States The report will contain “state-level data” and “a trend study comparing 2020 and 2021 data,” the latter based on partial data supplemented with estimates. It may also contain national estimates of crime trends, that is, data indicating whether rates of murder and other crimes have increased or decreased at the national level. If the FBI does eventually release these national estimates, they will likely be expressed with “confidence intervals” to indicate uncertainty.

Some major cities will also be missing from the data. San Francisco, for example, does not expect to complete its transition to the National Incident Reporting System until 2025, meaning it will be absent from FBI crime data until then. (San Francisco crime trend data will, of course, still be available directly from the city, but not in the new format.) And only 13% of law enforcement in New York State have ​​reported a full year of data to the FBI in 2021. New York City was not among them. On the other hand, some areas of the country with a high adoption rate of the National Incident Reporting System – such as Michigan, Texas and Virginia – will have strong data, allowing researchers to use FBI data to study crime trends in specific countries. state in great detail.

What challenges does this present for our understanding of crime trends?

It will likely be impossible to speak of an accurate “national” murder rate or “national” violent crime rate for 2021. Confidence intervals can make it difficult to determine whether rates for certain offenses have increased or decreased. Policy makers will need to exercise great caution when using these limited data.

Some challenges will remain even after full adoption of the National Incident Reporting System. Because it allows for the reporting of multiple “violations” per “incident”, comparing pre- and post-transition data can create a false appearance of an increase in certain violations. Remember the robbery example above: the summary reporting system would count it as one crime — murder — while the national incident-based reporting system would count it as of them violent crimes of this incident – a murder and a robbery. A 2019 study of early adopters of the National Incident Reporting System suggests that approximately 90% of incidents involve a single violation, and the increase in transition violations may be on the order of 2 at 5 %. But that might still be enough to influence the perception of “rising” or “falling” crime.

What about other crime data sources for 2021?

Importantly, the FBI is not the only source of public safety information. For one, the Centers for Disease Control provides national 3-month and 12-month homicide trends. The data, which uses a different methodology to track deaths but aligned with FBI estimates in 2020, shows homicides are rising at a slower rate than in 2020.

Researchers and think tanks have also produced their own analyzes of crime. Most notably, the Criminal Justice Council publishes regular reports on crime trends, including a 2021 year-end data collection from 27 cities. Its findings follow those of the CDC, suggesting murders increased in 2021 but at a much slower rate than the year before. Trends were mixed in other crime categories, which are more difficult for private organizations to study. Crime data analyst Jeff Asher also tracks year-to-date murders in select cities on a data dashboard, and his counts currently show a drop in murders in major cities in 2022.

Finally, while the FBI tracks offenses known to law enforcement, it cannot account for crimes that are never reported. The National Crime Victimization Survey, released by the Bureau of Justice Statistics, seeks to capture the difference using a national survey to estimate the rate at which people live non-fatal crimes and violence. This survey shows that non-fatal violent crime rates decrease in 2020 and increase very slightly in 2021.