Inspired by ProPublica’s investigation into the U.S./Mexico border crisis and controversies around Operation Lone Star, I decided to take a look at the Texas Department of Public Safety's (TDPS) Border Report data for the last six years, and answer the following questions:

• How accessible is the data?
• How complete is the data?
• What can I understand about the border crisis given the trends in the data?

• Editor: VSCode
• Python Version: 3.12.0

• General Purpose: os, re, datetime
• Data Manipulation: pandas, numpy geopandas, shapely
• Data Visualization: seaborn, matplotlib

The TDPS's Border Report data can be found on the Texas Unified Crime Reporting website. The report for each year needs to be downloaded individually. Each report is a .xlsx file with three tabs: By Agency, By Month, By Crime Statistic. For the purpose of this project, I analyzed the data on the 'By Agency' tab for the years 2017 to 2023.

Prior to 2023, crime reporting in Texas was voluntary. It was only in 2023 that the Texas Legislature mandated that local law enforcement agencies implement an incident-based reporting system and use it to report data and statistics to the Unified Crime Reporting (UCR) program¹. Currently, the TDPS is transitioning between the FBI's Summary Reporting System (SRS) to the new National Incident-Based Reporting System (NIBRS). The best way to understand the difference between the two reporting methods is that SRS typically aggregates crime data into broad categories and totals while NIBRS records detailed information about each individual crime incident (i.e. characteristics of victims and offenders, location and time of the incident, weapons used, and relationship between victim and offender). Despite the additional details in the NIBRS, the reports do not contain any information about the immigration status of either the victims or offenders, and although NIBRS has been approved for general use since March 1988 ², only 73% of the U.S.'s law enforcement agencies are participating as of the third quarter in 2023³.

The report covers 85 law enforcement agencies, 6 law enforcement agency types 14 counties and 11 NIBRS crime types.

Here's a breakdown of the agency types within the report:

• Murder and Nonnegligent Manslaughter: The willful (nonnegligent) killing of one human being by another, excluding deaths caused by negligence, attempts to kill, assaults to kill, suicides, accidental deaths, and justifiable homicides.

• Negligent Manslaughter: The killing of another person through gross negligence, typically involving a death that occurs as a result of careless actions showing a lack of concern for the life and safety of others.

• Rape: Includes the penetration, no matter how slight, of the vagina or anus with any body part or object, or oral penetration by a sex organ of another person, without the consent of the victim. This category includes Sexual Assault with an Object and Sodomy.

• Robbery: Taking or attempting to take anything of value from a person or persons by force or threat of force or violence and/or by putting the victim in fear.

• Assault: An unlawful attack by one person upon another for the purpose of inflicting severe or aggravated bodily injury. This category includes Aggravated Assault and Simple Assault.

• Burglary: The unlawful entry of a structure to commit a felony or theft, without the use of force to gain entry.

• Larceny, Theft: The unlawful taking, carrying, leading, or riding away of property from the possession or constructive possession of another.

• Motor Vehicle Theft: The theft or attempted theft of a motor vehicle, including automobiles, trucks, buses, motorcycles, scooters, snowmobiles, and other vehicles.

• Arson: Any willful or malicious burning or attempt to burn, with or without intent to defraud, a dwelling house, public building, motor vehicle or aircraft, or personal property of another.

• Human Trafficking, Commercial Sex Acts: The induction of a person to perform a commercial sex act through force, fraud, or coercion, or where the person induced is under 18 years of age.

• Human Trafficking, Involuntary Servitude: The obtaining of a person to perform labor or services through the use of force, fraud, or coercion for the purpose of subjecting that person to involuntary servitude, peonage, debt bondage, or slavery.

Population varies by agency and report year. There are 24 law enforcement agencies reporting a population of zero, but these the majority of these agencies (70.83%) are responsible school or university safety:

In 2023, there were approximately 2.6 million people living in the border counties. On average over the last six years, the population decreased by 4.65%, but Hidalgo (McAllen), El Paso (El Paso) and Starr County are growing:

El Paso and McAllen are both critical border towns, and the growth in Starr County appears to be due to its lower housing costs and proximity to McAllen.

• 01_consolidate_and_check_data
  • Combines the yearly reports (2017 - 2023)
  • Checks to make sure the agencies and crime categories are the same across each report
• 02_enrich_dataset.ipynb
  • Identifies the type of agency (sheriff's office, police department, marshal's office, school police, university police)
  • Calculates NIBRS contribution percentage per agency and report, adjusting for the eligible time period
  • Creates quarterly and yearly cohorts based upon when an agency started to contribute to NIBRS
  • Creates 'adoption_status' indicating if the agency started sending NIBRS data before, on or after the FBI's transition to NIBRS only in 2021

Data from the TDPS is hard to access and comprehend. Searching 'texas crime data' on Google will generate Crime in Texas | Department of Public Safety as the first link, but as of February 14, 2024, the most recent report is a 64 page PDF from 2022 that is difficult to read due to its preference for tables over graphs. The fourth result for 'texas crime data' will lead to the TDPS's Uniform Crime Reporting System (UCRS) website, but data can only be accessed via the 'Reports' option. Here users will find an option under 'Texas Reports' to download the 'Border Report' by year, but may have issues with the SQL server timing out. The data is only provided as a .xlsx file, and either programming or advanced Excel skills are required to combine the yearly reports into a usable format.

In 2021, the FBI stopped accepting SRS data and only accepted NIBRS data in order to fully modernize its crime reporting system⁴. Based upon the reported NIBRS start dates in the TDPS border reports, 88.23% of law enforcement agencies in the border counties had either transitioned or were in the process of transition in 2021, but it seems like the remaining agencies are still struggling to submit data.

As a result, the overall data completeness for the entire dataset is 49.31 %, and only two agencies (Roma PD and San Juan PD) submitted data for every month between 2017 and 2023.

Given the NIBRS data completeness issues, I normalized the data by adjusting the crime counts on the weighted contribution of active agencies reporting the crimes per year. This makes the data comparable across years regardless of the number of reporting agencies or their reporting intensity.

From this graph, we can see that larceny-theft and assault are the two most commonly record crimes. However, it seems that the border counties are rarely reporting human trafficking:

In six years, there has only been 37 reports of human trafficking-commercial sex acts and 310 reports of human trafficking-involuntary servitude. These numbers likely misrepresent the reality considering the severity of human trafficking on the US-Mexico border. Both large criminal organizations and smaller local groups target migrants for commercial sexual or labor exploitation⁵.

Given the inaccessibility and incompleteness of the TDPS's border report data, it is difficult to draw any conclusions about the border crisis. In addition, subject matter expertise is required to understand what the different crime types represent, and how they may or may not be indicative of the current situation on the border. To make this data more accessible to the public, the TDPS should provide clear documentation on what the report is designed to do as well as a data dictionary.

• Scrape news sources for allegations of corruption and incompetence within the various law enforcement agencies on the border
• Understand the resources and funding for the various law enforcement agencies on the border

MIT License