Police in Heber City, Utah, are re-evaluating an artificial intelligence tool for drafting reports after it incorrectly stated that an officer had transformed into a frog, apparently by misinterpreting background audio from a Disney movie.
According to Salt Lake City broadcaster Fox 13, the department was testing Draft One, an AI system that generates reports from body camera footage, when the error occurred. The software appears to have incorporated dialogue from the film The Princess and the Frog, which was playing in the background during a demonstration of the tool.
“The body cam software and the AI report writing software picked up on the movie that was playing in the background, which happened to be ‘The Princess and the Frog,’” police sergeant Rick Keel told Fox 13. “That’s when we learned the importance of correcting these AI-generated reports.”
The department has been testing Draft One to reduce the time officers spend on paperwork by automatically generating narratives from recorded audio. Even a simple mock traffic stop used to showcase the software produced a draft that needed extensive corrections, Fox 13 reported.
Despite the mistakes, Keel said the system has already reduced his administrative workload.
“I’m not the most tech-savvy person, so it’s very user-friendly,” he said, adding that the tool is saving him “six to eight hours weekly now.”
Draft One was introduced last year by police technology company Axon, which is also known for its Taser electroshock devices. The software uses OpenAI’s GPT large language models to turn body camera audio into full police reports.
Legal and civil liberties experts have raised concerns about the reliability and accountability of such tools. They warn that AI-generated “hallucinations” — fabricated or inaccurate details — could slip into official documents and go unnoticed.
“I am concerned that automation and the ease of the technology would cause police officers to be sort of less careful with their writing,” American University law professor Andrew Ferguson told the Associated Press last year.
Researchers and advocates also caution that generative AI systems can reinforce existing racial and gender biases. Those worries are heightened in policing, where bias and discrimination have been documented long before the introduction of AI tools. Studies have shown that generative models can reproduce and amplify prejudices against women and people of color.
“The fact that the technology is being used by the same company that provides Tasers to the department is alarming enough,” said Aurelius Francisco, cofounder of the Foundation for Liberating Minds in Oklahoma City, in comments to the Associated Press.
Critics argue that AI-written reports could make it harder to hold officers and vendors accountable for errors or misconduct. A recent investigation by the Electronic Frontier Foundation (EFF) into Draft One found that the system offers limited transparency into how reports are created.
According to records obtained by EFF, it is often difficult to determine which sections of a report were generated by AI and which were written or edited by an officer. The organization said the product “seems deliberately designed to avoid audits that could provide any accountability to the public.”
“Axon and its customers claim this technology will revolutionize policing, but it remains to be seen how it will change the criminal justice system, and who this technology benefits most,” the group wrote.
The Heber City Police Department has not yet decided whether it will continue using Draft One. The agency is also piloting a competing AI report-writing system called Code Four, introduced earlier this year.
The incident in Utah highlights both the promise of time-saving automation in law enforcement and the risks of relying on AI systems that can misinterpret context, embed bias, or obscure who is responsible for what is written in official records.