Table of Contents
The Public Safety Reporting System (PSRS) is a prototype web application that parses police blotters from unstructured PDF sources, applies editorial logic to the data to help journalists identify and select relevant incidents, and transmits collections of incidents as reports for publication to a commercial automated writing service, which then delivers them as a draft story into the CMS.
This project was originally built for Brainerd Dispatch in Brainerd, Minnesota. The Associated Press and University of Missouri collaborated to develop this application as part of the Local News AI Initiative, funded by the John S. and James L. Knight Foundation, which aims to leverage AI for the benefit of local news.
The development team thanks the staff at Brainerd Dispatch and Forum Communications for proposing this project, and for their participation, feedback, and encouragement. The development team also recognizes the efforts of commercial automated writing vendor Lede AI which built new capabilities to support this police blotter reporting project.
- Free Up Time: Automating the production of police blotters to reduce the time burden on journalists so they can report on more complex public safety stories.
- Allow for Journalist Review: The automation doesn't replace journalists, but it supports them in producing police blotters faster, so they can edit a final version sooner.
- Cost Effectiveness: Running the system should not incur expenses greater than the savings or revenue gains generated by using it.
PSRS functions through this process:
- Upload: The system takes an input of police blotter PDFs generated by public safety agencies. It is uploaded via the web application (see image above).
- Parsing: A parser extracts individual incidents from the uploaded PDF and inserts them into the MySQL database.
- Incident Selection: The incidents are presented in order of editorial priority, the system awaits a journalist to select incidents for publication (see image below).
- Story Generation: The selected incidents are assembled into a report per public safety agency. This process is provided by the commercial automated writing vendor.
- Publication: The draft story is uploaded to the CMS via API as a draft story for review by a journalist. This process is provided by the commercial automated writing vendor.
Note: In the second step, the parsers are customized to the format of the police blotters generated by three Minnesota public safety agencies. PSRS is built for flexibility in allowing developers to insert any number of new parsers which can be tied to one or many public safety agencies.
Note: While the fourth and fifth steps are tied to a commercial automated writing vendor, PSRS is built to allow developers flexibility to connect to any automated writing service via API.
PSRS was originally designed to be hosted on Amazon Web Services. See administrators for installation details.
The backend tech stack is Python with Flask RestAPIs and MySQL database with ORM tool as SQLAlchemy. See backend for details.
The front end is built with React. See front end for details.
- MySQL
- Postman (to access some maintenance features)
- Lede AI (for automated writing)
- Amazon Web Services (for production hosting)
For day-to-day usage, editorial staff should access PSRS via the UI at http://IP_ADDRESS/loginpage
For system maintenance, such as adding or deleting public safety agencies, adding or deleting users, there are a series of Postman scripts available in the postman-scripts folder.
Note: The Postman scripts need to be modified to adjust for urls, usernames, and passwords.
At the end of the MVP development period, these were the features that we thought would be useful to have in the future:
- Self-service matching feature that allows the user to match detected blotter document fields to database columns. This feature was partially developed, see the notes in backend and front end.
- The database currently collects all incident reports without any clean-up operations. The collected data can be useful for future data journalism tasks. Data management features will need to be considered, such as an export to CSV function, or perhaps an API link to a data visualization tool.
- Management features in the UI to eliminate the need for Postman scripts.
- Outbound connections with other automated writing systems.
- New parsers to handle a variety of popular blotter formats. (The ideal outcome would be an AI-powered parser that can handle all sorts of police blotter formats.)
- Improvements in password handling to add random bytes of salt.
Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the GNU GENERAL PUBLIC LICENSE. See LICENSE for more information.
The Associated Press does not provide technical support for this open-source application.
Prasad Calyam - calyamp@missouri.edu
Project Link: https://github.com/associatedpress/local-ai-brainerd-dispatch
- Hemanth Sai - @hemanth5555 - University of Missouri
- Kiran Neupane - @kngbq - University of Missouri