What's happening
This bill prohibits the use of AI and facial recognition outputs as court evidence, while allowing their use for criminal investigations and requiring oversight, transparency, and defendant protections.
Why it matters
Creates stronger checks on police use of AI and facial recognition by mandating transparency, record-keeping, and independent audits. potentially reducing risks of wrongful suspicion or abuse stemming from opaque or biased technologies.
Requires law enforcement to receive specific training on AI and facial recognition limitations and biases. , aiming to prevent over-reliance or misuse and promoting more informed, cautious deployment of these tools in investigations.
Guarantees defendants the right to expert witnesses and state funding to challenge or scrutinize AI and facial recognition systems used in their cases. , helping to level the playing field and address complex technical evidence that could otherwise go unchallenged.
Mandates prosecutorial disclosure of crucial technical details (like error rates, known biases, and, where possible, source code) about AI systems used in investigations. , which could expose flaws, encourage better technology, and support fairer outcomes in criminal justice.
Albany, before your first meeting.
Every bill that moved yesterday, summarized in a sentence and ranked by impact. The Albany Scorecard follows on Sundays.
Free. Unsubscribe in one click. Want alerts on specific bills? Create an account.
Who cares
Police agencies
The bill would allow investigative use of AI and facial recognition while requiring recordkeeping, independent audits and officer training under a standardized protocol.
Prosecutors and civil litigants
The bill would bar AI-generated outputs as evidence or references during questioning in criminal and civil cases, and require prosecutorial offices to disclose details about systems used in investigations.
Defendants and defense attorneys
Defendants would have a right to expert testimony on investigative AI systems, with funding through the Office of Indigent Legal Services for indigent defendants.
Judges
The bill would prohibit AI-generated outputs from forming the basis of any legal decision or judgment.
Headline styles
Limit AI Evidence in Court
Plain English, a few words. Used as the page title.
Sponsors
No co-sponsors yet.
History
- Referred to codes
Full text
2 pagesUnderlined is new[Struck] is removedOfficial text on nysenate.gov ↗
STATE OF NEW YORK
10625
IN ASSEMBLY
June 20, 2024
Introduced by COMMITTEE ON RULES -- (at request of M. of A. Bichotte Hermelyn) -- read once and referred to the Committee on Codes
AN ACT to amend the executive law, in relation to the regulation of the use of artificial intelligence and facial recognition technology in criminal investigations; and to amend the criminal procedure law, in relation to limiting the use of artificial intelligence-generated outputs in court
The People of the State of New York, represented in Senate and Assembly, do enact as follows:
Section 1. Legislative intent. Artificial intelligence tools like Cybercheck and facial recognition technologies have been used extensively in criminal investigations across the U.S., including nearly 8,000 cases in 40 states. However, their use has raised significant legal challenges due to the lack of transparency in their methodologies and concerns over accuracy and reliability. Defense attorneys have successfully argued in several cases that AI-generated evidence should be excluded from court proceedings due to these issues. This act aims to address these concerns by prohibiting the use of AI outputs, including facial recognition, as evidence while allowing their use for investigative purposes, thereby protecting the rights of defendants and maintaining the integrity of the judicial process.
§ 2. Section 837 of the executive law is amended by adding a new subdivision 24 to read as follows:
24. (a) Promulgate a standardized and detailed written protocol for the utilization of artificial intelligence by police agencies in the investigation of criminal activity. The protocol shall address the following topics:
(i) authorizing law enforcement agencies to utilize AI systems and FRT for investigative purposes, including but not limited to data analysis, pattern recognition, facial recognition, and predictive analytics;
(ii) authorizing law enforcement agencies to utilize AI systems and FRT to assist in identifying potential suspects, uncovering evidence, and generating investigative leads;
EXPLANATION--Matter in italics (underscored) is new; matter in brackets
[ ] is old law to be omitted.
LBD15799-01-4
(iii) requiring law enforcement agencies utilizing AI systems and FRT to implement transparency measures, including maintaining records of AI-generated outputs and the parameters used by the AI system. Such records shall be subject to audit and review by an independent oversight body designated by the department to ensure compliance with the protocol;
(iv) requiring law enforcement agencies utilizing AI systems and FRT to undergo regular independent audits of FRT systems to assess their accuracy and biases. The results of these audits shall be publicly accessible and disclosed in any case where FRT evidence is used; and
(v) requiring that law enforcement officers utilizing AI systems and FRT receive thorough training on the limitations and proper use of AI systems and FRT, emphasizing the importance of understanding and mitigating biases and errors.
(b) For the purposes of this subdivision:
(i) "artificial intelligence" or "AI" means a machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments;
(ii) "AI-generated output" means any information, analysis, or decision produced by an AI system during a criminal investigation, including but not limited to facial recognition, predictive policing models, and other machine learning algorithms; and
(iii) "facial recognition technology" or "FRT" means a technology that analyzes facial features and is capable of identifying or verifying a person from a digital image or a video frame from a video source.
§ 3. The criminal procedure law is amended by adding a new section 60.77 to read as follows: § 60.77 Rules of evidence; use of artificial intelligence.
1. Artificial intelligence-generated outputs, including facial recognition results, shall not be admissible as evidence in any criminal or civil court proceeding.
2. No party in a criminal or civil case shall reference or introduce artificial intelligence-generated outputs, including facial recognition results, during questioning of witnesses, suspects, or defendants.
3. Artificial intelligence-generated outputs shall not form the basis for any legal decision or judgment.
4. Defendants shall have the right to expert witnesses who can testify about the reliability and limitations of artificial intelligence and facial recognition technology systems used in the investigation of their cases. Funding shall be available through the office of indigent legal services to indigent defendants to access such experts.
5. All prosecutorial offices must disclose detailed information about any artificial intelligence systems and facial recognition technology used in investigations, including error rates, known biases, source code, and algorithmic transparency, where possible.
§ 4. This act shall take effect on the ninetieth day after it shall have become a law.
Reflowed for reading: line numbers and page breaks removed. Cite the official text.
Summaries, headlines and scores are AI-generated from the official bill text. How we score bills. Source: NYS Open Legislation.