This article is from the public account iFeng Technology (ID: ifeng_tech), compiled and organized Xiao Yu, and Ai Faner released it with permission.
Face recognition technology has become standard on high-end phones such as the iPhone, and it has also become a tool to assist police in solving cases. The New York Times published today an article on how the Florida police used the face recognition system to identify suspects. What are its shortcomings?
Here is the full text of the article:
After a high-speed chase in northern Orlando, Florida, the police broke the tire of a stolen Dodge Magnum car and forced it to stop. They arrested the driver but were unable to identify him. The man did not have an identity card and stuffed himself into his mouth before passing out. Police said he appeared to have bitten his fingerprints.
Investigators then had to resort to one of the county’s oldest and largest face recognition systems: a statewide face recognition that began operating in Pinellas County nearly 20 years ago. project. Twenty years ago, law enforcement agencies just started using face recognition technology.
The police searched the man’s photos in a huge database and found a man who might match. This 2017 case has become one of more than 400 successful cases of identifying suspects in the Pinilas County Face Recognition System since 2014.
The New York Times reviewed these Florida face recognition records. It is by far the most comprehensive analysis of the face recognition system of a local law enforcement agency.The potential and limitations of face recognition technology.
The potential and limitations of face recognition
Florida police said they had to query the system 4600 times a month. However, face recognition technology is not a panacea. Documents show that only a small number of queries were able to crack public police investigations of unknown suspects. When the image is clear, face recognition tools can effectively identify uncooperative detainees because suspects use false IDs and photos from anonymous social media accounts. But when investigators try to find suspects that appear briefly in granular surveillance video footage, the face recognition system’s role is greatly diminished.
At the same time, the Florida face recognition project also highlights concerns that new technologies may violate due process. Documentary records show that the system operates almost unsupervised and that its role in legal cases may not necessarily be disclosed to the defendant. Although police say investigators will not rely on facial recognition results to issue arrest warrants, documents show that police sometimes have difficulty collecting other evidence.
“This tool was advertised as being accurate enough to do crazy things when it was peddled,” said Clare Garvie, senior researcher at the Center for Privacy and Technology at Georgetown University Law School. Level. “
In recent years, although face recognition has become a daily tool for unlocking mobile phones and tagging photos on social media, it has also caused controversy. This area has attracted latecomers such as Amazon, and is used by law enforcement agencies in New York, Los Angeles, Chicago, and elsewhere, as well as the FBI and other federal agencies. Amazon also promotes face recognition technology to police. Research data on face recognition systems is scarce, but a 2016 study found that half of American adults have been included in a law enforcement’s face recognition database.
Police argue that face recognition will make the public safer. However, a few cities such as San Francisco have banned law enforcement from using face recognition tools because of concerns about user privacy leaks and face recognition technology making mismatches. Advocates for civil liberties warn that face recognition technology could be used maliciously.
Facial recognition has long been a part of daily law and order in Florida. Twenty years ago, the Pinilas County Sheriff’s Office pushed for public opinion to use federal funds to try face recognition technology. Now it’s actually become a face recognition service in Florida, with access to more than 30 million images, including driver’s licenses, photos of suspects, and photos of teen appointments.
“People think of it as something new,” said Pinellas County Sheriff Bob Gualtieri, referring to face recognition technology. “But the technology that everyone uses now is a long time coming. I used it before. “
Can face recognition work alone?
So far, only one U.S. court has ruled on the use of face recognition by law enforcement, which is increasing belief that the defendant’s right to information is limited.
In 2015, Willie Allen Lynch was accused of selling $ 50 million of Perak cocaine because Pinilas County’s face recognition system suggested he was a possible suspect. Lynch argued that he was mistaken for the suspect and sought to obtain other images that might match the suspect. The Florida Court of Appeal rejected his request and sentenced him to 8 years in prison.
Any technical findings presented as evidence should be analyzed through a special hearing, but face recognition results have never been considered reliable enough to withstand such challenges. However, until more reliable forensic technology has not been subject to judicial review, the face recognition results can still play a large role in the investigation.
▲ Facial Recognition System of Sheriff’s Office of Pinilas County
U.S. laws and court rules vary on which investigative materials must be shared with the defendant. This has led some law enforcement officials to argue that they are not required to disclose the use of face recognition.
In some Florida cases, face recognition technology was not mentioned in the initial arrest warrant or written court statement. Instead, detectives pointed out “investigation methods” or “confirmation attempts” in court documents, and listed these cases as success in face recognition applications in Pinellas County records. Defense lawyers said in an interview that the use of face recognition is sometimes mentioned at the end of the discovery process, but not necessarily.
Aimee Wyant is a senior assistant public defense attorney in the Judicial Circuit in Pinillas County. She said that defense lawyers should be aware of all the information used in the investigation. “Once the police find the suspect, they will be like dogs holding their bones. This is their suspect,” Wynte said. “So, we must figure out how they found the suspect?”>
Florida and other law enforcement officials have emphasized that face recognition technology should not be relied upon to make arrests. “Computer matching alone cannot arrest anyone,” New York Police Chief James O’Neill said in June last year.
In most Florida cases, investigators have followed similar guidelines. However, court records show that in a few cases, face recognition was the main basis for police arrests. For example, in April last year, a Tallahassee police officer investigating the theft of a $ 80 mobile phone obtained a surveillance image of a store and found possible suspects from the facial recognition system. She wrote in court documents that she watched the surveillance video and confirmed the identity of the suspect.
A police department spokesperson hinted that the suspect’s identification had been verified by the face recognition system. “We won’t just say‘ it ’s him, ’we wo n’t even investigate,” she said. “This is a very clear picture.” The case is under trial.
The “calling game” becomes history
The launch of the Face Analysis Comparison and Inspection System (FACES) in Pinellas County began with a $ 3.5 million federal grant arranged by Rep. Bill Young in 2000. Bill Young is a Florida Republican who leads the House Appropriations Committee.
Early test results from other states ’law enforcement agencies were not satisfactory, such as California ’s arrest of only one person in four years. However, the potential of face recognition technology remains attractive. The first use of Pinellas County’s face recognition technology is to use it in the current prison photo system. After 9/11, the FACES project was expanded to the airport. Eventually, the Pinilas County police were able to upload photos taken with a digital camera to the system while on patrol.
The Pinilas County Sheriff’s Office said that before the county began bearing annual maintenance costs in 2014, the face recognition project received more than $ 15 million in federal funding and is now approximately $ 100,000 per year.
The first arrest in Florida under a face recognition system occurred in 2004. According to local media reports at the time, a woman who had been chased for breach of probation provided a false name to the police. As the use of FACES spreads across the state, the driver’s license system is integrated into the image library, and arrests are increasing. By 2009, the Pinilas County Sheriff’s Office had attributed nearly 500 arrests to facial recognition systems. By 2013, that number was close to 1,000. Only a few cases have been made public.
The latest data shows that Pinellas County’s face recognition system has successfullySuspects have been identified in more than 00 cases. However, this data is flawed: not all successful identification cases have been recorded, and doubtful or negative results have not been recorded. However, coupled with relevant court documents, about half of the case records are easily accessible. The latest face recognition case list also analyzes which crimes face recognition is best suited to assist with: shoplifting, forged checks, and identity fraud.
In this series of cases, police are seeking identity checks. “We call it a ‘calling game’,” said Sheriff Valley Austria. “We would stop someone on the street, and they would say, ‘My name is John Doe.’ No proof of identity. “
Face recognition has played an important role in approximately 36 court cases, despite blurry images. Nearly 20 of these cases involved petty theft and others were more serious.
▲ Police use face recognition to identify suspects in robbery
Documentation in Pinilas County shows that investigators used face recognition technology to lock the suspect after a gun robbery at an ATM near Hillsborough County in 2017. They showed the surveillance video of the ATM to the girlfriend of the suspect, who confirmed the identity of the suspect. The suspect subsequently pleaded guilty.
In nearly 20 cases of face recognition in Pinilas County, investigators are trying to identify people who cannot know who they are, such as patients with Alzheimer’s disease and murder victims. The Pinilas County Sheriff’s Office says facial recognition technology is sometimes used to assist in identifying witnesses.
Some of the cutting-edge applications of face recognition technology in the field of crime detection have not shown positive results and have been abandoned, such as the use in airports.
Image quality is key
“It depends on the image quality,” said Jake Ruberto, a technical support specialist who assists in operating the face recognition system with the Pinilas County Sheriff’s Office. Information, but also useless results. “
The FACES system was developed by the French company Idemia, whose prototype algorithm is also in the United States.The Institute of Standards and Technology has performed well in recent tests. However, the FACES system used by law enforcement does not necessarily contain the latest algorithms. For example, the last major update to the FACES system used in Pinellas County was in 2014, but the county has been evaluating other updated products. Idemia declined to comment.
In recent years, the upgrade of excellent face recognition technology has been shocking. In government tests, the face recognition algorithm can be compared with a database of 1.6 million facial photos. In 2010, the error rate of the face recognition system under ideal conditions was slightly less than 8%. The ideal conditions here are well-lit, high-resolution, front-face photos. By 2018, this error rate had dropped to 0.3%. But under surveillance video conditions, law enforcement cannot count on face recognition to achieve such high reliability.
Perhaps, the biggest controversy in face recognition technology is its unstable performance in identifying different ethnic groups. Test data released by the National Institute of Standards and Technology last December showed that the types of face recognition systems used in government surveys are more error-prone when evaluating images of black women. Florida law enforcement officials said the technology’s performance did not indicate a degree of racial bias.
Pinilas County and other officials also emphasized the role of manual review. However, tests using passport images found that it was also difficult for human auditors to identify the correct person in similar face recognition results. During these tests, passport system employees often make the wrong choice.
Poor quality images are causing matching errors. Dim lighting conditions, face twisting to a certain angle, and mild camouflage using a baseball cap or sunglasses can affect the accuracy of face recognition.
In some countries with stricter civil liberties laws, the shortcomings of face recognition technology are exposed, especially when used to find criminals in the crowd. London, England has a huge network of CCTV cameras, but a face recognition study found that the system completed 42 matches during the test, but only 8 of them proved to be accurate.
Former and current officials of Pinilas County said they were not surprised by the result. “If you want to investigate bank robberies and convenience store robberies, face recognition technology doesn’t work,” said Jim Main, who had been responsible for the face recognition project technology at the Pinilas County Sheriff’s Office In part, retiring in 2014, “you can’t say ‘stop, let me find your picture’.”
So far, only one U.S. court has ruled on the use of face recognition by law enforcement, which is increasing belief that the defendant’s right to information is limited.
In 2015, Willie Allen Lynch was accused of selling $ 50 million of Perak cocaine because Pinilas County’s face recognition system suggested he was a possible suspect. Lynch argued that he was mistaken for the suspect and sought to obtain other images that might match the suspect. The Florida Court of Appeal rejected his request and sentenced him to 8 years in prison.
Any technical findings presented as evidence should be analyzed through a special hearing, but face recognition results have never been considered reliable enough to withstand such challenges. However, until more reliable forensic technology has not been subject to judicial review, the face recognition results can still play a large role in the investigation.
▲ Facial Recognition System of Sheriff’s Office of Pinilas County
U.S. laws and court rules vary on which investigative materials must be shared with the defendant. This has led some law enforcement officials to argue that they are not required to disclose the use of face recognition.
In some Florida cases, face recognition technology was not mentioned in the initial arrest warrant or written court statement. Instead, detectives pointed out “investigation methods” or “confirmation attempts” in court documents, and listed these cases as success in face recognition applications in Pinellas County records. Defense lawyers said in an interview that the use of face recognition is sometimes mentioned at the end of the discovery process, but not necessarily.
Aimee Wyant is a senior assistant public defense attorney in the Judicial Circuit in Pinillas County. She said that defense lawyers should be aware of all the information used in the investigation. “Once the police find the suspect, they will be like dogs holding their bones. This is their suspect,” Wynte said. “So, we must figure out how they found the suspect?”>
Florida and other law enforcement officials have emphasized that face recognition technology should not be relied upon to make arrests. “Computer matching alone cannot arrest anyone,” New York Police Chief James O’Neill said in June last year.
In most Florida cases, investigators have followed similar guidelines. However, court records show that in a few cases, face recognition was the main basis for police arrests. For example, in April last year, a Tallahassee police officer investigating the theft of a $ 80 mobile phone obtained a surveillance image of a store and found possible suspects from the facial recognition system. She wrote in court documents that she watched the surveillance video and confirmed the identity of the suspect.
A police department spokesperson hinted that the suspect’s identification had been verified by the face recognition system. “We won’t just say‘ it ’s him, ’we wo n’t even investigate,” she said. “This is a very clear picture.” The case is under trial.
The “calling game” becomes history
The launch of the Face Analysis Comparison and Inspection System (FACES) in Pinellas County began with a $ 3.5 million federal grant arranged by Rep. Bill Young in 2000. Bill Young is a Florida Republican who leads the House Appropriations Committee.
Early test results from other states ’law enforcement agencies were not satisfactory, such as California ’s arrest of only one person in four years. However, the potential of face recognition technology remains attractive. The first use of Pinellas County’s face recognition technology is to use it in the current prison photo system. After 9/11, the FACES project was expanded to the airport. Eventually, the Pinilas County police were able to upload photos taken with a digital camera to the system while on patrol.
The Pinilas County Sheriff’s Office said that before the county began bearing annual maintenance costs in 2014, the face recognition project received more than $ 15 million in federal funding and is now approximately $ 100,000 per year.
The first arrest in Florida under a face recognition system occurred in 2004. According to local media reports at the time, a woman who had been chased for breach of probation provided a false name to the police. As the use of FACES spreads across the state, the driver’s license system is integrated into the image library, and arrests are increasing. By 2009, the Pinilas County Sheriff’s Office had attributed nearly 500 arrests to facial recognition systems. By 2013, that number was close to 1,000. Only a few cases have been made public.
The latest data shows that Pinellas County’s face recognition system has successfullySuspects have been identified in more than 00 cases. However, this data is flawed: not all successful identification cases have been recorded, and doubtful or negative results have not been recorded. However, coupled with relevant court documents, about half of the case records are easily accessible. The latest face recognition case list also analyzes which crimes face recognition is best suited to assist with: shoplifting, forged checks, and identity fraud.
In this series of cases, police are seeking identity checks. “We call it a ‘calling game’,” said Sheriff Valley Austria. “We would stop someone on the street, and they would say, ‘My name is John Doe.’ No proof of identity. “
Face recognition has played an important role in approximately 36 court cases, despite blurry images. Nearly 20 of these cases involved petty theft and others were more serious.
▲ Police use face recognition to identify suspects in robbery
Documentation in Pinilas County shows that investigators used face recognition technology to lock the suspect after a gun robbery at an ATM near Hillsborough County in 2017. They showed the surveillance video of the ATM to the girlfriend of the suspect, who confirmed the identity of the suspect. The suspect subsequently pleaded guilty.
In nearly 20 cases of face recognition in Pinilas County, investigators are trying to identify people who cannot know who they are, such as patients with Alzheimer’s disease and murder victims. The Pinilas County Sheriff’s Office says facial recognition technology is sometimes used to assist in identifying witnesses.
Some of the cutting-edge applications of face recognition technology in the field of crime detection have not shown positive results and have been abandoned, such as the use in airports.
Image quality is key
“It depends on the image quality,” said Jake Ruberto, a technical support specialist who assists in operating the face recognition system with the Pinilas County Sheriff’s Office. Information, but also useless results. “
The FACES system was developed by the French company Idemia, whose prototype algorithm is also in the United States.The Institute of Standards and Technology has performed well in recent tests. However, the FACES system used by law enforcement does not necessarily contain the latest algorithms. For example, the last major update to the FACES system used in Pinellas County was in 2014, but the county has been evaluating other updated products. Idemia declined to comment.
In recent years, the upgrade of excellent face recognition technology has been shocking. In government tests, the face recognition algorithm can be compared with a database of 1.6 million facial photos. In 2010, the error rate of the face recognition system under ideal conditions was slightly less than 8%. The ideal conditions here are well-lit, high-resolution, front-face photos. By 2018, this error rate had dropped to 0.3%. But under surveillance video conditions, law enforcement cannot count on face recognition to achieve such high reliability.
Perhaps, the biggest controversy in face recognition technology is its unstable performance in identifying different ethnic groups. Test data released by the National Institute of Standards and Technology last December showed that the types of face recognition systems used in government surveys are more error-prone when evaluating images of black women. Florida law enforcement officials said the technology’s performance did not indicate a degree of racial bias.
Pinilas County and other officials also emphasized the role of manual review. However, tests using passport images found that it was also difficult for human auditors to identify the correct person in similar face recognition results. During these tests, passport system employees often make the wrong choice.
Poor quality images are causing matching errors. Dim lighting conditions, face twisting to a certain angle, and mild camouflage using a baseball cap or sunglasses can affect the accuracy of face recognition.
In some countries with stricter civil liberties laws, the shortcomings of face recognition technology are exposed, especially when used to find criminals in the crowd. London, England has a huge network of CCTV cameras, but a face recognition study found that the system completed 42 matches during the test, but only 8 of them proved to be accurate.
Former and current officials of Pinilas County said they were not surprised by the result. “If you want to investigate bank robberies and convenience store robberies, face recognition technology doesn’t work,” said Jim Main, who had been responsible for the face recognition project technology at the Pinilas County Sheriff’s Office In part, retiring in 2014, “you can’t say ‘stop, let me find your picture’.”
“It depends on the image quality,” said Jake Ruberto, a technical support specialist who assists in operating the face recognition system with the Pinilas County Sheriff’s Office. Information, but also useless results. “
The FACES system was developed by the French company Idemia, whose prototype algorithm is also in the United States.The Institute of Standards and Technology has performed well in recent tests. However, the FACES system used by law enforcement does not necessarily contain the latest algorithms. For example, the last major update to the FACES system used in Pinellas County was in 2014, but the county has been evaluating other updated products. Idemia declined to comment.
In recent years, the upgrade of excellent face recognition technology has been shocking. In government tests, the face recognition algorithm can be compared with a database of 1.6 million facial photos. In 2010, the error rate of the face recognition system under ideal conditions was slightly less than 8%. The ideal conditions here are well-lit, high-resolution, front-face photos. By 2018, this error rate had dropped to 0.3%. But under surveillance video conditions, law enforcement cannot count on face recognition to achieve such high reliability.
Perhaps, the biggest controversy in face recognition technology is its unstable performance in identifying different ethnic groups. Test data released by the National Institute of Standards and Technology last December showed that the types of face recognition systems used in government surveys are more error-prone when evaluating images of black women. Florida law enforcement officials said the technology’s performance did not indicate a degree of racial bias.
Pinilas County and other officials also emphasized the role of manual review. However, tests using passport images found that it was also difficult for human auditors to identify the correct person in similar face recognition results. During these tests, passport system employees often make the wrong choice.
Poor quality images are causing matching errors. Dim lighting conditions, face twisting to a certain angle, and mild camouflage using a baseball cap or sunglasses can affect the accuracy of face recognition.
In some countries with stricter civil liberties laws, the shortcomings of face recognition technology are exposed, especially when used to find criminals in the crowd. London, England has a huge network of CCTV cameras, but a face recognition study found that the system completed 42 matches during the test, but only 8 of them proved to be accurate.
Former and current officials of Pinilas County said they were not surprised by the result. “If you want to investigate bank robberies and convenience store robberies, face recognition technology doesn’t work,” said Jim Main, who had been responsible for the face recognition project technology at the Pinilas County Sheriff’s Office In part, retiring in 2014, “you can’t say ‘stop, let me find your picture’.”