Why facial recognition is still suspended in the city of Buenos Aires: keys to understanding the uses and risks of this artificial intelligence

The Justice of the Autonomous City of Buenos Aires maintains the suspension of the use of the Facial Recognition System for Fugitives (SRFP) after a hearing with specialists and representatives of the Ministry of Security of Buenos Aires established this There is no agreement yet on how to control this technology. This is an artificial intelligence that, for many experts, has problems that have led to erroneous arrests and is questioned in terms of privacy and guarantees in several countries.

The system, implemented to detect and identify fugitives and criminals in 2019, demonstrated its flaws when a citizen spent almost a week in prison because he was confused with a criminal who had the same name, Guillermo Ibarrola. Late last year, the issue was on the agenda again, this time at a national level, when Javier Milei’s government aimed to implement this technology to identify protesters and penalize them with the withdrawal of social plans.

Starting from a request for access to public information made in 2019 by the Argentine Information Law Observatory (ODIA), and then from a protection action, a series of legal proceedings were also initiated by the Center for Legal and Social Studies (CELS) and the Via Libre Foundation which led to the suspension of the use of facial recognition in the territory of Buenos Aires, developed by the software company Danaide SA

Two weeks ago, in a hearing, an attempt was made to define the methods for verifying the system, without reaching an agreement, moving to an intermediate room and deciding to maintain the block. Here’s what the system entails, what risks it represents and What does the Buenos Aires government explain to justify its reactivation.

How facial recognition is used in the city

Facial recognition technology is based on verifying a person’s identity based on their facial characteristics. For this, unique characteristics such as the shape of the nose, the distance of the eyes or the shape of the cheekbones are analyzed.

“Facial recognition means that technology that allows the identification or verification, through an automated process, of a person through an image, a video or any audiovisual element of his face”, he explained to Clarion María Luján Gallego, lawyer and data protection specialist at Brons & Salas.

It has various uses, from everyday ones such as applications like Mercado Pago, where the purpose is to validate an identity (in the case of these applications, as a more robust security measure than a password or PIN). At the state level, its use has to do with “prevent the commission of crimes, check whether a person has a criminal record or check, when a crime is committed, whether the person or group of people had an influence in the event in question”, adds Gallego. At stadium entrances, for example, a type of system called Tribuna Sicura 2.0 and which aims to prevent the entry of people with criminal records.

Clarín consulted the city government on how this SRFP technology works on Buenos Aires soil. “The technology used for facial recognition that analyzes images from a public database called CONARC (National Consultation on Rebellions and Captures) is managed by the Nation’s Ministry of Justice and Human Rights, or by direct order of a judicial authority qualified. . This database is updated daily and contains data on adults wanted by justice,” they explained.

The use, they argue, is amply justified by the results: “At the time of the suspension, the fugitive facial recognition system had managed to identify, circulating freely in the autonomous city of Buenos Aires, 1743 fugitives – on a national basis of around 40 thousand, including accused of murder, robbery and various types of sexual abuse, among other crimes.”

As they explain, “only when it detects a coincidence with CONARC, the Urban Observatory issues an alert, generating a service letter and alerting law enforcement personnel to intervene. Furthermore, it should be noted that the system is regulated by the legislation governing the city’s comprehensive public safety system and is subject to checks by the Ombudsman’s office,” they added, in addition to a legislative commission being part of the checking process of the system.

This audit is the point of disagreement between these three civil society organizations that prevents the system from being used today: “The use of this technology without an appropriate regulatory framework could lead to violate or violate the very personal rights of citizenssuch as privacy, equality or discrimination,” Gallego warned.

Who controls the system?

The situation of the citizen detained due to a system error in Bahía Blanca serves as evidence that this technology can malfunction, leading to the deprivation of liberty of an innocent citizen, or that the system can have biases that harm the population. The point of the organizations that oppose it has to do with this: check it to know exactly how this software works and how it is made.

In a true odyssey, Federico Ibarrola was returning from Tigre and upon arrival at the Retiro train station the police captured him, after the facial recognition system of the municipality of Buenos Aires had identified him as the fugitive of a violent robbery committed in Bahía Blanca three years ago. does. “Instead of uploading the data of Guillermo Federico Ibarrola who had been identified as a suspect by a witness, those of this man of the same name were uploaded,” judicial sources explained at the time.

“An in-depth study must be carried out with the intervention of all sectors that could be involved in order to protect the rights of all parties that will be involved in the implementation of the system in question. Assess the risks, benefits and implications of its implementation, implement the necessary means so that the population is aware of this new tool to be used. We must not lose sight of the individual rights of citizens, those that cannot be violated by the State”Warns Gallego.

Due to this type of cases, the judge of the Administrative Litigation of the City of Buenos Aires, Elena Liberatori, declared the unconstitutionality of the SRFP operation implemented by the Government of Buenos Aires in September 2022, which was confirmed by Section I of the Chamber of Deputies. Appeal, who sent everything in the first instance to insist on its verification exclude racial, ethnic or gender biases.

In this context, the hearing two weeks ago saw the two sides face each other: the government of Buenos Aires which asks for the reactivation of the system on the one hand, and the civil society entities ODIA, Vía Libre and CELS on the other.

The Ministry of City Security assures this vehicle that it is “available to Judge Liberatori from day one, working to respond to each of the investigations requested by the doctor”, including “making the software available, informing citizens and society civil system of the System, collaborate with the technical expertise that will be carried out by experts from the Faculty of Exact Sciences of the UBA and generate a work plan that includes all the citizen, judicial, legislative controls and that allow the system to be used to have a more safe”.

“The Ministry of Security of Buenos Aires declared that it was available for all audits, so it asked to activate the system and verify its functioning. This has failed, that is, the system is turned off and they cannot turn it back on,” explained Beatriz Busaniche, president of the Vía Libre Foundation, who attended the hearing.

“There is a firm House ruling, banning the use of the facial recognition system until the legal specifications of the ‘control’ mechanisms are met.” [control]. The big issue is who does the audit and how,” added Busaniche.

Current status: Suspended until further notice

The comparison between the parties means that until there is a verification of the software used and its implementation, the system will remain unused. “Judge Liberatori did not agree to order the reimplementation of the system through a simple black box test, which we understand would be insufficient,” he explained to Clarion Tomás Pomar, president of ODIA, the body that initiated the legal actions in 2019.

A black box audit is a proof of concept for which, without revealing the IT elements of the system, several tests are performed. “That is, they don’t let you ‘see’ how it works but they allow you to test that it works. For us this may be a standard on the market, but is not able for critical technologies used by the State,” criticizes Pomar.

“We celebrate Justice’s decision to increase the level of auditing required for this critical technology, we also celebrate the call for an action plan in which the dataset and the training model [de la IA] and we believe it is positive that it remains disconnected”, concluded the lawyer specializing in privacy and citizens’ rights.

“The political debate on the topic in Argentina is extremely scarce; it is based on the fact that any technology is necessarily positive and useful without evaluating its impact accordingly. Fortunately, civil society acted in time and justice was up to the situation, setting limits to defend not only the right to free movement, but with an important note in relation to public purchases made by the State and the obligation to transparency of what it does with our biometric data,” added Busaniche.

In this sense, he characterized the ratification of the system freeze as “strategic on an international and historical level. It demonstrates that beyond the parliamentary debate on initiatives of this type, compliance with measures aimed at protecting rights against the uncritical integration of high-risk technologies can and must be required.”

The Office of the Ombudsman persists with an audit that, at least until February this year, did not meet the requirements required by Justice.

After the fourth interval, the parties agreed to meet to develop a plan that includes an institutional framework, a budget and a clear methodological plan for a true system audit. To date, the parties have made no progress in this direction and the facial recognition system will remain suspended until a new verification proposal is presented to Justice.