Police and security forces around the world are testing out automated facial recognition systems as a way of identifying criminals and terrorists. But how accurate is the technology and how easily could it and the artificial intelligence (AI) it is powered by - become tools of oppression?
Imagine a suspected terrorist setting off on a suicide mission in a densely populated city centre. If he sets off the bomb, hundreds could die or be critically injured.
CCTV scanning faces in the crowd picks him up and automatically compares his features to photos on a database of known terrorists or "persons of interest" to the security services.
The system raises an alarm and rapid deployment anti-terrorist forces are despatched to the scene where they "neutralise" the suspect before he can trigger the explosives. Hundreds of lives are saved. Technology saves the day.
But what if the facial recognition (FR) tech was wrong? It wasn't a terrorist, just someone unlucky enough to look similar. An innocent life would have been summarily snuffed out because we put too much faith in a fallible system.
What if that innocent person had been you?
This is just one of the ethical dilemmas posed by FR and the artificial intelligence underpinning it.
Training machines to "see" - to recognise and differentiate between objects and faces - is notoriously difficult. Computer vision, as it is sometimes called - not so long ago was struggling to tell the difference between a muffin and a chihuahua - a litmus test of this technology.
Computer scientists, Joy Buolamwini of MIT Media Lab (and founder of the Algorithmic Justice League) and Timnit Gebru the technical co-lead of Google's Ethical Artificial Intelligence Team, have shown that facial recognition has greater difficulty differentiating between men and women the darker their skin tone. A woman with dark skin is much more likely to be mistaken for a man.
"About 130 million US adults are already in face recognition databases," Dr Gebru told the AI for Good Summit in Geneva in May. "But the original datasets are mostly white and male, so biased against darker skin types - there are huge error rates by skin type and gender."
The Californian city of San Francisco recently banned the use of FR by transport and law enforcement agencies in an acknowledgement of its imperfections and threats to civil liberties. But other cities in the US, and other countries around the world, are trialling the technology.
In the UK, for example, police forces in South Wales, London. Manchester and Leicester have been testing the tech to the consternation of civil liberties organisations such as Liberty and Big Brother Watch, both concerned by the number of false matches the systems made.
This means innocent people being wrongly identified as potential criminals.
"Bias is something everyone should be worried about," said Dr Gebru. "Predictive policing is a high stakes scenario."
With black Americans making up 37.5% of the US prison population (source: Federal Bureau of Prisons) despite the fact that they make up just 13% of the US population - badly written algorithms fed these datasets might predict that black people are more likely to commit crime.
It doesn't take a genius to work out what implications this might have for policing and social policies.
image credit: Getty Images