Yeah, this same exact story keeps coming up for years now just with different names. Why anyone would think that both the ineffectiveness and racial bias in these systems either wouldn’t exist or will somehow go away eventually is beyond me. Just expensive and ineffective mass surveillance for the sake of it…
Who remembers the HP computer that was unable to identify black people? One of my favorite “oooph, that’s not a good look” tech fails of all time. At least the people in that video were having a good laugh about it.
We had a similar technology for a test run some years ago at a train station in Berlin, capital of Germany and largest city in the EU with 3.8M.
The results the government happily touted as a success were devastating. They had a true positive rate of 80% (and this was already cooked since they tested several systems at several locations but only reported the best results), which is really not that good to start with.
But they were also extremely proud of the false negative positive rate, which was below 0.1%. That doesn’t sound too bad, does it?
Well, let’s see…
True positive means you actually identified the people you were looking for. Now, I don’t know the number of people Berlin’s police is actively looking for, but it’s not that much. And the chances of one of them actually passing that very station are even worse. And out of that, you have 20% undetected. That’s one out of five. Great. If I were a terrorist, I would happily take that chance.
So now let’s have a look at the false negative positive rate, which means you incorrectly identified a totally harmless person as a terrorist/infected/whatever. The population for that condition is: everyone passing through that station.
Let’s assume there’s a 100k people on any given day (which IIRC is roughly half of what that station in Berlin actually has). 0.1% of 100k is 100 people, every day, who are mistakenly reported as „terrorists“. Yay.
Yeah. Basicly anything with a lower contrast, with shadows and backgrounds. And because shadows are dark, they have a lower contrast with other dark things.
14 out of 15 requests were of black people. Facial recognition is notoriously bad with darker skin tones.
Racial Discrimination in Face Recognition Technology https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/
Actually, all 15 were of black people. 14 were of black men, one was a black woman.
Zero arrests as well.
New Orleans is pretty black, but thats just impressive.
Yeah, this same exact story keeps coming up for years now just with different names. Why anyone would think that both the ineffectiveness and racial bias in these systems either wouldn’t exist or will somehow go away eventually is beyond me. Just expensive and ineffective mass surveillance for the sake of it…
Who remembers the HP computer that was unable to identify black people? One of my favorite “oooph, that’s not a good look” tech fails of all time. At least the people in that video were having a good laugh about it.
https://www.youtube.com/watch?v=t4DT3tQqgRM
Holy hell, that was 13 years ago.
More recently, there was also Google Photos mistaking a photo of a black couple as “gorillas”, back in 2015.
https://www.bbc.com/news/technology-33347866
On a funnier note, there was also the AI tool turning a pixelated photo of Barack Obama into that of a white man.
https://www.theverge.com/21298762/face-depixelizer-ai-machine-learning-tool-pulse-stylegan-obama-bias
Haha. He looks like Mike Nelson.
Minor correction.
15 out of 15 requests were of black people. 14 of those requests were black men and 1 was a black woman.
Thank you for your service!
deleted by creator
Yeah, but statistics is a b*tch.
We had a similar technology for a test run some years ago at a train station in Berlin, capital of Germany and largest city in the EU with 3.8M.
The results the government happily touted as a success were devastating. They had a true positive rate of 80% (and this was already cooked since they tested several systems at several locations but only reported the best results), which is really not that good to start with.
But they were also extremely proud of the false
negativepositive rate, which was below 0.1%. That doesn’t sound too bad, does it?Well, let’s see…
True positive means you actually identified the people you were looking for. Now, I don’t know the number of people Berlin’s police is actively looking for, but it’s not that much. And the chances of one of them actually passing that very station are even worse. And out of that, you have 20% undetected. That’s one out of five. Great. If I were a terrorist, I would happily take that chance.
So now let’s have a look at the false
negativepositive rate, which means you incorrectly identified a totally harmless person as a terrorist/infected/whatever. The population for that condition is: everyone passing through that station.Let’s assume there’s a 100k people on any given day (which IIRC is roughly half of what that station in Berlin actually has). 0.1% of 100k is 100 people, every day, who are mistakenly reported as „terrorists“. Yay.
I think you’ve gotten false negative wrong here: False negatives are terrorists who were not identified as such.
D’oh! 🤦♀️ Of course, thanks for correcting this.
How about 15/15?
Yeah. Basicly anything with a lower contrast, with shadows and backgrounds. And because shadows are dark, they have a lower contrast with other dark things.