If you haven’t heard yet, consider yourself lucky. There’s a new service called Predictim. I’m not going to link to it because I think it’s abhorrent, but Google’s always there to help.
I don’t even know where to begin or how to spin this post into a pertinent, simple message about ethics in machine learning.
So let’s start with the idea. Their business model is built around giving parents peace of mind. They want to give parents and caregivers the tools to evaluate childminders and (bizarrely enough) dog walkers, to ensure that those they most love are safe from harm.
That sounds reasonable. What isn’t reasonable, however, is how they do it. They scan and analyse the social media posts of your potential service provider and flag their accounts for things like Explicit Content, Drug Abuse, Disrespectful Attitude, and Bullying / Harassment.
I can’t begin to list the ways that these algorithms can go wrong. The number of things that could cause false positives are making my head spin.
The invasion of privacy makes me sick to my stomach. The things that go on the internet are hard to ever remove. Ask any revenge porn victim. This un-erasability is why Snapchat exists. Of course, Snap is losing the fight to almighty social media behemoth Facebook.
Surely people aren’t the sum of all the things they’ve said and done. Especially when those things are on social media.
I wonder how Predictim works when you don’t have any social media history. Is it like credit? You can’t get any until you’ve already built up a record?
Everybody who babysat me was a family member or family friend. Surely, surely, they’re not going to target this product at families like mine and watch communities disintegrate over false positives derived from the bogus analysis of (seemingly) private, or at the very least, personal, information?
In yesterday’s post, I discussed how I don’t do any predictive work that reduces people to mere numbers. And then I found out about this. It’s kind of ironic.
I just want to know a few things.
When did your social media feed become your CV?
When did feeding on people’s most primal fears become the go-to method for attracting venture capital money?
When did we forget that we’re buying services, not people?