{"id":3148,"date":"2022-09-07T15:42:00","date_gmt":"2022-09-07T19:42:00","guid":{"rendered":"https:\/\/www.davideriknelson.com\/sbsb\/?p=3148"},"modified":"2022-09-06T20:44:13","modified_gmt":"2022-09-07T00:44:13","slug":"an-unconsciously-biased-mind-will-produce-an-unconsciously-biased-machine","status":"publish","type":"post","link":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/2022\/09\/an-unconsciously-biased-mind-will-produce-an-unconsciously-biased-machine\/","title":{"rendered":"An Unconsciously Biased Mind Will Produce an Unconsciously Biased Machine🤖"},"content":{"rendered":"\n

Yet another example of how absurdly easy it is to manipulate artificial intelligence\u2014or even just accidentally make them into terrible bigots<\/a> (and slightly above average antisemites).\u00a0<\/p>\n\n\n\n

From University of Cambridge\u2019s Ross Anderson (via security guru Bruce Schneier, \u201cManipulating Machine-Learning Systems through the Order of the Training Data<\/a>\u201d):<\/p>\n\n\n\n

Most deep neural networks are trained by stochastic gradient descent. Now \u201cstochastic\u201d is a fancy Greek word for \u201crandom\u201d; it means that the training data are fed into the model in random order.<\/em><\/p>

So what happens if the bad guys can cause the order to be not random? You guessed it \u2013\u00a0<\/em>all bets are off<\/em><\/a>. Suppose for example a company or a country wanted to have a credit-scoring system that\u2019s secretly sexist, but still be able to pretend that its training was actually fair. Well, they could assemble a set of financial data that was representative of the whole population, but start the model\u2019s training on ten rich men and ten poor women drawn from that set \u2013 then let initialisation bias do the rest of the work.<\/em><\/p><\/blockquote>\n\n\n\n

Anderson concludes \u201cIt\u2019s time for the machine-learning community to carefully examine their assumptions about randomness.\u201d<\/strong><\/em><\/p>\n\n\n\n

I think that\u2019s tangent to the real lesson, which is this:<\/p>\n\n\n\n

All machines (including AIs) are created things, and created things bear the biases of their creators in unexpected, but ironclad ways: early color film was shite at photographing people of color simply because the folks who created the color film were all White and unintentionally selected techniques and chemical processes that worked better for their own paler skin tones than they did for darker ones<\/a>. Similarly, male engineers built crash-test dummies that were roughly their own size and weight<\/a>\u2014and thus created \u201csafety\u201d features that killed women and children<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Yet another example of how absurdly easy it is to manipulate artificial intelligence\u2014or even just accidentally make them into terrible bigots (and slightly above average antisemites).\u00a0 From University of Cambridge\u2019s Ross Anderson (via security guru Bruce Schneier, \u201cManipulating Machine-Learning Systems through the Order of the Training Data\u201d): Most deep neural networks are trained by stochastic … Continue reading “An Unconsciously Biased Mind Will Produce an Unconsciously Biased Machine🤖”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":3153,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_share_on_mastodon":"1"},"categories":[11],"tags":[804,572,53,180,41],"share_on_mastodon":{"url":"","error":""},"_links":{"self":[{"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/posts\/3148"}],"collection":[{"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/comments?post=3148"}],"version-history":[{"count":7,"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/posts\/3148\/revisions"}],"predecessor-version":[{"id":3156,"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/posts\/3148\/revisions\/3156"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/media\/3153"}],"wp:attachment":[{"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/media?parent=3148"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/categories?post=3148"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.davideriknelson.com\/sbsb\/index.php\/wp-json\/wp\/v2\/tags?post=3148"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}