Behind any artificial intelligence lies an algorithm. Sometimes, those algorithms can create unplanned (or planned?) distortions because of
– complexity of the algorithm
– forgotten exceptions
– datasets chosen
– etc.
Here are some real-life examples listed here are really embarassing:
“Users discovered that Google’s photo app, which applies automatic labels to pictures in digital photo albums, was classifying images of black people as gorillas. Google apologized; it was unintentional.”
“But similar errors have emerged in Nikon’s camera software, which misread images of Asian people as blinking, and in Hewlett-Packard’s web camera software, which had difficulty recognizing people with dark skin tones.”
List of example here:
http://mobile.nytimes.com/2016/06/26/opinion/sunday/artificial-intelligences-white-guy-problem.html