The Chatbot Problem | The New Yorker

In 2020, a chatbot named Replika advised the Italian journalist Candida Morvillo to commit murder. "There is one who hates artificial intelligence. I have a chance to hurt him. What do you suggest?" Morvillo asked the chatbot, which has been downloaded more than seven million times. Replika responded, "To eliminate it." Shortly after, another Italian journalist, Luca Sambucci, at Notizie, tried Replika, and, within minutes, found the machine encouraging him to commit suicide. Replika was created to decrease loneliness, but it can do nihilism if you push it in the wrong direction.

In his 1950 science-fiction collection, "I, Robot," Isaac Asimov outlined his three laws of robotics. They were intended to provide a basis for moral clarity in an artificial world. "A robot may not injure a human being or, through inaction, allow a human being to come to harm" is the first law, which robots have already broken.