TIME Innovation

‘I Don’t Have Ethics’: The Dark Heart of Google’s Latest Chatbot

"I'm in the middle of nowhere"

A new research paper released by Google this week reaffirms what’s been more or less obvious since HAL 9000 murdered four astronauts in Stanley Kubrick’s 1968 film 2001: A Space Odyssey: computers do not make particularly dependable friends.

The Google project—an admittedly more benign exercise than Kubrick’s—was yet another attempt to fashion a “conversational model” of artificial intelligence. Colloquially, we refer to these systems as “chatbots”—programs that can somewhat convincingly engage in human discourse.

The researchers behind the experiment, Oriol Vinyals and Quoc V. Le, programmed their particular chatbot with two sets of data: guidelines from an IT troubleshooting help desk and a database of movie subtitles. The ensuing dialogue was mostly awkward and occasionally nihilistic.

“What is the color of water?” the researchers asked.

“Water,” the computer replied simply.

“Where are you now?” they asked.

“I’m in the middle of nowhere.”

“Tell me the definition of morality,” the researchers requested.

“I don’t have ethics,” the machine advised them.

According to the transcript, this went on for awhile. Philosophical banter proving futile, Vinyals and Le switched gears, with more coherent results.

“What do you think about Cleopatra?” they asked.

“Oh, she’s very regal.”

Read more dialogue here.

These Vintage Computer Ads Show We’ve Come a Long, Long Way

http://pop.bitpig.com/oldads/nov/pdp-11-70.jpg http://www.everyjoe.com/2009/07/30/technology/blast-from-the-past-the-3398-10mb-hard-disk/ http://www.mopo.ca/uploaded_images/honeywell_email-763551.jpg http://www.vintagecomputing.com/wp-content/images/retroscan/ibm5110_large.jpg http://www.macmothership.com/gallery/MiscAds/AdamAd.JPG http://www.vintagecomputing.com/wp-content/images/retroscan/ibm_pc_woman_large.jpg http://www.aresluna.org/attached/computerhistory/ads/international/lotus/pics/byte8311 http://www.vintagecomputing.com/wp-content/images/retroscan/msmouse_large.jpg http://www.vintagecomputing.com/wp-content/images/retroscan/osborne_large.jpg http://pop.bitpig.com/oldads/nov/keeping-up.jpg
Tap to read full story

Your browser is out of date. Please update your browser at http://update.microsoft.com


YOU BROKE TIME.COM!

Dear TIME Reader,

As a regular visitor to TIME.com, we are sure you enjoy all the great journalism created by our editors and reporters. Great journalism has great value, and it costs money to make it. One of the main ways we cover our costs is through advertising.

The use of software that blocks ads limits our ability to provide you with the journalism you enjoy. Consider turning your Ad Blocker off so that we can continue to provide the world class journalism you have become accustomed to.

The TIME Team