Why Facebook Shut Down Its Artificial Intelligence Program That Went Rogue

Facebook does not appear to have “shut down an AI” that invented its own language, for example out of any fear that it would get out of control.

Here is a more realistic reporting of Facebook’s AI experiment: A Facebook AI Unexpectedly Created Its Own Unique Language.

In experimenting with language learning, a research algorithm ended up deviating from human language in a way that wasn’t really useful… it started generating what one might call “functional gibberish”. It was functional in that it continued to carry information, but it wasn’t very efficient or useful.

You could compare it to the weird routes that Google Maps generates sometimes… sure they might save thirty seconds of driving, but they involve ten turns down obscure side streets instead of three turns on main streets.

The result was intriguing because it showed the algorithm’s capacity for generating its own encoding scheme, and also showed what can happen with unconstrained feedback in an automated social language product.

The algorithm wasn’t “shut down” any more than myriad algorithms are shut down every time engineers change the algorithm. Most likely the algorithm was only running on a test dataset and was never "live” or interacting with real humans.

Could this idea, taken to its logical extreme, lead one day to software being “alive” and “conscious”? Maybe. It is certainly an intriguing possibility.