Geoffrey Everest Hinton (born 6 December 1947) is a British-Canadian cognitive psychologist and computer scientist, most noted for his work on artificial neural networks. Since 2013, he has divided his time working for Google (Google Brain) and the University of Toronto.
Hinton received the 2018 Turing Award, together with Yoshua Bengio and Yann LeCun, for their work on deep learning. They are sometimes referred to as the “Godfathers of Artificial Intelligence”.
In an interview that was published on Youtube on March 25, 2023 Mr Hinton talks about the impact and potential of AI. Among other things he explains the difference between two different routes to AI: The biological and the digital route.
The digital route involves computers that run an AI program. The design of the program is based on the anatomy of the brain in which neurons (nodes) are connected to each other by nerve fibers. The digital neural network can be trained for a specific task, for example to recognise a cat, by feeding it many pictures of different animals. Each picture is labelled (by the human programmer) with the name of the animal species. After seeing many pictures of animals, the image of a cat gets encoded in the connection strength between the nodes in the artificial neural network. The AI can be said to have built a model of a cat. When shown a new picture of an animal, the trained AI program is now able to tell you with what probability the animal in the picture is a cat.
The same AI program can run on a different computer and can be trained in the same way to recognise a different animal, for example a dog. When both trained AI’s are digitally connected they share the connection strengths between the different nodes in their network: They are able to share their ‘knowledge’ about what cats and dogs look like. Both AI’s are now able to recognize both a cat and a dog.
This efficient sharing of knowledge between AI programs comes at a cost. It only works when both AI programs are the same, use the same algorithms and when both computer systems operate exactly in the same way. To accomplish this requires a lot of energy: A typical AI with many copies of the program running on different computers requires about 1 megawatt of power
If these systems were to run on less power, “noise” will be introduced. Mr Hinton does not elaborate on it but I imagine “noise” as missing, deformed or just wrong pieces in the model that the AI has made of a cat or dog. The AI will become less accurate.
According to Mr Hinton every system will adapt to its specific “noise”: It will try to fill in the missing pieces and try to correct or ignore the wrong pieces while developing its model of a cat. Because each system will try to compensate for its “noise” in its own way, it becomes impossible to share knowledge between two AI programs by just connecting the two systems. It is as if they are no longer able to communicate with perfect understanding.
The brain on the other hand runs on 30 watts. So there must be a lot of “system-specific” noise. Because each brain deals with this noise in its own way, we are not able to make two brains behave in exactly the same way. Therefore knowledge can not be shared by just connecting two brains. To solve this problem, humans use language as a tool to communicate, to share knowledge.
Mr Hinton made me think. My personality is defined by how I react to and interact with other people and – in general – how I interact with the outside world. My personality is what makes me different from all those other personalities out there. Mr Hinton’s explanation does not change anything about all that. But it is somewhat sobering to realize that what I call my – very special – personality is actually not much more than my very special way of coping with the disturbances that creep into how my low powered brain perceives and interprets the world around me.
And it does make me wonder how disturbed you all are.
We do need to talk.
References:
Link
Interview: https://www.youtube.com/watch?v=qpoRO378qRY
Wikipedia: https://en.wikipedia.org/wiki/Geoffrey_Hinton
