Google's Artificial Brain Learns to Find Cat Videos
wired.comGoogle’s X Lab designed a particular neural network to discern whether an algorithm could learn to recognize faces without being fed information on exactly what a face was. It did great at that task.
It also found cat videos quite well:
The “brain” simulation was exposed to 10 million randomly selected YouTube video thumbnails over the course of three days and, after being presented with a list of 20,000 different items, it began to recognize pictures of cats using a “deep learning” algorithm. This was despite being fed no information on distinguishing features that might help identify one.
On the surface, it’s a joke about the impending cat video singularity. But it’s a pretty awesome example of machine learning, too. It means that with careful programming, a synthetic network can learn fairly advanced patterns in completely novel data, like cat faces, or what a cat even IS.
“A feedback loop involves four distinct stages. First comes the data: A behavior must be measured, captured, and stored. This is the evidence stage. Second, the information must be relayed to the individual, not in the raw-data form in which it was captured but in a context that makes it emotionally resonant. This is the relevance stage. But even compelling information is useless if we don’t know what to make of it, so we need a third stage: consequence. The information must illuminate one or more paths ahead. And finally, the fourth stage: action. There must be a clear moment when the individual can recalibrate a behavior, make a choice, and act. Then that action is measured, and the feedback loop can run once more, every action stimulating new behaviors that inch us closer to our goals.”
—Harnessing the Power of Feedback Loops | Magazine
This article explains my work and the universe too I guess. A great read or listen via the Mp3 audio.
Google Working on Simulating the Human Brain - semanticweb.com
semanticweb.com(N.B. IBM Research is also well along on various aspects of AI, such as Watson, and a new generation of cognitive computing chips and other frontiers of neuroscience, supercomputing and nanotechnology)

John Markoff of the New York Times reports, “Inside Google’s secretive X laboratory, known for inventing self-driving cars and augmented reality glasses, a small group of researchers began working several years ago on a simulation of the human brain. There Google scientists created one of the largest neural networks for machine learning by connecting 16,000 computer processors, which they turned loose on the Internet to learn on its own. Presented with 10 million digital images found in YouTube videos, what did Google’s brain do? What millions of humans do with YouTube: looked for cats.”
He continues, “The neural network taught itself to recognize cats, which is actually no frivolous activity. This week the researchers will present the results of their work at a conference in Edinburgh, Scotland. The Google scientists and programmers will note that while it is hardly news that the Internet is full of cat videos, the simulation nevertheless surprised them. It performed far better than any previous effort by roughly doubling its accuracy in recognizing objects in a challenging list of 20,000 distinct items. The research is representative of a new generation of computer science that is exploiting the falling cost of computing and the availability of huge clusters of computers in giant data centers. It is leading to significant advances in areas as diverse as machine vision and perception, speech recognition and language translation.”
Google Puts Its Virtual Brain Technology to Work - Technology Review
technologyreview.comThis summer Google set a new landmark in the field of artificial intelligence with software that learned how to recognize cats, people, and other things simply by watching YouTube videos (see “Self-Taught Software”). That technology, modeled on how brain cells operate, is now being put to work making Google’s products smarter, with speech recognition being the first service to benefit.
Google’s learning software is based on simulating groups of connected brain cells that communicate and influence one another. When such a neural network, as it’s called, is exposed to data, the relationships between different neurons can change. That causes the network to develop the ability to react in certain ways to incoming data of a particular kind—and the network is said to have learned something.
Neural networks have been used for decades in areas where machine learning is applied, such as chess-playing software or face detection. Google’s engineers have found ways to put more computing power behind the approach than was previously possible, creating neural networks that can learn without human assistance and are robust enough to be used commercially, not just as research demonstrations.
Javascript and the brain: Why Javascript is the future of programming languages
I discover myself all the time comparing programming languages to brain functions.
While I have never designed a programming language myself, I have used many of them, and think, that I know their strengths and weaknesses quite well by now.
But more importantly, I have studied brain functions and neural sciences to some extend over the years (especially starting with reading von Neumann’s and Turings books)
It is not a case that von Neumann, was also one of the founders of the computer paradigm. After all he just invented computers after the human prototype he was studying.
Over the decades however programming languages diverged from his principles and I always found myself reaching a dead end after some while with modern programming languages like Ruby, Java, .Net. etc. Strangely at first contact with Javascript over 15 years ago, I suddenly had this “feels right” feeling, but couldn’t describe why. Today, after some more years and some more experience with computer languages, I finally understand why Javascript not only feels right to me, but objectively is the right language for building future applications.
Let me explain why:
Preamble: I must explain some basics of how basic brain functionality in order to explain my conclusions:
- The brain is a neural network
- Neural networks are connections of neurons
- Neurons are cells
So what’s the brain for?
The brain is a world simulator. It’s a big approximator. It takes input from it’s senors (the eyes, etc.) and produces output (which in someway should reproduce what’s happening or will happen in the real world)
How does the brain do that? It does it by a process called “learning”, which was studied quite extensively by von Neumann, reaching also very surprising conclusions about the learning process itsself. All that neural networks do, is they modify their connection weightings all time, until the output produced by the network is coherent to what the sensor input says. It’s some sort of “Calibration” that takes place during learning. When you learn things right, chances are you will make the right conclusions, before things happen, and you probably will survive. Yes, you see that car rushing at you with 100 km/h. Well your brain should be able to precalculate, that if it continous on its trail like that it will just hit you in the next minute. Therefore it’s better to jump in the bush before. Your brain might even be able to precalculate, that the driver might be drunk or that he will see you in the next 20 seconds and therefore tell you to wait a moment to not make too fast or early conclusions. Your brain is fantastic!
Ok. Probably you knew this already.
What does this have to do with programming languages?
Well. If you think about the above, there is a powerful conclusion you can make, which is:
The brain is a (sophisticated) computer! It’s a big processor. But most importantly and this is what computer languages forgot over the years. It’s a parallel processor! That’s what most programming languages are not. They just took some of von Neumann’s paradigms, but forgot about the most important ones. Parallelism and self modification. Aehm, self modification? Yes, the brain modifies it’s functions by learning. Neural networks function, in that electronic impulses coming from the human sensors, run through it parallely. It’s evented non-blocking IO. Yes! I hope now you understand the hype about node.js. Node is an evented non-blocking IO framework written in Javascript.
Ok so let’s see why Javascript fits in perfectly in imitating the brain.
a) No types: types are stupid. The brain has no notion of a type. A neuron is a neuron. A type is an artifical creation of computer scientists, in order to distinguish, what an object can do. It turns out, that object are actually all the same. They are objects in OO programming languages. The type comes over their metadata and the objects they are connected with in memory. In fact one of the big “revolution” in OOP languages was to move away from basic types like int, float, etc. to a generic object oriented model. However still some languages use types to distinguish such basic objects. It’s not necessary. The compiler can find out by itself over heuristics. That’s what Javascripts tracing JIT does! And as we see in the last years with Tracemonkey, Spidermonkey and Google V8 or Crankshaft it starts doing it really well and fast!
b) Dynamic: Javascript is a dynamic language. This means that code can modify code during runtime. This was something like the devil for programming language designers over the last decades. They thought it was evil to do that and would lead to code, that wasn’t clear anymore and so they tried to prevent such use in their languages. Well it turns out that they were wrong. Self modifying code will be the future, and while we are not there yet, I think it will be the 4th and last revolution in software design in this century.
c) Functions in Javascript are first class citizens: Well to explain this I have again to make reference to brain functions and “brain functions”.
What are equivalent to functions in the brain? Well the brain has no distinction between functions and memory. Everything is taking up memory in the neural network. Functions and data. But Functions are like a set of neurons, with a specific scope. That’s data (signals) runs through them to produce some output data (other signals). In fact the way this works in Javascript is completely the same. Functions are just again objects in memory and thus can group functionality. Not only! They will also map to process memory perfectly.
The objects that run through such functions are electric impulse codificated objects coming directly from our sensors.
In computers sensors quantize these information to 0-1 bit streams.
So a binary diff function somewhat operates however on the same memory and therefore is a first class citizen. That’s why javascript is the future of languages, cause it treats functions as first class citizens, whereas other languages don’t.
d) Javascript scope and closures: Javascripts scope keeps your working environment and variable in place, also after you come back from some asynchronous function. This is important and is like the brain is connected to neurons it uses more often in a more “near” or “local scope” fashion.
e) IPC communication: well todo communication we need to marshal data in a protocol. The way this is done nowadays is mainly http. And parsing text on both sides. Javascript is a perfect fit for this with it’s human readable JSON object notation.
f) Push and not pull, and Javascripts event model: The brain works by emitting electrical signals to connected neurons. The brain doesn’t “call” functions it always “emits” events. In computer science it is well known by now that passing objects to functions leads to tight coupling, which makes code less maintainable over time. Javascript although it is not perfect has a quite simple notation for emitting events. Whilst other languages can do that too over patterns like inheritance, delegates and interfaces or notification centers. Javascript’s implementation is more powerful cause it uses simple function pointers, which are more leightweight, than passing around objects.
g) Inheritance, Annotations and Mixins: The brain has no notion of inhertitance. It’s an interconnected graph so there is no hierarchy. While other languages try to create object hierarchies through inheritance, Javascript doesn’t. Instead you can modify or extend every existing object or just “connect” to it over a pointer. That’s why Javascript immitates better the connection between neurons, than any other language I know.
This article was really fun writing. I think creating the brain analogy was a good idea. Comparing programming language features to brain functions, somewhat works quite well, so this means that there is a strong correlation between the two.
Have another brain analogy. Post me a line.
Brain reasoning, bayesian networks, abstraction and Markov chains
Surely just reinventing the wheel here, but human brain looks like a sophisticated bayesian Markov chain machine with abstraction capabilities.
It is a Markov chain machine in the sense that our brain, through learning and experience, builds and updates over time a large matrix of conditional probabilities. By counting the number of instances of concurrent events A, B, C, …, I, X in real life, the brain constantly updates the probability of outcome X given the occurrence of A, B, C, …, I. Or in mathematical terms, P(X|A,B,C,…I).
Abstraction capabilities allow then the brain to build upon fundamental conditional probabilities P1, P2, P3, Px based on direct experiences, to create a second, third and iteratively n-th layer of more complex probabilities, for example P(Px|P1,P2,P3,…).
This wealth of information could be encode in chromosome-like strings to be passed on and further processed.
Mon 5:55 Spontaneous Creativity
Uprooted from these desolate mental trenches, we flash forward into scenes not quite real in the ordinary sense, yet still vivid in their proliferation round neurons somewhere near the frontal cortex of now wildly ecstatic electric insights. An unseen flurry of activity happening with both text and synapses takes hold of imaginations, overlapping the interiors of enclosed eyelids with an orchestra of spontaneous cerebral events, lighting and coloring in a myriad of shadowy imagery. As the written word trails on, the conceptual landscape of this printed text etches into thought the forefront of everlasting creativity.
