What does deep learning do exactly? How does it do it?
In: Technology
Deep Learning is a bit of jargon that describes a neural network with a large number of consecutive layers.
So what is a ‘neural network’?
Let’s say you want to buy a house. You want a certain number of bedrooms, a certain number of bathrooms, a certain type of neighborhood, a certain kind of garage, a number of floors, price range, etc. All of these are called ‘features’ by the kind of people who use phrases like ‘neural network’.
So you take all these features, weight them somehow and end up with a number that expresses how much you want any particular house. Maybe you think price range is more important than bathroom count or number of floors is less important neighborhood – you assign weights to these and then just compute which option is best.
But what happens when you don’t know what weights to use?
This happens with classification problems all the time. I can show you two pictures – one with a stop sign and one without – that you can easily discriminate. You can sort all such pictures into two piles pretty easily. But what you can’t do easily is write an equation that takes the colors of every individual pixel and computes the difference between ‘stop sign picture’ and ‘not a stop sign picture’. It’s simply too complex.
Instead what we do is ‘train’ an equation. We write an incredibly complex set of simultaneous equations where every feature has a weight. All we need to do is figure out what those weights should be and we’ll have a mathematical structure that can predict whether a picture has a stop sign or not based solely on the colors of each individual pixel.
That’s basically all a neural net is.
Deep Learning is simply a way of building a neural net that’s more manageable (and more computable) than simply creating a single massive set of simultaneous equations. Instead of trying to do the whole problem at once, we divide it up and incrementally solve the problem in an abstract fashion layer-by-layer. The more layers, the better we can discriminate our two sets – but the more difficult figuring out the weights becomes.
Essentially you program a basic model of a brain, but it’s completely random at first, you give it some data, and then it will give a random value, but another program tries to tweak it, so that the values get closer to what it should be, repeat forward thousands of times and you get a pretty good ai
It’s essentially like tuning an instrument, you know which way to turn the knobs, but not by how much, but instead of one knob, there are thousands of knobs
In conventional programming, you write very exact instructions. A computer programme won’t do anything you didn’t instruct first. Most bugs are really just mistakes in our instructions.
With deep learning methods we write software that teaches itself certain skills through examples. For example, a deep learning image recognition algorithm can be shown thousands or millions of examples of kittens so that it can learn to recognise kittens in images it’s never seen before.
[Here’s a fantastic video](https://www.youtube.com/watch?v=K-wIZuAA3EY) on someone who wrote some code that creates a simple virtual creature with legs and then runs dozens upon dozens simulations where it tries to move it’s legs in different fashions in an attempt to learn how to walk.
Just watch the vid, it explains everything.
Latest Answers