Deep Learning
To define clearly what we’re talking about when we mention AI. What is artificial intelligence, machine learning, and deep learning ? How do they relate to each other
Artificial intelligence (AI) : Whether computers could be made to “think”.
Machine learning (ML): System will trained rather than explicitly programmed.
Deep learning (DL): Learning representations from data that puts an emphasis on learning successive layers of increasingly meaningful representations
In classical programming, Developer inputs the rules and data depends on inputted data according to program guidelines will get answers as out put.
In machine learning programmer need to give input as existing / available data and output / Answers from input data and trained the system, that eventually allows the system to come up with rules for automating the task.
Machine learning discovers rules to execute a data-processing task, given examples of what’s expected.
Learning: This is necessary in order to determine the distance between the algorithm’s current output and its expected output. The measurement is used as a feedback signal to adjust the way the algorithm works. This adjustment step is what we call learning.
The deep in deep learning stands for this idea of successive layers of representations. How many layers contribute to a model of the data is called the depth of the model. they’re all learned automatically from exposure to training data these layered representations are learned via models called neural networks. The term neural network is a reference to neuro biology (inspiration from our understanding of the brain), deep learning is a mathematical framework.
The primary reason deep learning took off so quickly is that it offered better performance on many problems, Deep learning also makes problem-solving much easier, because it completely automated.
What is Neural Network?
a neural network developed depends on human brain working, hence it will matches the human brain. A neuron communicates with other neurons at special places called synapses. This is abstracted as a graph of nodes (neurons) connected by weighted edges (synapses), If sufficient synaptic inputs fire to a neuron, that neuron will also fire…. we call this process only “thinking”:)
Same logic used in neural network as given below.
Neurons. A neural network is a graph of neurons. A neuron has inputs and outputs. Similarly, in neural network we are giving inputs and getting outputs.as given in Sample neuron network image X1, X2, X3 are input neurons and Y is output neuron.
Connections. Internally a neural network connected(linked) with connections, each connection transferring the data from one-layer neuron to another layer of a neuron, I.E output of one layer will be input to another layer
Weights. while transferring the data from one-layer neuron to another layer of a neuron each connection will allocate a weight as shown in the Image: Sample Neural Network W1, W2, W3 are weights.
Propagation Function. The propagation function computes the input of a neuron from the outputs of previous layer neurons. The propagation function is leveraged during the forward propagation stage of training, as given in Image: Sample Neural Network propagation function doing dot product of input neuron value with allocated weight and summing the value of the same will be giving as output to the next layer.
Learning Rule. If we allotted weight’s are not correct that time propagation function will give output which is not assured the proper results, where accuracy rate will be less, to handle this we have a learning rule which is a function that modifies the weights allotted to the connections. This will repeatedly happen until will get good accuracy rate. This rule will be called at the time of backward propagation while working with training data on given data set.
How Deep Learning work?
layer: The fundamental data structure in neural networks is the layer, A layer is a data-processing module that takes as input one or more tensors and that outputs one or more tensors. Some layers are stateless, but more frequently layers have a state, the layer’s weights, one or several tensors learned with stochastic gradient descent, which together contain the network’s knowledge.
Models /How many Layers :
A deep-learning model is a directed, acyclic graph of layers. The most common instance is a linear stack of layers, mapping a single input to a single output. Picking the right network architecture / model is more an art than a science.
Loss function and optimizer:
Once the network architecture is defined, you still have to choose two more things.
Loss function (objective function) — The quantity that will be minimized during training. It represents a measure of success for the task at hand.
A neural network that has multiple outputs may have multiple loss functions (one per output). But the gradient-descent process must be based on a single scalar loss value;so, for multi loss networks, all losses are combined (via averaging) into a single scalar quantity.
Optimizer — Determines how the network will be updated based on the loss function.It implements a specific variant of stochastic gradient descent (SGD).
Choosing the right objective function for the right problem is extremely important
above given image will explain about the internal architecture of neural network. In Deep Learning model will target to composed of layers, that are chained together and maps the input data to predictions.
The loss function compares these predictions to the targets and it will producing a loss value, in this stage architect need to think a measure of how well the network’s predictions match what was expected and then The optimizer uses this loss value to tune and update the network’s weights.
Dynamic Programming- Memoization:
it’s better to use Memoization in developing any model, Memoization is a caching technique that is used as a software optimization technique(compute once and reuse the same). A cache is responsible for storing the results of a process with the aim to use it later on. In the case of Deep learning, memorization is applied to a function whenever we wish to store its output given its input. This is very useful when we run functions that are computationally expensive but it will work fast.
Conclusion:
There are mainly three types of neural networks available, we can focus on the same. They are:
MLP : Multi-Layer Perceptron’s
CNN : Convolutional Neural Networks
RNN: Recurrent Neural Networks
we will discuss about each individual type in separate block, which i will publish in future with examples and sample code.
References:
- Deep Learning with Python by FRANÇOIS CHOLLET
- https://medium.com/machinevision/overview-of-neural-networks-b86ce02ea3d1
- https://www.appliedaicourse.com/
- Google images
