day 15 deep learning and neural network

Deep learning and neural network

Deep learning

What is the difference between deep learning and machine learning?

AI is the science of machines building its own self rule system.

Out of those science, machine learning is self learning data by analyzing data and learning the patterns and making predictions or decisions based on it.

Deep learning is a type of machine learning where the model is based on neuron networks.

According to Yoshua Bengio an expert in deep learning,

• Deep learning is inspired by neural networks of the brain to build learning machines which discover rich and useful internal representations, computed as a composition of learned features and functions.

Representation Learning

What does Representation learning mean?

We have represented the data on the right in four ways.

• particle: data it self

• image: visualized representation of the particle

• dataframe: organize data’s features in columns and rows

• category: values we want to predict

The more internal abstract the data is, the more it is handled by humans.

Below is an example of VGG16 model processing images.

VGG16 model can process single input images internally as 3rd dimension array. It returns a 1 dimension vector conmposed of 1000 numerals. VGG16 is a already well trained network and can represent features in 1000 numerals. If the image size is bigger, then it will return 150000 numeral data instead to accurately represent the data even if there is no human intervention.

In deep learning, the goal is to return a internal representation of the raw data.

Philosophy

Behaviorism

Behaviorism states that stimulus leads to response.

Skinner’s box shows that through operant conditioning, animals can learn by stimuls. To repeat this process to reincorce the wanted response is called reinforcement learning.

Cognitive

Coginitiveness states that there is a cognitive process in between stimulus and response, and this led to neurological science.

Connectionism

Information proccesing in the brain happens through many different conneting neurons.

• connecting neurons experience many cases and learn slowly

• learned information is stored in the connecting neurons, and with outside stimulus, the connecting patterns change and the learned information changes accordingly

Deep learning follows connectivism through input, output, representaiton.

neuron networks

functions

y is a function of x.

This means that the change in y is dependent on chagne of x.

linear algebra

import numpy as np

def linear_transformation(X, A):
    transformed_vector = np.matmul(X, A)
    return transformed_vector

X = np.array([-1, 2])

A = np.array([[1, -2],
              [3, 0]])

linear_transformation(X, A)

array([5, 2])

more on functions

• scala: only has magnitude but no direction. Expressed with a single number

• vector: magnitutde and direction. expressed with many numbers.

one-to-one

Univariate Regression

y is a scala

many-to-one

Multivariate Regression

we input vectors and we return a scala value

generally this example is called a regression in ML/DL

models

In ML/DL, we look for the function that best closely expresses the target value

1. We must decide which function to express the model

   • if the input is a single value, we can use one-to-one function such as y = wx + b

   • if the input is a vector vlaue, we can use many-to-one function such as a Multivariate function

   • use neural network

if the data is a structured data, we can use linear functions. However, if the data is an Unstructured data that means we probabily should use neural network.

Inductive Bias is the presume that there will be an optimal function that expresses the data.

1. Find the most optimal function from the model funciton we chose

We perform learning to decide which function best predicts the values.

ML/DL calls this model learning.

In deep learning it is called Gradient Descent.

AI and the future

ML/DL

Machine learning is about learning from the given data and making predictions.

Deep learning focuses on bringing out the internal expression itself from the input data.

In ML it is essential for people to curiate a feature for input for high efficient learning. However this takes a lot of effort.

In DL, it reduces that. The model learns the features on its own. Therefore it is very useful for unstructured data which would be difficult to transform it into a structured data

LIME

Interpretable Machine Learning library LIME