SlideShare ist ein Scribd-Unternehmen logo
1 von 24
Downloaden Sie, um offline zu lesen
Neural Networks
and Deep Learning
Tijmen Blankevoort
Scyfer

Prof dr. Max Welling

Drs. Jorgen Sandig

Msc. Taco Cohen
Deep Learning
All purpose machine learning
Using Neural Networks:
- Using large amounts of data
- Learning very complex problems
- Automatically learning features
A new era of machine learning
Deep learning wins all competitions
- IJCNN 2011 Traffic Sign Recognition Competition
- ISBI 2012 Segmentation of neuronal structures in EM
stacks challenge
- ICDAR 2011 Chinese
handwriting recognition
Applications
A lot of state of the art systems use deep learning to some
extent:
- IBMs Watson: Jeopardy contest 2011
- Google’s self-driving car
- Google Glasses
- Facebook face recognition
- Facebook user modelling
Mostly image and sound recognition tasks (difficult)
Google Brain (2011)
- 10 million youtube/imagenet images
- 1 billion parameters
- 16.000 processors
- Largely unsupervised!
- 20.000 categories
- 15.8% accuracy
Bigger, better
Deep Learning:
- The scope of what
computers can learn has
greatly been increased
- Interaction with the real
world
Biological Inspiration

Neuron
Neuron computer model
Activation function

Sigmoid activation function
Neuron computer model

Perceptron - 1957 Rosenblatt
Easy functions with a neuron
Linking neurons and training
- Initialize randomly
- Sequentially give it data.
- See what the difference is between
network output and actual output.
- Update the weights according to this error.
- End result: give a model input, and it
produces a proper output.
Quest for the weights. The weights are the
model!
The Perceptron (1958)

“A machine which senses, recognizes, remembers, and responds like the human mind”
“Remarkable machine… [was] capable of what amounts to thought” - The New Yorker
Criticism and downfall (1969)
- Perceptrons are painfully limited. They can not even learn a
simple XOR function!
- No feasible way of learning networks with multiple layers
- Interest in neural networks close to fully disappeared
Renewed interest (90’s)
- Learning multiple layers
- “Back propagation”
- Can theoretically learn any
function!
But…
Very slow and inefficient
- Machine learning attention
towards SVMs, random forests
etc.
Deep learing (2006)
- Quest: Mimic human brain representations
- Large networks
- Lots of data

Problem:
Simple back propagation fails
on large networks.
Deep learning (2006)
- Exactly same networks as
before, just BIGGER

- Combination of three factors:
- (Big data)
- Better algorithms
- Parallel computing (GPU)
Better algorithms

Restricted Boltzmann machine
Pre-training: Learn the representation by parts!
Very strong unsupervised learning
After pre-training, use back propagation
Parallel (GPU) power
- Every set of weights can be stored as a matrix (w_ij)
- GPUs are made to do common parallel problems fast!
- All similar calculations done at the same time, huge performance boost.
- CPU parallelizing
Future of Deep Learning
- Currently an explosion of developments
- Hessian-Free networks (2010)
- Long Short Term Memory (2011)
- Large Convolutional nets, max-pooling (2011)
- Nesterov’s Gradient Descent (2013)
- Currently state of the art but...
- No way of doing logical inference (extrapolation)
- No easy integration of abstract knowledge
- Hypothetic space bias might not conform with reality
When to apply Deep Learning
- Generally, vision and sound
recognition, but...
- Works great for any other problem too!
- A lot of data / features
- Don’t want to make your own features
- State of the art results
How to apply Deep Learning
Deep learning is very difficult!
- No easy plug and play software
- Far too many different networks/options/additions
- Mathematics and programming very challenging
- Research is fast paced
- Learning a network is both an art and a science
My advice:
Cooperation university <=> business
How to apply Deep Learning
- For most current business problems, no need for
expensive hardware. e.g. we use a laptop

Weitere ähnliche Inhalte

Was ist angesagt?

Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Simplilearn
 
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Simplilearn
 
Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...
Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...
Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...
Sri Ambati
 

Was ist angesagt? (20)

Deep learning
Deep learningDeep learning
Deep learning
 
Deep Learning Explained
Deep Learning ExplainedDeep Learning Explained
Deep Learning Explained
 
Deep learning ppt
Deep learning pptDeep learning ppt
Deep learning ppt
 
Deep learning - what is it and why now?
Deep learning - what is it and why now?Deep learning - what is it and why now?
Deep learning - what is it and why now?
 
Deep learning presentation
Deep learning presentationDeep learning presentation
Deep learning presentation
 
Deep Learning: Application & Opportunity
Deep Learning: Application & OpportunityDeep Learning: Application & Opportunity
Deep Learning: Application & Opportunity
 
Machine learning
Machine learningMachine learning
Machine learning
 
Introduction to Deep Learning
Introduction to Deep Learning Introduction to Deep Learning
Introduction to Deep Learning
 
Deep learning
Deep learningDeep learning
Deep learning
 
Convolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep LearningConvolutional Neural Network Models - Deep Learning
Convolutional Neural Network Models - Deep Learning
 
Introduction to Neural Networks
Introduction to Neural NetworksIntroduction to Neural Networks
Introduction to Neural Networks
 
Introduction of Deep Learning
Introduction of Deep LearningIntroduction of Deep Learning
Introduction of Deep Learning
 
Introduction to Deep Learning
Introduction to Deep LearningIntroduction to Deep Learning
Introduction to Deep Learning
 
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...
 
Machine learning
Machine learningMachine learning
Machine learning
 
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
Supervised and Unsupervised Learning In Machine Learning | Machine Learning T...
 
Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...
Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...
Interpretable Machine Learning Using LIME Framework - Kasia Kulma (PhD), Data...
 
Introduction to Machine Learning Classifiers
Introduction to Machine Learning ClassifiersIntroduction to Machine Learning Classifiers
Introduction to Machine Learning Classifiers
 
What is Deep Learning?
What is Deep Learning?What is Deep Learning?
What is Deep Learning?
 
machine learning
machine learningmachine learning
machine learning
 

Andere mochten auch

What Is Conversational Marketing?
What Is Conversational Marketing?What Is Conversational Marketing?
What Is Conversational Marketing?
Drift
 

Andere mochten auch (6)

Learning To Rank data2day 2017
Learning To Rank data2day 2017Learning To Rank data2day 2017
Learning To Rank data2day 2017
 
Machine learning with scikitlearn
Machine learning with scikitlearnMachine learning with scikitlearn
Machine learning with scikitlearn
 
Deep Learning: a birds eye view
Deep Learning: a birds eye viewDeep Learning: a birds eye view
Deep Learning: a birds eye view
 
Artificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep LearningArtificial Intelligence, Machine Learning and Deep Learning
Artificial Intelligence, Machine Learning and Deep Learning
 
What Is Conversational Marketing?
What Is Conversational Marketing?What Is Conversational Marketing?
What Is Conversational Marketing?
 
Deep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word EmbeddingsDeep Learning for NLP: An Introduction to Neural Word Embeddings
Deep Learning for NLP: An Introduction to Neural Word Embeddings
 

Ähnlich wie Neural networks and deep learning

Y conf talk - Andrej Karpathy
Y conf talk - Andrej KarpathyY conf talk - Andrej Karpathy
Y conf talk - Andrej Karpathy
Sze Siong Teo
 

Ähnlich wie Neural networks and deep learning (20)

Fascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange TomorrowFascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange Tomorrow
 
Y conf talk - Andrej Karpathy
Y conf talk - Andrej KarpathyY conf talk - Andrej Karpathy
Y conf talk - Andrej Karpathy
 
Deep learning & Humanity's Grand Challenges
Deep learning & Humanity's Grand ChallengesDeep learning & Humanity's Grand Challenges
Deep learning & Humanity's Grand Challenges
 
Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial IntelligenceDeep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
 
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
Artificial Intelligence is back, Deep Learning Networks and Quantum possibili...
 
Deep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and HypeDeep learning: Cutting through the Myths and Hype
Deep learning: Cutting through the Myths and Hype
 
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2ODeep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
Deep Water - Bringing Tensorflow, Caffe, Mxnet to H2O
 
Null
NullNull
Null
 
A Platform for Accelerating Machine Learning Applications
 A Platform for Accelerating Machine Learning Applications A Platform for Accelerating Machine Learning Applications
A Platform for Accelerating Machine Learning Applications
 
Artificial intelligence and Neural Network
Artificial intelligence and Neural NetworkArtificial intelligence and Neural Network
Artificial intelligence and Neural Network
 
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro..."Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
"Methods for Understanding How Deep Neural Networks Work," a Presentation fro...
 
Introduction to deep learning
Introduction to deep learningIntroduction to deep learning
Introduction to deep learning
 
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
Lebanon SoftShore Artificial Intelligence Seminar - March 38, 2014
 
Deep learning introduction
Deep learning introductionDeep learning introduction
Deep learning introduction
 
From web 2 to web 3
From web 2 to web 3From web 2 to web 3
From web 2 to web 3
 
Fascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange TomorrowFascinating Tales of a Strange Tomorrow
Fascinating Tales of a Strange Tomorrow
 
Deep Learning with GPU Instances - Pop-up Loft TLV 2017
Deep Learning with GPU Instances - Pop-up Loft TLV 2017Deep Learning with GPU Instances - Pop-up Loft TLV 2017
Deep Learning with GPU Instances - Pop-up Loft TLV 2017
 
Introduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolutionIntroduction to the Artificial Intelligence and Computer Vision revolution
Introduction to the Artificial Intelligence and Computer Vision revolution
 
Deep learning tutorial 9/2019
Deep learning tutorial 9/2019Deep learning tutorial 9/2019
Deep learning tutorial 9/2019
 
Deep Learning Tutorial
Deep Learning TutorialDeep Learning Tutorial
Deep Learning Tutorial
 

Kürzlich hochgeladen

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Kürzlich hochgeladen (20)

A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Advantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your BusinessAdvantages of Hiring UIUX Design Service Providers for Your Business
Advantages of Hiring UIUX Design Service Providers for Your Business
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 

Neural networks and deep learning

  • 1. Neural Networks and Deep Learning Tijmen Blankevoort
  • 2. Scyfer Prof dr. Max Welling Drs. Jorgen Sandig Msc. Taco Cohen
  • 3. Deep Learning All purpose machine learning Using Neural Networks: - Using large amounts of data - Learning very complex problems - Automatically learning features A new era of machine learning
  • 4. Deep learning wins all competitions - IJCNN 2011 Traffic Sign Recognition Competition - ISBI 2012 Segmentation of neuronal structures in EM stacks challenge - ICDAR 2011 Chinese handwriting recognition
  • 5. Applications A lot of state of the art systems use deep learning to some extent: - IBMs Watson: Jeopardy contest 2011 - Google’s self-driving car - Google Glasses - Facebook face recognition - Facebook user modelling Mostly image and sound recognition tasks (difficult)
  • 6. Google Brain (2011) - 10 million youtube/imagenet images - 1 billion parameters - 16.000 processors - Largely unsupervised! - 20.000 categories - 15.8% accuracy
  • 7. Bigger, better Deep Learning: - The scope of what computers can learn has greatly been increased - Interaction with the real world
  • 11. Neuron computer model Perceptron - 1957 Rosenblatt
  • 13. Linking neurons and training - Initialize randomly - Sequentially give it data. - See what the difference is between network output and actual output. - Update the weights according to this error. - End result: give a model input, and it produces a proper output. Quest for the weights. The weights are the model!
  • 14. The Perceptron (1958) “A machine which senses, recognizes, remembers, and responds like the human mind” “Remarkable machine… [was] capable of what amounts to thought” - The New Yorker
  • 15. Criticism and downfall (1969) - Perceptrons are painfully limited. They can not even learn a simple XOR function! - No feasible way of learning networks with multiple layers - Interest in neural networks close to fully disappeared
  • 16. Renewed interest (90’s) - Learning multiple layers - “Back propagation” - Can theoretically learn any function! But… Very slow and inefficient - Machine learning attention towards SVMs, random forests etc.
  • 17. Deep learing (2006) - Quest: Mimic human brain representations - Large networks - Lots of data Problem: Simple back propagation fails on large networks.
  • 18. Deep learning (2006) - Exactly same networks as before, just BIGGER - Combination of three factors: - (Big data) - Better algorithms - Parallel computing (GPU)
  • 19. Better algorithms Restricted Boltzmann machine Pre-training: Learn the representation by parts! Very strong unsupervised learning After pre-training, use back propagation
  • 20. Parallel (GPU) power - Every set of weights can be stored as a matrix (w_ij) - GPUs are made to do common parallel problems fast! - All similar calculations done at the same time, huge performance boost. - CPU parallelizing
  • 21. Future of Deep Learning - Currently an explosion of developments - Hessian-Free networks (2010) - Long Short Term Memory (2011) - Large Convolutional nets, max-pooling (2011) - Nesterov’s Gradient Descent (2013) - Currently state of the art but... - No way of doing logical inference (extrapolation) - No easy integration of abstract knowledge - Hypothetic space bias might not conform with reality
  • 22. When to apply Deep Learning - Generally, vision and sound recognition, but... - Works great for any other problem too! - A lot of data / features - Don’t want to make your own features - State of the art results
  • 23. How to apply Deep Learning Deep learning is very difficult! - No easy plug and play software - Far too many different networks/options/additions - Mathematics and programming very challenging - Research is fast paced - Learning a network is both an art and a science My advice: Cooperation university <=> business
  • 24. How to apply Deep Learning - For most current business problems, no need for expensive hardware. e.g. we use a laptop