Codeless Deep Learning for Language Modeling and Image Classification

© 2020 KNIME AG. All Right Reserved.
Codeless Deep Learning for Language Modeling
and Image Classification
Corey Weisinger
Corey.Weisinger@knime.com
Rosaria Silipo
Rosaria.Silipo@knime.com

© 2020 KNIME AG. All Rights Reserved.
Agenda
• Introduction to the open source tool KNIME Analytics Platform
• Introduction to RNN units and their Applications
• Transfer learning for image classification
2

What is KNIME Analytics Platform?
• A tool for data analysis, manipulation, visualization, and reporting
• Based on the graphical programming paradigm

Visual KNIME Workflows
NODES perform tasks on data
Nodes are combined to create
WORKFLOWS
Status
Inputs Outputs
Not Configured
Configured
Executed
Error

Analysis & Mining
Statistics
Data Mining
Machine Learning
Web Analytics
Text Mining
Network Analysis
Social Media Analysis
R, Weka, Python
Community / 3rd
Data Access
MySQL, Oracle, ...
SAS, SPSS, ...
Excel, Flat, ...
Hive, Impala, ...
XML, JSON, PMML
Text, Doc, Image, ...
Web Crawlers
Industry Specific
Community / 3rd
Transformation
Row
Column
Matrix
Text, Image
Time Series
Java
Python
Community / 3rd
Visualization
R
JFreeChart
JavaScript
Plotly
Community / 3rd
Deployment
via BIRT
PMML
XML, JSON
Databases
Excel, Flat, etc.
Text, Doc, Image
Industry Specific
Community / 3rd
Over 2000 Native and Embedded Nodes Included:

KNIME Software - Better Decision-making, Faster!
DeployManageAutomateCollaborate
OpenSource
KNIME Server
Partner
Extensions
KNIME
Analytics Platform
Community
Extensions
KNIME
Extensions
KNIME
Integrations
VisualizeAnalyzeTransformIntegrateLoad

© 2020 KNIME AG. All Rights Reserved. 7
Introduction to RNN and their Applications

Neural Network Review

Let’s start with a history lesson
Neuron Networks
9
Neural networks are inspired by biological neural networks
𝑥
𝑥
∑ σ0,8
𝑤
𝑤 𝑏
𝑎 = 𝜎(𝑥 𝑤 + 𝑥 𝑤 + 𝑏)

Fully connected, feed forward networks
10
Input
Layer
Hidden
Layer
Output
Layer Forward pass:
∑
∑
∑
∑
𝑤 ,
𝑤 ,
𝑤 ,
𝑤 ,
𝑤 ,
𝑏
𝑏
𝑏
𝑤 ,
𝑤 ,
𝑤 ,
𝑤 ,
𝑏

Frequently used activation functions
11
Sigmoid Tanh Rectified Linear Unit (ReLU)

Fully connected, feed forward networks- simplified
12
Input
Layer
Hidden
Layer
Output
Layer Forward pass:
∑
∑
∑
∑
𝒙
𝟐
𝒚
𝟑

What are RNNs and LSTMs?

What are RNNs?
• Recurrent Neural Network (RNN) are a family of neural networks used
for processing sequential data
• RNNs are used for all sorts of tasks:
– Language modeling / Text generation
– Text classification
– Neural machine translation
– Image captioning
– Speech to text
– Numerical time series data, e.g. sensor data
14

Why do we need RNNs for sequential data?
• Goal: Translation network from German to English
“Ich mag Schokolade”
=> “I like chocolate”
• One option: Use feed forward network to
translate word by word
• But what about this sentence?
“Kathrin mag Schokolade”
=> “Kathrin likes chocolate”
15
𝑥 ∑ σ 𝑦
∑ σ
∑ σ
∑ σ
∑ σ
∑ σ
∑ σ
∑ σ
Input x Output y
Ich I
mag like
Schokolade chocolate

Why do we need RNNs for sequential data?
• Problems:
– Each time step is completely independent
– For translations we need context
– More general: we need a network that
remembers inputs from the past
• Solution: Recurrent neural networks
16
𝑥 ∑ σ 𝑦
∑ σ
∑ σ
∑ σ
∑ σ
∑ σ
∑ σ
∑ σ
Input x Output y
Ich I
mag like
Schokolade chocolate

What are RNNs?
17
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-LSTMs/

From feed forward to recurrent neural networks
18
∑
∑
∑
∑
∑
∑
∑
𝒙
𝟐
𝒚
𝟑
𝒙
𝟐
𝒚
𝟑

From feed forward to recurrent neural networks
19
∑
∑
∑
𝒙
𝟐
𝒚
𝟑
∑
∑
∑
𝒙
𝟐
𝒚
𝟑
∑
∑
∑
𝒙
𝟐
𝒚
𝟑
∑
∑
∑
𝒙
𝟐
𝒚
𝟑

Simple RNN unit
20
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-LSTMs/

Limitations of simple layer structures
21
The “memory” of simple RNNs is sometimes too limited to be
useful
– “Cars drive on the ” (road)
– “I love the beach – my favorite sound is the crashing of the “
(cars? glass? waves?)

LSTM = Long Short Term Memory Unit
• Special type of unit with three gates
– Forget gate
– Input gate
– Output gate
22
Image Source: Christopher Olah, https://colah.github.io/posts/2015-08-Understanding-L

Different network-structures and applications
Many to Many
23
A A A
<sos>
A
like sailing
sailinglikeI
I
<eos>
Language model Neural machine translation
E E E
like sailingI
D D D
Ich gehe gerne
D
segeln
Ich gehe gerne

Different network-structures and applications
Many to one
24
A A A A
I
A
like to go sailing
English
Language classification
Text classification
One to many
A A A AA
Couple onsailing a lake
Image captioning

How can we train a recurrent neural network for text generation
• Decide between character, sub word, or word level
– Word level
• How we structure language
• Length of sequences are much shorter
– Character level
• Smaller dictionary => much more manageable
• Ability to generate new words
– Sub-word level
• Happy medium
• What kind of data can we use?
• How can we prepare our data for the training?
25

The Creative Side of AI: Naming New Products

The Product Naming Phase
Naming new products is not as easy as it might sound …
27
What happens when a new product is born?

The Case Study: new line of outdoor clothing
Outdoor or hiking clothing line needs
new names that:
• Evoke the feeling of nature
• Sound familiar to customers
• Stand out from the competition
• Are not covered by copyright
28
• The team also needs many potential candidates to evaluate
• Could a neural network help in this creative process?

Let’s build a model to generate fictional mountain names!
29

Quick sneak preview – Find the fake mountain names
30
Set 1
Barlock Mountain
Casterland Mountain
Shafford Peak
Set 2
Roblin Hill
Buxley Mountain
Baldy Rock
Set 3
Terrey Hill
Walter Hill
Little Buck Butte
Correct answer: ALL!

The Dataset
31
• 33,012 names of mountains in
the US
• Extracted from Wikipedia via a
Wikidata query

Many to many structure
32

LSTM based Neural Network: Many to Many
33
0,1,0,0,0,0,0,0,0,...,0
0,0,0,1,0,0,0,0,0,...,0
Mount Baker
u o M
0,0,0,0,0,0,1,0,0,...,0
0,0,0,0,0,0,0,1,0,...,0
0,0,0,0,1,0,0,0,0,...,0
t n
0,0,0,0,0,0,0,0,1,...,0
B
1,0,0,0,0,0,0,0,0,...,0
95=dictionarysize
M samples
95=dictionarysize
a
b
k
m
n
z
...
N=256 LSTM states
M = max. Length of mountain name
0,0,0,1,0,0,0,0,0,...,0
u o
0,0,0,0,0,0,1,0,0,...,0
0,0,0,0,0,0,0,1,0,...,0
0,0,0,0,1,0,0,0,0,...,0
t n
0,0,0,0,0,0,0,0,1,...,0
B
1,0,0,0,0,0,0,0,0,...,0
M samples
0,1,0,0,0,0,0,0,0,...,0
a
linear
softmax
N
LSTMstates

The KNIME Keras Integration
34
• Codeless GUI based
• Fully opensource
• Keras functions
https://www.knime.com/deeplearning/keras

Codeless?
35
LSTMstates

Model training workflow
• Read in and transform the input
dataset
• Define the structure of the
neural network
• Train the model using Keras
Learner
• Do a bit of post-training model
editing to introduce temperature
• Convert model to TensorFlow
and save for use in deployment
workflow
36

Now that we have a model, how do we use it?
37

Neural network: Code-free
Layers of our neural network:
• Input layer of size [?, 95]
• LSTM for sequence analysis
• Dropout layer to prevent
overfitting
• Dense layer 1 – linear activation
• Dense layer 2 (output) – Softmax
activation
Once we’ve defined the network
structure and pre-processed our
input, we’re ready to train.
38

Want to see the names?
39
Advantages:
- They remind you of mountains. Do
not they?
- Sound familiar enough
- Evoke the feeling of nature
- No copyright issues
- Automatic generation in 10 seconds
- You can generate as many as you
want
- No people involved as of now
Can this be considered a creative task?

Yo! AI Generated Rap Songs

Bots & speaking styles
41
- Articulate
- Chatty
- Clean
- Conversational
- Crisp
- Declamatory
- Diffuse
- Discoursive
- Eloquent
- Emphatic
- Epigrammatic
- Epistolary
- Euphemistic
- Flowery
- Funny
- Fluent
- Formal
- Gossipy
- Idiomatic
- Incoherent
- Informal
- Journalistic
- Literary
- Lyric
- Ornate
- Parenthetical
- Pejorative
- Picturesque
- Poetic
- Prolix
- Punchy
- Rambling
- Rhetorical
- Rough
- Sesquipedalianhttps://writerswrite.co.za/60-words-used-to-describe-writing-or-speech-style/

How should my bot speak?
• I want my bot to answer in a polite tone in some situations, even
almost poetic
• I want my bot to answer in an affirmative tone in other situations,
borderline impolite
42

Let‘s start with impolite ...
• Let‘s generate a rap song!
• This is a free text generation problem
• Similar to the problem of the product name generation, just on longer
complete sentences
• Let’s build a similar network from scratch!
• Maybe not.
• Let’s search Kathrin’s example on the KNIME Hub
• and readapt it!
43

On the KNIME Hub
44
https://hub.knime.com

On the KNIME Hub
45
https://kni.me/w/mGO9nXhmjzIKiqHU

LSTM based Neural Network
46
N
0,1,0,0,0,0,0,0,0,...,0
0,0,0,1,0,0,0,0,0,...,0
u o h
0,0,0,0,0,0,1,0,0,...,0
0,0,0,0,0,0,0,1,0,...,0
s
86=dictionarysize
M past samples
86=dictionarysize
a
b
e
m
n
z
...
Dictionary size = 86
256 LSTM states
100 past characters
Training set = 23 popular rap songs
LSTMstates

Creative AI: the training workflow
47

Creative AI: the deployment workflow
48

don't need no advice
You're not here and we both know why, so
Move from me when you're extra
Move from me with the passa
I'm building up a house where they raised me
You move with me I'll go crazy
Don't switch on me, I got big plans
We need to forward to the islands and get you gold,
no spray tans
I need you to stop running back to your ex he's a wasteman
I wanna know how come we can never slash and stay friends
I'm blem for real, I might just say how I feel
Together forever
Good morning, good afternoon, goodnight
I'm here to talk about
More Life On the Phostest, *** tike and push it
And push it good for the cremm
Mand a ray
*** the but bo che turn me
But dow of the
Purb everydor get my mus, ****
**** you move my home back hoot on the als to a lot on the
crim just the yourh
I'm a sixe and got a proup in the stake of the
Ond I won't stop abusing it
To groupie girls stop false accusing it
Back to the music
The Maybach roof is translucent
**** got a problem Houston
What up be
The
Let’s rappify some text
Brick X6,
Phey, cabe,
make you feel soom the way (I smoke good!)
I probably make (What?)
More money in six months,
Than what's in your papa's safe (I'm serious)
Look like I robbed a bank (Okay Okay)
I set it off like Queen Latifah
'Cause I'm living single I'm feeling cautious
I ain't scream when they served a subpoena
(Can't go back to jail)
I heard that he a leader
(Who pood, what to be *****' up
The baugerout Black alro Black X6,
Phantom White X6 looks like a panda
Goin' out like I'm Montana
Hundred killers, hundred hammers Black X6,
Phantom White X6, panda
Pockets swole, Danny
Sellin' bar, candy
Man I'm the macho like Randy
The choppa go Oscar for Grammy
**** **** pull up ya panty
Hope you killas understand me
Hey Panda, Panda Panda,
Panda, Panda, Panda, Panda
I got broads in Atlanta
Twistin' dope, lean, and the Fanta
Credit cards and the scammers
Hittin' off licks in the bando
49
th eaf hard fortars
I wanna lister for ald be gadg to call up in the
Phantom
Know *****, they come and kill you on the camera
Big Rollie, it dancin' bigger than a Pandie
Go Oscar for Grammy, **** pull up your panty
Fill up
I'ma flip it, I got **** pull up and peese vire
Reepin me sher wing these hoes
Don't settle for less one you lise one
And my cas up in the
Pun I arn’t **** with me
If you wanted to
These expensive, these is red bottoms
These is bloody somes
It the street to you heare it all while I be
You in the club just to party I'm there,
I get paid a fee
I be in and out them banks so much
I know they're tired of me
Honestly, don't give a ****
'Bout who ain't fond of me
Dropped two mixtapes in six months
What **** working as hard as me?
I don't bother with these hoes
Don't let the truth to the young black youth
But shorty's running with my ****, for the man,
I as what
I was gang affiliated, got on TV and told on me
I guess that's why last winter she got so cold on me
She s
When the cat is away
The mice shall play
When the cat is away
The mice shall play
Never mind I'll find someone like you
I wish nothing but the best
For you too, don't forget me
I beg, I
lease, yua do not a aze me I around my dramy
And when I'm finished, bring the yellow tape
To tape off the scene of the slaughter
Still gettin' swoll off bread and water I don't know if they
*** or what
Search a *** down, and grabbin' his ***
And on the other hand, without a gun they can't get none
But don't let it be a black and a white one
'Cause they'll slam ya down to the street top
Black police showin' out for the white cop
Ice Cube will swarm
On any mother**** in a blue uniform
Just cause I'm from, the CPT
Punk police are afraid of man en
I han to we have a keround hord **** it
I'm right
What more can I say to you?
Get my grown man on
Let's go! (What more can I say?)
Now you know ass is willie
When they got you in a mag
For like half a billi
And your ass ain't Lilly
White
That mean that shit you write must be illy
Either that or your flow is silly
It's both I don't mean to boast
But damn if I don't brag
Them crackers gonna act like I ain't on they ads
The Martha Stewart
That
Yo!
This post is about generating free text
with a deep learning network
particularly it is about
This License refers to version of the GNU General Public
License.
Copyright also means copyright-bi

To be or not to be … Shakespearian text?

Now for the poetic text ...
• How would Shakespeare say it?
• I can reuse the same LSTM based network
• With a different training set
• Training set is the full texts of
– “King Lear”
– “Othello”
– “Much ado about nothing”
• The only difference in the network is in the dictionary size
51
“Many a true word hath been spoken in jest.”
― William Shakespeare, King Lear
“O, beware, my lord, of jealousy;
It is the green-ey’d monster, which doth mock
The meat it feeds on.”
― William Shakespeare, Othello
“There was a star danced, and under that was I born.”
― William Shakespeare, Much Ado About Nothing

Want to see some AI/Shakespeare text?
To see you here before me. O my soul’s joy!
I am a man a worm.
– Gloucester
I know not, sweet:
I found your like a thief from the heart
That pirchas will be well.
The general speaks to be so
Turn a man, I think, besoou.
– Cassio
I pray you, sir, to lie: in this hand is not a tend and talking
of it;
I would not be threaten dispatch.
Our good old friend,
Lay comforts the state, seek for him;
I will grife you be faster’d!
And the blessed course of dower:
Net forth enough to do you;
And that the Moor is defective in the letter!
Abhorre, heaven, I will go sor;
And the other too.
– Othello
I have a seet me such a trial of his speech,
That he shall live the Moor in the lies
That with daush’er
Holds it is a most poor man,
Whose welchers of the state,
A man that many call of their life
That have to lost the boy look to’t.
– Regan
Sir, to my sister?
– Oswald
I pray you, have your hand: you may receive it all to his
sorrage, and makes the heavens
Cassio lies that in the heart
That I may speak: I’ll wast …
52
– Desdemona
I pray, talk me of Cassio.
Ay, so I hear the write to prive and there,
That she would seen him that present so lich wored
old wat,
and the best conscionable than in this revolumance
against him:
There’s son against father,
and thy father’s son and the best of our time hath no
exalse your counsel watch
The worst is not a tender to the warlike isle,
That sunded must needs take the shame
which the revenges of the self-same malk than the
best of our times;
keeps our fortunes fend for bearing to a strength,
Sight in their natures, letting go safely by the rack: I
swear ‘tis, to be so
That she will send back my messenger.
– Gloucester
I see the rust in the stocks.
– King Lear
What a trifore be some cartiou, I can tell my way
Than should be assurather, despise my brother;
That I have passed him, tell me I was every think of
fear,
That she may be honest yet he hath confess’d in him
entertains
and think the next way to stain it,
That the main commet in the least
Would fail her breath,
That she may …
-bick,
Remade me any thing to his sword
To his salt and most hidden loose to be so for sings,
but not in a libutt of his matter than that shall be sure
as will be soldye
As master compary, do not live in traitor.
Bless thy five wits!
-Kent
O pity!
Sir, where is the patience now,
That this is so far from the sea and some bidings to
dismantle
So many folds of save and honest.
-Brabantio
I must not think the Turk of Cassio in the strange
metting the cribles of a charmer be the reviling of libe
to say
That I can deceive him to the best advantage,
In her prophetic fairs of a little to presently at your
powers; whereof I thank you, sir.
-Albany
Gloucester, I will prove upancy of his sport and first
accuriors and guard and talking on the white.
-King Lear
Where are the thief?
Thou shalt never have the captains at the letter
To the Moor and thing we have not the better shall be
sure as worth if he be anger—
-Regan
I pray you, have a countend more than think to do a
proclaim’d there of my heart, Hot
and the best of our that she could else was not a
toman.
Good faith, hold, I beseech your grace,—
-King Lear
Then there’s so much to him.
-King Lear
Thou hast seen a part of this plainness I had thought to
see the ride on the sea sevel never second to take the
foul fiend.
Still through the way that shoulds: I know not what.
-Othello
What dost thou see her? O unhappy girl!
Sir, this desperate, bastard! what news?
Mistage on my father!
-King Lear
What say’st thou st?
-Kent
Sir, I do know you;
And to the Moor and rain, I beserve her take my
sisters?
-King Lear
No, no, no, no! where’t the castle.
Enter Gloucester, with King of France and Cordelia
Goneril
Sir, I had thought it not.
-Iago
Indeed!
-Othello
Indeed! ay, indeed: desceme, sir.
-Cornwall
Sighing, would!
-Othello
What dost thou see how this paper shall I carry out a
fellow there, that makes his son a poor poor power
That makes his son
When the rain came to the devil wrath!
I have lost the king from the cold wind:
Take them what the …
– Othello
O my fair warrior!
– Desdemona
My dear Othello!
– Othello
It gives me wonder great as my content
SCENE I.
Venice. A street.
Enter Roderigo and Iago
This License refers to version of the GNU General
Public License. Copyright also means copyright-
Smokin’ on cookie in the hotbox cookie
F*****’ on your b**** she a thot, thot, thot thot
Cookin fellow,

Conclusions

Can AI be creative?
Yo!
This post is about generating free text
with a deep learning network
particularly it is about Brick X6,
Phey, cabe,
make you feel soom the way (I smoke good!)
I probably make (What?)
More money in six months,
Than what's in your papa's safe (I'm serious)
Look like I robbed a bank (Okay Okay)
I set it off like Queen Latifah
'Cause I'm living single I'm feeling cautious
I ain't scream when they served a subpoena
(Can't go back to jail)
I heard that he a leader
(Who pood, what to be f*****' up
The baugerout Black alro Black X6,
Phantom White X6 looks like a panda
Goin' out like I'm Montana
Hundred killers, hundred hammers Black X6,
Phantom White X6, panda
Pockets swole, Danny
Sellin' bar, candy
Man I'm the macho like Randy
The choppa go Oscar for Grammy
B**** n**** pull up ya panty
Hope you killas understand me
Hey Panda, Panda Panda,
Panda, Panda, Panda, Panda
I got broads in Atlanta
Twistin' dope, lean, and the Fanta
Credit cards and the scammers
Hittin' off licks in the bando
54
This License refers to version of the GNU General Public License.
Copyright also means copyright-bick,
Remade me any thing to his sword
To his salt and most hidden loose to be so for sings, but not in a
libutt of his matter than that shall be sure as will be soldye
As master compary, do not live in traitor.
Bless thy five wits!
-Kent
O pity!
Sir, where is the patience now,
That this is so far from the sea and some bidings to dismantle
So many folds of save and honest.
-Brabantio
I must not think the Turk of Cassio in the strange metting the
cribles of a charmer be the reviling of libe to say
That I can deceive him to the best advantage,
In her prophetic fairs of a little to presently at your powers;
whereof I thank you, sir.
-Albany
Gloucester, I will prove upancy of his sport and first accuriors and
guard and talking on the white.
-King Lear
Where are the thief?
Thou shalt never have the captains at the letter
To the Moor and thing we have not the better shall be sure as
worth if he be anger—
-Regan
I pray you, have a countend more than think to do a proclaim’d
there of my heart, Hot
The words save, honest, thief, master, traitor, and deceive seem to
fit the context. Notice also that the dialogue sprouting from the
start text of the license agreement interestingly involves mainly
minor, less tragic characters from the plays.
Caro amico ti scrivo così mi distraggo un po'E
siccome sei molto lontano più forte ti scriverò.
Da quella prima folla strana, che aveva preso il suo
nome, e di correre alla casa di don Abbondio, con
un viso bene di non poterci andar la casa del padre
Cristoforo, e gli disse che s'avvicinava all'uscio, e si
mise a sparse di corsa, e di stare a sé, verso la
strada di servizio, chiesto le parole che gli andavan
dall'altra stanza, e con la sua condizione de'
cappuccini, e di consigli ricerche di confidenza
delle gride, nel suo passaggio, se non pensava con
una certa ripugnanza a casa sua, che andavano a
scomparire in un campo di buone ragioni che
avevan potuto raccogliere i suoi pensieri, e di
sopra non senza interrogare, che la sua avventura
aveva fatto predicare, e con la forza d'un fatto
come fuggitive che aveva preso il suo nome, e di
correre alla casa di don Abbondio, con un
cappuccino di quella sorte, con un certo sospiro,
alzando le sue finestre, e le diede un'occhiata in
carrozza. Si vendano a metter nelle mani di chi era
stato a sedere sur una strada così fatta con le
braccia in

References
• https://www.datanami.com/2019/04/04/product-naming-with-deep-learning/
• https://opendatascience.com/how-to-use-deep-learning-to-write-shakespeare/
• https://customerthink.com/ai-generated-rap-songs/
55

Free Book as a Thank You
Free Copy of e-book:
“Practicing Data Science.
A Collection of Case Studies”
from KNIME Press
https://www.knime.com/knimepress
with this code: AUSTIN-0120
56

The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by
KNIME AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.
The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by KNIME AG under license from KNIME GmbH,
and are registered in the United States. KNIME® is also registered in Germany.
Thank you!
Questions?

What is Transfer Learning?
• Transfer learning can be defined
as the attempt to utilize
predictive ability in one input /
output space to aid in the
learning of new input spaces,
output spaces, or both.
• In this use case the input space,
3x64x64, remains the same, but
the output space changes from
1000 image categories to 3
cancer types.
A
B
CNew

Transfer Learning – Basic Example
Input Data
Sepal Length
Sepal Width
Petal Length
Petal Width
Output
Prediction
Safe to Eat
Lots of data likely needed to tune
weights for logistic regression
Input Data
Sepal Length
Sepal Width
Petal Length
Petal Width
Output Prediction
Safe to Eat
Transferred
Learning
Type of Flower
Easy to train small decision
tree on little data
StandardLearningTransferLearning

Cancer Cell Classification Use Case
• Uses Keras Deep Learning
model.
• Learning Transferred from
VGG16 image classifier
• Completely code free with
KNIME’s Deep Learning
Integration
• Can be adapted to a wide range
of image classification problems
This Photo by Unknown
Author is licensed under
CC BY-SA
VGG16 Cat
VGG16 New CLL
Original Task
New Task
Image From:
https://ome.grc.nia.nih.g
ov/iicbu2008/lymphoma/
index.html

Why is it helpful?
• Deep Learning requires tons of
data
• Image classification requires
tons and tons of data
• We have ~5,000 labeled cancer
images
• VGG16 was trained on more
than 1,000,000 images from
ImageNet dataset. 5k
1,200k
CANCER CELLS IMAGENET
DATA POINTS

Feature Extraction in Neural Networks
• One of the benefits of
deep learning is its ability
to perform its own feature
engineering
• This can occur through
vector embedding,
convolutional layers,
LSTM layers etc.
Figure available under Creative Commons Attribution 4.0 International

Convolutional Layer
• Instead of connecting every
neuron to the new layer a
sliding window is used
• Use when your data has spatial
relationships
– 2D: Image
– 3D: Video
• Some convolutions may detect
edges or corners, while others
may detect cats, dogs, or street
signs inside an image Image from: https://towardsdatascience.com/a-comprehensive-guide-
to-convolutional-neural-networks-the-eli5-way-3bd2b1164a53

Convolutional Layer Cont.
• Stride:
– Distance between adjacent
windows (windows can overlap)
• Window/Kernel/Filter size:
– dimension(s) of windows
• Pooling:
– When the convolution maps
many to one. Commonly taking
the max or average value in a
window.
• Benefits:
– Fewer weights than dense layers
– Less prone to overfitting
– Feature extraction useful for
transfer learning
Image from: https://www.freecodecamp.org/news/an-intuitive-guide-
to-convolutional-neural-networks-260c2de0a050/

VGG16 Model
• Several blocks of small convolution layers followed by max pooling
layers
• Trained on ImageNet dataset
– Over 1,000,000 images from 1,000 thousand classes
• Won ILSVRC-2013 competition in single network category
Final Layer
We used
VGG16 Architecture: https://neurohive.io/en/popular-networks/vgg16/

Everything in KNIME
1. Download Data Set
2. Pre-Process Data Set
1. Crop, Normalize, Rearrange
3. Configure Network
1. Load VGG16
2. Add and Freeze Layers
4. Train Network
5. Post-Process Predictions
1. Group images, create Classification
Workflow on KNIME Hub:
https://kni.me/w/1_w_dxiltpHmsBQA

Pre-Processing
• Input size
– VGG16: 3x64x64 image
– Us: 1388x1040x3 image
• Image Values
– VGG16: Normalized 0-1
– Cancer Cells: 0-256
• Pre-Processing
– Multiple patches per
image
– Normalize (per image)
– Re-arrange dimensions
Image processing nodes part of KNIME Image Processing Extension:
https://kni.me/e/Uq6QE1IQIqG4q_mp
Create Patches Metanode

Configuring the Network
• Load VGG16 Network
• Add custom layers to produce 3 class predictions
• Freeze original VGG16 layers
– Keep feature detection intact
– Significantly reduces number of parameters to be learned
Keras nodes part of KNIME Deep Learning – Keras Integration:
https://kni.me/e/XOee1uZPrzE36EPH
# create the base pre-trained model
model = VGG16(weights='imagenet',
include_top=False, input_shape=(64, 64, 3))
# add Flatten, dense, and dropout layers
model.add(Flatten())
model.add(Dense(32), activation=‘ReLu’)
model.add(Dropout(0.5))
# add a fully-connected layer output layer
model.add(Dense(3, activation=‘Softmax’))
# first: train only the top layers
for layer in model.layers[0:-4]:
layer.trainable = False

Training the Network
• We train the newly
configured network for 25
epochs
• Only updating the
weights in our unfrozen
end layers
• Using Categorical Cross
Entropy Loss function
Learning Monitor, part of Keras Network Learner node

Post-Processing
• Combine image patches
– Average prediction by class
– Use max probability as
classification
• Scoring
– High Accuracy for small training
set
– High Cohen’s Kappa value shows
significant divergence from
random chance

Finding the Example

Related KNIME Blog Articles
• Original use case, involves
some coding:
– https://www.knime.com/blog/u
sing-the-new-knime-deep-
learning-keras-integration-to-
predict-cancer-type-from-
histopatholog
• Updated use case, totally code
free:
– https://www.knime.com/blog/tr
ansfer-learning-made-easy-with-
deep-learning-keras-integration

Citations
• Convolution Layer: https://towardsdatascience.com/a-
comprehensive-guide-to-convolutional-neural-networks-the-
eli5-way-3bd2b1164a53
• Cancer Cell Slide:
https://ome.grc.nia.nih.gov/iicbu2008/lymphoma/index.html
• VGG16 Architecture: https://neurohive.io/en/popular-
networks/vgg16/

KNIME Course Books
Course books downloadable from
KNIME Press
https://www.knime.com/knimepress
with code: AUSTIN-0120

The KNIME® trademark and logo and OPEN FOR INNOVATION® trademark are used by
KNIME AG under license from KNIME GmbH, and are registered in the United States.
KNIME® is also registered in Germany.
Thank you!
Questions?

Next Meetup
• Planning for April, then afterward June
• Next location: IBM campus on Burnet Road
• April Meetup topic ideas?
– Active Learning talk
– KNIME Use Cases talk
– Raw Data to Deployment Learnathon
– Time Series Learnathon
– ...or something else?
• Want to present on KNIME and your project? Let us know!

Codeless Deep Learning for Language Modeling and Image Classification

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Codeless Deep Learning for Language Modeling and Image Classification

Ähnlich wie Codeless Deep Learning for Language Modeling and Image Classification (20)

Mehr von KNIMESlides

Mehr von KNIMESlides (10)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Codeless Deep Learning for Language Modeling and Image Classification