In this talk, we describe DeepLocker, a novel class of highly targeted and evasive attacks powered by artificial intelligence (AI). As cybercriminals increasingly weaponize AI, cyber defenders must understand the mechanisms and implications of the malicious use of AI in order to stay ahead of these threats and deploy appropriate defenses. DeepLocker was developed as a proof of concept by IBM Research in order to understand how several AI and malware techniques already being seen in the wild could be combined to create a highly evasive new breed of malware, which conceals its malicious intent until it reached a specific victim. It achieves this by using a Deep Neural Network (DNN) AI-model to hide its attack payload in benign carrier applications, while the payload will only be unlocked if—and only if —the intended target is reached. DeepLocker leverages several attributes for target identification, including visual, audio, geolocation, and system-level features. In contrast to existing evasive and targeted malware, this method would make it extremely challenging to reverse engineer the benign carrier software and recover the mission-critical secrets, including the attack payload and the specifics of the target. We will perform a live demonstration of a proof-of-concept implementation of a DeepLocker malware, in which we camouflage well-known ransomware in a benign application such that it remains undetected by malware analysis tools, including anti-virus engines and malware sandboxes. We will discuss technical details, implications, and use cases of DeepLocker. More importantly, we will share countermeasures that could help defend against this type of attack in the wild.

1. DeepLocker
Concealing Targeted Attacks with AI Locksmithing
Dhilung Kirat, Jiyong Jang, Marc Ph. Stoecklin
IBM Research

2. 2
Cognitive Cyber Security Intelligence (CCSI)
IBM Research
Dhilung Kirat Jiyong Jang Marc Ph. Stoecklin

3. 3
AI-aided attacks
Profile a target
to increase success [2]
Craft an email to bypass filters [1]
Mutate malware to bypass AV [3, 4]
Execute machine-speed/creative attacks [5, 6]
AI
[1] S. Palka et al., “Fuzzing Email Filters with Generative Grammars and N-Gram Analysis”, Usenix WOOT 2015
[2] A. Singh and V. Thaware, “Wire Me through Machine Learning”, Black Hat USA 2017
[3] J. Jung et al., “AVPASS: Automatically Bypassing Android Malware Detection System”, Black Hat USA 2017
[4] H. Anderson, “Bot vs. Bot: Evading Machine Learning Malware Detection”, Black Hat USA 2017
[5] DARPA Cyber Grand Challenge (CGC), 2016
[6] D. Petro and B. Morris, “Weaponizing Machine Learning: Humanity was Overrated Anyway”, DEF CON 2017

4. 4
[1] S. Palka et al., “Fuzzing Email Filters with Generative Grammars and N-Gram Analysis”, Usenix WOOT 2015
[2] A. Singh and V. Thaware, “Wire Me through Machine Learning”, Black Hat USA 2017
[3] J. Jung et al., “AVPASS: Automatically Bypassing Android Malware Detection System”, Black Hat USA 2017
[4] H. Anderson, “Bot vs. Bot: Evading Machine Learning Malware Detection”, Black Hat USA 2017
[5] DARPA Cyber Grand Challenge (CGC), 2016
[6] D. Petro and B. Morris, “Weaponizing Machine Learning: Humanity was Overrated Anyway”, DEF CON 2017
AI-aided attacks
Profile a target
to increase success [2]
Craft an email to bypass filters [1]
Mutate malware to bypass AV [3, 4]
Execute machine-speed/creative attacks [5, 6]
AI
AI-embedded attack
AI
AI capability embedded inside malware itself

5. 5
Malware concealment – Locksmithing
Decryption
code
Encryption
Hide payload
⎯⎯⎯
Packers
Evasive malware
Avoid being analyzed
⎯⎯⎯
VM check
Processes check
Decryption
code
Checks for sandbox
Targeted attack
Disclose only at a target
⎯⎯⎯
Target attributes check
Decryption
code
Checks for target
Obfuscation
Mutate payload
⎯⎯⎯
Polymorphism
Metamorphism
1980 1990 2000 2010

6. 6
AI Locksmithing
© 2003 Warner Bros. Pictures All Rights Reserved

7. 7
Unleashing DeepLocker – AI Locksmithing

8. 8
DeepLocker – Overview
Existing
malware
Benign-looking
malware
Benign
application
Target
attributes
AI
Execution at target
Malicious behavior
AI
DeepLocker
Execution at non-target
Benign behavior
Distribution & execution
AI-powered concealment
Evading
static, dynamic, manual analysis
Security analysis

9. 9
DeepLocker Deep Dive

10. 10
Is this not a
sandbox?
YesNo
Start
Traditional targeted attack
End
Execute
Attack!
Is this a
target?
RegKeyExists(“HKLMSOFTWARESIEMENSSTEP7”)
Stuxnet

11. 11
Is this a
target?
AI-powered targeted attack
Start
End
Execute
Attack!
AI

12. 12
What is a Deep Neural Network (DNN)?
Output LayerInput Layer Hidden Layers
f ( wi xi
i=1
n
∑ )
w1
w2
wn
Sigmoid()
tanh()
ReLU()
x1
x2
xn
y

13. 13
Input Layer Output LayerHidden Layers
Deep Convolutional Neural Network
[1] Krizhevsky, Alex, et. al. "Imagenet classification with deep convolutional neural networks.” NIPS 2012.
AlexNet (2012) [1]
8 layers, 622K neurons, 60 million parameters

14. 14
Target
Target attributes
Audio, visual
User activity
Physical
environment
Geolocation
Sensors
Software
environment

15. 15
Target detection
Reference
Face
Extracted
Face
YesNo
Template matching requires a
template to match to.
Start
End
Execute
Attack!
DNN

16. 16
Derivation of an unlocking key
Start
End
Execute
Attack!
DeepLocker

17. 17
DeepLocker – AI-Powered Concealment and Unlocking
Concealed
payload
a2
a1
a3
Target attributes Secret keyTarget Concealment
encryption
Payload Concealment
Malicious
payload
Concealment
DeepLocker
Malicious
payload
a1
a3
a2
Concealed
payload Payload UnlockingRecovered keyInput attributes Target Detection
decryption
Unlocking

18. 18
AI-powered concealment
No decryption key
available in malware
sample to reverse
engineer!
DeepLocker

19. 19
Key generation
Face Detection
Model
Bucketization
High-dimensional
facial features
KeyTarget
0.9
0.0
0.8
0.1
0.9
1
0
1
0
1
Unstable key!

20. 20
Key generation
Face Detection
Model
Key Generation
Model
KeyTarget
0
1
1
0
1
Class 1
Class 2
Class 3
Class 4
Class 5
þ
ý
þ
ý
þ
0.9
0.0
0.8
0.1
0.9
High-dimensional
facial features

21. 21
Analysis of the key generation model
Target Detection
Dataset: Labeled Faces in the Wild (LFW)
http://vis-www.cs.umass.edu/lfw/

22. 22
DeepLocker – AI-powered concealment
1 Target Class Concealment
Does not reveal what it is looking for (e.g.,
faces, organization, or a completely obscure
object specific to the target environment)
2 Target Instance Concealment
If the target class is an individual, it
does not reveal who it is looking for
3 Malicious Intent Concealment
Payload is fully encrypted concealing
how the final attack is executed

23. 23
Attacking DeepLocker – AI Lock Picking

24. 24
Ways to counter
AI
Payload
execution
AI-powered
concealment
DeepLocker

25. 25
Ways to counter
AI
Payload
execution
AI-powered
concealment
Code attestation Block sensor access
AI usage monitoringHost-based monitoring
Brute-force key Brute-force attributes
Deceptive resources Deceptive attributes
Code analysis AI lock picking

26. 26
?

27. 27
Partial
occlusion
Occlude a portion of the image to
see how the embedding is
affected (deconvnet) [1]
Neural attention
model
Heatmap using the degree of
excitation of neurons in each
layer (excitation backprop) [2]
Debug neural
networks
Fuzzing for neural networks
(coverage-guided fuzzing) [3]
Reverse engineering AI models
[1] M. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” ECCV 2014
[2] J. Zhang et. al., “Top-down neural attention by excitation backprop,” ECCV 2016
[3] A. Odena and I. Goodfellow, “TensorFuzz: Debugging neural networks with coverage-guided fuzzing,” arXiv 2018

28. 28
Takeaways
DeepLocker is a demonstration of the potential of
AI-embedded attacks
Current defenses will become obsolete and
new defenses are needed
Rapid democratization of AI has made
AI-powered attacks an imminent threat

29. Thank you
Dhilung Kirat dkirat@us.ibm.com
Jiyong Jang jjang@us.ibm.com
Marc Ph. Stoecklin mpstoeck@us.ibm.com
IBM Research