Friday, October 15th, 2021, Sapporo, Hokkaido, Japan.
Hokkaido University ICReDD - Faculty of Medicine Joint Symposium
https://www.icredd.hokudai.ac.jp/event/5993
ICReDD (Institute for Chemical Reaction Design and Discovery)
https://www.icredd.hokudai.ac.jp
1. Machine Learning for Molecules
Ichigaku Takigawa
takigawa@icredd.hokudai.ac.jp
15 October 2021 @ Hokkaido University
Hokkaido Univ ICReDD-Faculty of Medicine Joint Symposium
2. / 23
2
RIKEN Center for AI Project @ Kyoto
Medical-risk Avoidance based on iPS Cells Team
(A joint lab with Kyoto Univ CiRA)
Inst. Chemical Reaction Design & Discovery
Hokkaido Univ
A machine learning (ML) researcher working for
ML for Stem Cell Biology ML for Chemistry
3. / 23
2
RIKEN Center for AI Project @ Kyoto
Medical-risk Avoidance based on iPS Cells Team
(A joint lab with Kyoto Univ CiRA)
Inst. Chemical Reaction Design & Discovery
Hokkaido Univ
A machine learning (ML) researcher working for
ML for Stem Cell Biology ML for Chemistry
Interests: Machine Learning and Machine Discovery
An intersection of ML with combinatorial structures + ML for natural sciences
4. / 23
2
RIKEN Center for AI Project @ Kyoto
Medical-risk Avoidance based on iPS Cells Team
(A joint lab with Kyoto Univ CiRA)
Inst. Chemical Reaction Design & Discovery
Hokkaido Univ
A machine learning (ML) researcher working for
ML for Stem Cell Biology ML for Chemistry
Interests: Machine Learning and Machine Discovery
An intersection of ML with combinatorial structures + ML for natural sciences
Joint project with HU Med Dept:
• Prof. Shinya Tanaka: Cancer diagnosis with fluorescent markers, Enzyme-catalyzed reaction design
• Prof. Shigetsugu Hatakeyama: Mediator complex of transcription regulations (Nat Commun 2020, 2015)
• Prof. Ichiro Yabe: Video-based predictions of motor symptom severity
• Prof. Yasuyuki Fujita: Cell competition (Cell Reports 2018; Sci Rep 2015)
• Prof. Hidenao Sasaki: Copy number variations for neurodegenerative diseases (Mol Brain 2017)
5. / 23
3
X-informatics: Bio- and Chemo-informatics
• Biochemical reaction networks (Metabolic pathways)
Bioinformatics 2007, 2008a, 2008b, 2009, 2010
Nucleic Acids Res 2011, PLoS One 2012, 2013, KDD’07
• Drug-target interactions (Polypharmacology)
PLoS One 2011, Drug Discov Today 2013, Brief Bioinform 2014
• Modulatory proteolysis
Mol Cell Proteom 2016, Genome Informatics 2009
(We also developed a database http://calpain.org)
• Genomic repeats
Discrete Appl Math 2013, 2016, AAAI 2020
• Genetic variations in cancer cells
Brief Bioinform 2014
• Mediator complex and transcription regulation
Nat Commun 2015, 2020 (w/ Prof. Hatakeyama)
• Genomic copy number variations for neurodegenerative diseases
Mol Brain 2017 (w/ Prof. Sasaki)
• Cell competitions and cancer cells
Cell Reports 2018, Sci Rep 2015 (w/ Prof. Fujita)
In addition to pure ML research, I’ve worked for bio/chemo-informatics
6. / 23
4
Molecules have a combinatorial aspect
https://cen.acs.org/physical-chemistry/computational-chemistry/Exploring-chemical-space-AI-take/98/i13
7. / 23
5
simulation
ML: A new way for (lazy) programming
computer program
input output
full specification in
every detail required
deductive (rationalism)
8. / 23
5
simulation
ML: A new way for (lazy) programming
computer program
input output
full specification in
every detail required
input output
computer program
ML
give up explicit model
instead, grab a tunable
model, and show it many
input-output instances
inductive (empiricism)
airhead with a god-like
learning capability
Random Forest Neural Networks SVR Kernel Ridge
All about fitting a very-flexible function to finite points in high-dimensional space.
deductive (rationalism)
9. / 23
6
ML: A new way for (lazy) programming
x1
x2
y
p1 p2 p3 p5
p4
All about statistical and algorithmic techniques for
surface-model fitting to data points by adjusting model parameters.
Variable 1
Variable 2
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
x1
x2
y
ML
ML = Tweak these parameter values to
best fit a surface model to the given points.
5 params
10. / 23
7
A modern aspect of ML
ResNet50: 26 million params
ResNet101: 45 million params
EfficientNet-B7: 66 million params
VGG19: 144 million params
12-layer, 12-heads BERT: 110 million params
24-layer, 16-heads BERT: 336 million params
GPT-2 XL: 1558 million params
GPT-3: 175 billion params
Modern ML: Can we imagine what would happen if we try to fit a function having 175
billion parameters to 100 million data points in 10 thousand dimension??
simulation
input output
computer program
input output
computer program
ML
full specification in
every detail required
give up explicit model
instead, grab a tunable
model, and show it many
input-output instances
inductive (empiricism)
airhead with a god-like
learning capability
deductive (rationalism)
All about fitting a very-flexible function to finite points in high-dimensional space.
11. / 23
8
This Talk: ML for Molecular Graphs
Latent
variables
Representation
learning
Reactions
Materials
Molecules
Graphs (of different size)
Node
features
Edge
features
CC1CCNO1
Graph Neural
Networks (GNNs)
NCc1ccoc1.S=(Cl)Cl>>[RX_5]S=C=NCc1ccoc1
…
Classifier or
Regressor
Diverse
Downstream
Tasks
Modular Hierarchy
Amide
Proline
Oxazoline
Compositionality
Phenyl
Carboxyl Methyl Ethyl Tert-butyl
Isoprophyl
Trifluoromethyl
Benzyl
Substituents
Graph
Coarsening
Combinatorial aspects
12. / 23
9
Representation Learning
Use some inductive biases to constrain/regularize the model space.
Prediction
Input
variables
Classifier or
Regressor
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Function
model
13. / 23
10
Representation Learning
Prediction
Input
variables
Function
model
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Latent
variables
Learnable variable
transformation
Representation learning
Classifier or
Regressor
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
Use some inductive biases to constrain/regularize the model space.
14. / 23
10
Representation Learning
Prediction
Input
variables
Function
model
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Latent
variables
Learnable variable
transformation
Representation learning
Classifier or
Regressor
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
Use some inductive biases to constrain/regularize the model space.
15. / 23
10
Representation Learning
Prediction
Input
variables
Function
model
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Latent
variables
Learnable variable
transformation
Representation learning
Classifier or
Regressor
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
Use some inductive biases to constrain/regularize the model space.
16. / 23
10
Representation Learning
Prediction
Input
variables
Function
model
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Latent
variables
Learnable variable
transformation
Representation learning
Classifier or
Regressor
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
Use some inductive biases to constrain/regularize the model space.
17. / 23
10
Representation Learning
Prediction
Input
variables
Function
model
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Latent
variables
Learnable variable
transformation
Representation learning
Classifier or
Regressor
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
Use some inductive biases to constrain/regularize the model space.
18. / 23
10
Representation Learning
Prediction
Input
variables
Function
model
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Latent
variables
Learnable variable
transformation
Representation learning
Classifier or
Regressor
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
Use some inductive biases to constrain/regularize the model space.
19. / 23
10
Representation Learning
Prediction
Input
variables
Function
model
<latexit sha1_base64="QFtMwnKe2I12XGZu0bNJbdnDaaE=">AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhAVIwV0caShzwSJGR3HXHDvrK7EJH4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlTutJqqhCIsxN8LDIO6BCLxIG6FHHOIIBmQ0oIFDh0NYhQibvjLiYDCJq6BNnEVIcfc5zhEgbYOyOGWIxNbpX6NV2WN1Wvdr2q5aplNUGhYpw4iyF3bPeuyZPbBX9vlnrbZbo++lRbM00HKzGrxYzn38q9JodnDyrRrp2cExtl2vCnk3XaZ/C3mgb551ermdbLS9xm7ZG/m/YV32RDfQm+/yXYZnr0f4kcgLvRg1KP67HcOgkIjFt2KJzEYkteu1yo8VrGKd+pFECvtII0/1a7jEFTqCX4gJm0JykCr4PM0SfoSQ+gJWLpCb</latexit>
x2
<latexit sha1_base64="lFhRrRrVTrFR31ebbMgRp5myJpc=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCIDPjCuiG5c8pBHgoS0dcCG0jZtISLxB0zcysKVJi6MH+AHuPEHXPAJxiUmblx4KU2MEvE20zlz5p47Z+ZKhqpYNmNdjzA2PjE55Z32zczOzfsDC4s5S2+YMs/KuqqbBUm0uKpoPGsrtsoLhsnFuqTyvFTb7+/nm9y0FF07tFsGL9XFqqZUFFm0icqcljfKgRCLMCeCwyDqghDcSOqBRxzhGDpkNFAHhwabsAoRFn1FRMFgEFdCmziTkOLsc5zDR9oGZXHKEImt0b9Kq6LLarTu17QctUynqDRMUgYRZi/snvXYM3tgr+zzz1ptp0bfS4tmaaDlRtl/sZz5+FdVp9nGybdqpGcbFew4XhXybjhM/xbyQN886/Qyu+lwe43dsjfyf8O67IluoDXf5bsUT1+P8CORF3oxalD0dzuGQS4WiW5HYqnNUGLPbZUXK1jFOvUjjgQOkESW6ldxiSt0BK8QEbaE+CBV8LiaJfwIIfEFWE6QnA==</latexit>
x3
<latexit sha1_base64="0IPXcU0UIDvzZlYURjV2A/THv9U=">AAACiXichVG7SgNBFD2ur/hM1EawEYNiFWZFNKQKprGMj0TBBNndTHR0X+xOFmLwB6zsRK0ULMQP8ANs/AELP0EsFWwsvNksiAbjXWbnzJl77pyZq7um8CVjz11Kd09vX39sYHBoeGQ0nhgbL/pOzTN4wXBMx9vWNZ+bwuYFKaTJt12Pa5Zu8i39MNfc3wq45wvH3pR1l5ctbc8WVWFokqhiKag40t9NJFmKhTHdDtQIJBFF3knco4QKHBiowQKHDUnYhAafvh2oYHCJK6NBnEdIhPscxxgkbY2yOGVoxB7Sf49WOxFr07pZ0w/VBp1i0vBIOY1Z9sRu2Rt7ZHfshX3+WasR1mh6qdOst7Tc3Y2fTG58/KuyaJbY/1Z19CxRRTr0Ksi7GzLNWxgtfXB09raRWZ9tzLFr9kr+r9gze6Ab2MG7cbPG1y87+NHJC70YNUj93Y52UFxIqUuphbXFZHYlalUMU5jBPPVjGVmsIo8C1T/AKc5xoQwpqpJWMq1UpSvSTOBHKLkvAi+SPA==</latexit>
.
.
.
Latent
variables
Learnable variable
transformation
Representation learning
Classifier or
Regressor
Linear
<latexit sha1_base64="Ill3Als4zZd947f5Xm9sW99d0QA=">AAAChnichVHLTsJAFD3UF+ID1I2JGyLBuCJTomJcEd245CGPBAlp64gNpW3aQkTiD5i4lYUrTVwYP8APcOMPuOATjEtM3LjwUpoYJeJtpnPmzD13zsyVTU21Hca6PmFsfGJyyj8dmJmdmw+GFhbzttGwFJ5TDM2wirJkc03Vec5RHY0XTYtLdVnjBbm2198vNLllq4Z+4LRMXq5LVV09VhXJISp7WhEroQiLMTfCw0D0QARepIzQIw5xBAMKGqiDQ4dDWIMEm74SRDCYxJXRJs4ipLr7HOcIkLZBWZwyJGJr9K/SquSxOq37NW1XrdApGg2LlGFE2Qu7Zz32zB7YK/v8s1bbrdH30qJZHmi5WQleLGc//lXVaXZw8q0a6dnBMbZdryp5N12mfwtloG+edXrZnUy0vcZu2Rv5v2Fd9kQ30Jvvyl2aZ65H+JHJC70YNUj83Y5hkI/HxK1YPL0RSe56rfJjBatYp34kkMQ+UshR/SoucYWO4BdiwqaQGKQKPk+zhB8hJL8AVA6Qmg==</latexit>
x1
Simple model is enough
when we have good features.
Use some inductive biases to constrain/regularize the model space.
20. / 23
11
Use Case 1: Virtual Screening (QSAR/QSPR)
• Mutagenic potency
• Carcinogenic potency
• Endocrine disruption
• Growth inhibition
• Aqueous solubility
N
NH
O
O
H
H
H
H H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
O
O
O
O
O
O
Cl
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
Br
Br O P
O
O Br
Br
O
Br
Br
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
N
S
N
N
H
H
H
H
H
H
H
H
H
H
H
H
H
H
H
O
N
O
O
H
H
H
O
O
H
H
N
O
O
Cl
Cl
Cl
H
H
H
H
H
H H
N
O
O
H
H
H
H
H
H
H H
H
N
O
O
H
H
H
H
H
H
H
N
H
N
O
O
N
O
O
H
H
H
H
H
H
H
H
N
CH3
O
O
H
N Cl
Cl
Cl
Cl
Cl
H3C
O O
O
O
O
O
H3C
CH3
CH2
O
HN
O
O
NH
CH3
HO
OH
CH3
N
O
O
CH3
N
N
H
N
H
H3C
N
H3C
H3C
NH
O
N
O
N
O
CH3
O N
NH2
O
CH3
Br
CH3
N
H3C
H
N
S
N
O
CH3
N
OH
CH3
CH3
N
N
N
CH3
H3C
H2N NH2
H
OH
O
HO
CH3
H
H
O
CH3
H
O
O
H3C H
H
H
O
H3C
S
CH3
O
H
H
O
CH3
CH3
O
O
HO
H3C
H
HO
F
H
O
H3C
NH2
O
N
HO
H
O
O
H
H
O
O
O
H3C
O
O
O
CH3
O
CH3
H
O
CH3
H
O
O
CH3
H
H
N
H
N O
H3C
O
O
O
22. / 23
13
Use Case 1: Virtual Screening (QSAR/QSPR)
input output
ML
activity: “Active”
LogGI50: -7.8811
CID 11978790
GI50: concentration required
for 50% inhibition of growth
23. / 23
14
Input representation (molecular graph)
Use Case 1: Virtual Screening (QSAR/QSPR)
1
2
1
3
explicit Hs
4
5
6
7
8
9
10
11
12
13
14
15
16
17
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Any permutation of
this numbering should
not change the results.
❗
CID 204
atoms → nodes
bonds → edges
1. permutation
equivariance
2. permutation
invariance
24. / 23
14
Input representation (molecular graph)
Use Case 1: Virtual Screening (QSAR/QSPR)
1
2
1
3
explicit Hs
4
5
6
7
8
9
10
11
12
13
14
15
16
17
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Any permutation of
this numbering should
not change the results.
❗
CID 204
• atomic_num (one-hot, 101)
• total_degree (one-hot, 7)
• formal_charge (one-hot, 6)
• chiral_tag (one-hot, 5)
• num_Hs (one-hot, 6)
• hybridization (one-hot, 6)
• is_aromatic (binary, 1)
• atomic_mass (real, 1)
17
edge(bond) features
• no_bond (binary, 1)
• is_single (binary, 1)
• is_double (binary, 1)
• is_triple (binary, 1)
• is_aromatic (binary, 1)
• is_connjugated (binary, 1)
• is_in_ring (binary, 1)
• stereo (one-hot, 7)
17
14
133
node(atom) features
133 features 14 features
e.g. Features for ChemProp (Yang et al, 2019)
atoms → nodes
bonds → edges
1. permutation
equivariance
2. permutation
invariance
25. / 23
14
Input representation (molecular graph)
Use Case 1: Virtual Screening (QSAR/QSPR)
1
2
1
3
explicit Hs
4
5
6
7
8
9
10
11
12
13
14
15
16
17
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
Any permutation of
this numbering should
not change the results.
❗
atom features bond features
topology
Molecular Graph
read out
graph-level
output
• sum, mean or max
• attentive pooling
CID 204
• atomic_num (one-hot, 101)
• total_degree (one-hot, 7)
• formal_charge (one-hot, 6)
• chiral_tag (one-hot, 5)
• num_Hs (one-hot, 6)
• hybridization (one-hot, 6)
• is_aromatic (binary, 1)
• atomic_mass (real, 1)
17
edge(bond) features
• no_bond (binary, 1)
• is_single (binary, 1)
• is_double (binary, 1)
• is_triple (binary, 1)
• is_aromatic (binary, 1)
• is_connjugated (binary, 1)
• is_in_ring (binary, 1)
• stereo (one-hot, 7)
17
14
133
node(atom) features
133 features 14 features
e.g. Features for ChemProp (Yang et al, 2019)
atoms → nodes
bonds → edges
1. permutation
equivariance
2. permutation
invariance
26. / 23
15
Graph Neural Networks (GNNs)
N O
C
C
C
C
H
H
H
H
H
N O
C
C
C
C
H
H
H
H
H
GNN Layer
GNN updates
features
27. / 23
15
Graph Neural Networks (GNNs)
N O
C
C
C
C
H
H
H
H
H
N O
C
C
C
C
H
H
H
H
H
GNN Layer
GNN updates
features
28. / 23
15
Graph Neural Networks (GNNs)
N O
C
C
C
C
H
H
H
H
H
N O
C
C
C
C
H
H
H
H
H
GNN Layer
GNN updates
features
<latexit sha1_base64="+I1ZH8a510AHL/VRK05INyOdDHc=">AAACi3ichVHLSsNAFL2Nr1qtrboR3BRLxVWZaFEpLooiuOzDPqAtJYnTdmheJGmhhv6ASzcu6kbBhfgBfoAbf8BFP0FcVnDjwps0IFqsN0zmzJl77pyZK+oyMy1CBj5uanpmds4/H1hYDC6FwssrBVNrGxLNS5qsGSVRMKnMVJq3mCXTkm5QQRFlWhRbR85+sUMNk2nqqdXVaVURGiqrM0mwkCpVRMVu9mqsFo6SOHEjMg54D0TBi7QWfoQKnIEGErRBAQoqWIhlEMDErww8ENCRq4KNnIGIufsUehBAbRuzKGYIyLbw38BV2WNVXDs1TVct4SkyDgOVEYiRF3JPhuSZPJBX8vlnLdut4Xjp4iyOtFSvhS7Wch//qhScLWh+qyZ6tqAO+65Xht51l3FuIY30nfOrYS6Zjdmb5Ja8of8bMiBPeAO18y7dZWi2P8GPiF7wxbBB/O92jIPCdpzfjScyiWjq0GuVH9ZhA7awH3uQghNIQ97twyX04ZoLcjtckjsYpXI+T7MKP4I7/gL6JJMZ</latexit>
hi
<latexit sha1_base64="iVE2zwSrY7uMFp5lULN8y4s7ZXg=">AAACjnichVHLSsNAFL2Nr1ofjboR3BRLxVWZSKkiiEU3XfZhH9CWksSpjs2LJC3U0B9wLy4ERcGF+AF+gBt/wEU/QVxWcOPCmzQgWqw3TObMmXvunJkrGQqzbEJ6AW5sfGJyKjgdmpmdmw/zC4tFS2+ZMi3IuqKbZUm0qMI0WrCZrdCyYVJRlRRakpr77n6pTU2L6dqB3TFoTRWPNNZgsmgjValKqkO7dYeddOt8lMSJF5FhIPggCn5kdP4RqnAIOsjQAhUoaGAjVkAEC78KCEDAQK4GDnImIubtU+hCCLUtzKKYISLbxP8Rrio+q+HarWl5ahlPUXCYqIxAjLyQe9Inz+SBvJLPP2s5Xg3XSwdnaaClRj18tpz/+Fel4mzD8bdqpGcbGrDleWXo3fAY9xbyQN8+vejnt3MxZ43ckjf0f0N65AlvoLXf5bsszV2O8COhF3wxbJDwux3DoLgRF5LxRDYRTe35rQrCCqzCOvZjE1KQhgwUvBc9hyu45nguye1wu4NULuBrluBHcOkvYEmUlg==</latexit>
eij
atom features
bond features
29. / 23
15
Graph Neural Networks (GNNs)
N O
C
C
C
C
H
H
H
H
H
N O
C
C
C
C
H
H
H
H
H
GNN Layer
GNN updates
features
<latexit sha1_base64="rQGx6XMOvsCjXRp7b9gbmKjgm1M=">AAAC/HichVHNTttAEB4b2tI0lNBeKnGxiKiChKJNhVrUU1QuPaHwE0DCkbU2m3jD+kfrTVBquQ/AC3DgVNQeEOIKD9BLX4ADBx4AcUylXnroxHEVtah0LHu//Wa+8bc7dih4pAi50vSx8QcPH008zj3JTz6dKkw/24yCjnRY3QlEILdtGjHBfVZXXAm2HUpGPVuwLXtveZDf6jIZ8cDfUL2QNTza8nmTO1QhZRX2TduL3cTihilYU1Epg33DDKNsX/qdXsiZNm8FoehEVtw2TO4bpkeV61ARr2A+QZHLjVG9kaH2ELHEink7mc+ZkrdcNW8ViqRM0jDugkoGipBFLShcgAm7EIADHfCAgQ8KsQAKET47UAECIXINiJGTiHiaZ5BADrUdrGJYQZHdw28LdzsZ6+N+0DNK1Q7+ReArUWnAHLkkJ6RPvpFTckN+/rNXnPYYeOnhag+1LLSmDl6s//ivysNVgTtS3etZQROWUq8cvYcpMziFM9R3Pxz219+uzcUvyTG5Rf+fyBX5iifwu9+dL6ts7egePzZ6wRvDAVX+HsddsPmqXHldXlxdLFbfZaOagBmYhRLO4w1U4T3UoI79r7UxLa9N6h/1z/qpfjYs1bVM8xz+CP38F+WDvZo=</latexit>
hi
0
@hi,
M
j2Ni
(hi, hj, eij)
1
A
Update by “Message Passing”
<latexit sha1_base64="+I1ZH8a510AHL/VRK05INyOdDHc=">AAACi3ichVHLSsNAFL2Nr1qtrboR3BRLxVWZaFEpLooiuOzDPqAtJYnTdmheJGmhhv6ASzcu6kbBhfgBfoAbf8BFP0FcVnDjwps0IFqsN0zmzJl77pyZK+oyMy1CBj5uanpmds4/H1hYDC6FwssrBVNrGxLNS5qsGSVRMKnMVJq3mCXTkm5QQRFlWhRbR85+sUMNk2nqqdXVaVURGiqrM0mwkCpVRMVu9mqsFo6SOHEjMg54D0TBi7QWfoQKnIEGErRBAQoqWIhlEMDErww8ENCRq4KNnIGIufsUehBAbRuzKGYIyLbw38BV2WNVXDs1TVct4SkyDgOVEYiRF3JPhuSZPJBX8vlnLdut4Xjp4iyOtFSvhS7Wch//qhScLWh+qyZ6tqAO+65Xht51l3FuIY30nfOrYS6Zjdmb5Ja8of8bMiBPeAO18y7dZWi2P8GPiF7wxbBB/O92jIPCdpzfjScyiWjq0GuVH9ZhA7awH3uQghNIQ97twyX04ZoLcjtckjsYpXI+T7MKP4I7/gL6JJMZ</latexit>
hi
<latexit sha1_base64="iVE2zwSrY7uMFp5lULN8y4s7ZXg=">AAACjnichVHLSsNAFL2Nr1ofjboR3BRLxVWZSKkiiEU3XfZhH9CWksSpjs2LJC3U0B9wLy4ERcGF+AF+gBt/wEU/QVxWcOPCmzQgWqw3TObMmXvunJkrGQqzbEJ6AW5sfGJyKjgdmpmdmw/zC4tFS2+ZMi3IuqKbZUm0qMI0WrCZrdCyYVJRlRRakpr77n6pTU2L6dqB3TFoTRWPNNZgsmgjValKqkO7dYeddOt8lMSJF5FhIPggCn5kdP4RqnAIOsjQAhUoaGAjVkAEC78KCEDAQK4GDnImIubtU+hCCLUtzKKYISLbxP8Rrio+q+HarWl5ahlPUXCYqIxAjLyQe9Inz+SBvJLPP2s5Xg3XSwdnaaClRj18tpz/+Fel4mzD8bdqpGcbGrDleWXo3fAY9xbyQN8+vejnt3MxZ43ckjf0f0N65AlvoLXf5bsszV2O8COhF3wxbJDwux3DoLgRF5LxRDYRTe35rQrCCqzCOvZjE1KQhgwUvBc9hyu45nguye1wu4NULuBrluBHcOkvYEmUlg==</latexit>
eij
atom features
bond features
30. / 23
15
Graph Neural Networks (GNNs)
N O
C
C
C
C
H
H
H
H
H
N O
C
C
C
C
H
H
H
H
H
GNN Layer
GNN updates
features
Message
Permutation
equivariant
operations
• nn.Linear
Bond features can be
used (typically in )
<latexit sha1_base64="rQGx6XMOvsCjXRp7b9gbmKjgm1M=">AAAC/HichVHNTttAEB4b2tI0lNBeKnGxiKiChKJNhVrUU1QuPaHwE0DCkbU2m3jD+kfrTVBquQ/AC3DgVNQeEOIKD9BLX4ADBx4AcUylXnroxHEVtah0LHu//Wa+8bc7dih4pAi50vSx8QcPH008zj3JTz6dKkw/24yCjnRY3QlEILdtGjHBfVZXXAm2HUpGPVuwLXtveZDf6jIZ8cDfUL2QNTza8nmTO1QhZRX2TduL3cTihilYU1Epg33DDKNsX/qdXsiZNm8FoehEVtw2TO4bpkeV61ARr2A+QZHLjVG9kaH2ELHEink7mc+ZkrdcNW8ViqRM0jDugkoGipBFLShcgAm7EIADHfCAgQ8KsQAKET47UAECIXINiJGTiHiaZ5BADrUdrGJYQZHdw28LdzsZ6+N+0DNK1Q7+ReArUWnAHLkkJ6RPvpFTckN+/rNXnPYYeOnhag+1LLSmDl6s//ivysNVgTtS3etZQROWUq8cvYcpMziFM9R3Pxz219+uzcUvyTG5Rf+fyBX5iifwu9+dL6ts7egePzZ6wRvDAVX+HsddsPmqXHldXlxdLFbfZaOagBmYhRLO4w1U4T3UoI79r7UxLa9N6h/1z/qpfjYs1bVM8xz+CP38F+WDvZo=</latexit>
hi
0
@hi,
M
j2Ni
(hi, hj, eij)
1
A
Update by “Message Passing”
<latexit sha1_base64="+I1ZH8a510AHL/VRK05INyOdDHc=">AAACi3ichVHLSsNAFL2Nr1qtrboR3BRLxVWZaFEpLooiuOzDPqAtJYnTdmheJGmhhv6ASzcu6kbBhfgBfoAbf8BFP0FcVnDjwps0IFqsN0zmzJl77pyZK+oyMy1CBj5uanpmds4/H1hYDC6FwssrBVNrGxLNS5qsGSVRMKnMVJq3mCXTkm5QQRFlWhRbR85+sUMNk2nqqdXVaVURGiqrM0mwkCpVRMVu9mqsFo6SOHEjMg54D0TBi7QWfoQKnIEGErRBAQoqWIhlEMDErww8ENCRq4KNnIGIufsUehBAbRuzKGYIyLbw38BV2WNVXDs1TVct4SkyDgOVEYiRF3JPhuSZPJBX8vlnLdut4Xjp4iyOtFSvhS7Wch//qhScLWh+qyZ6tqAO+65Xht51l3FuIY30nfOrYS6Zjdmb5Ja8of8bMiBPeAO18y7dZWi2P8GPiF7wxbBB/O92jIPCdpzfjScyiWjq0GuVH9ZhA7awH3uQghNIQ97twyX04ZoLcjtckjsYpXI+T7MKP4I7/gL6JJMZ</latexit>
hi
<latexit sha1_base64="iVE2zwSrY7uMFp5lULN8y4s7ZXg=">AAACjnichVHLSsNAFL2Nr1ofjboR3BRLxVWZSKkiiEU3XfZhH9CWksSpjs2LJC3U0B9wLy4ERcGF+AF+gBt/wEU/QVxWcOPCmzQgWqw3TObMmXvunJkrGQqzbEJ6AW5sfGJyKjgdmpmdmw/zC4tFS2+ZMi3IuqKbZUm0qMI0WrCZrdCyYVJRlRRakpr77n6pTU2L6dqB3TFoTRWPNNZgsmgjValKqkO7dYeddOt8lMSJF5FhIPggCn5kdP4RqnAIOsjQAhUoaGAjVkAEC78KCEDAQK4GDnImIubtU+hCCLUtzKKYISLbxP8Rrio+q+HarWl5ahlPUXCYqIxAjLyQe9Inz+SBvJLPP2s5Xg3XSwdnaaClRj18tpz/+Fel4mzD8bdqpGcbGrDleWXo3fAY9xbyQN8+vejnt3MxZ43ckjf0f0N65AlvoLXf5bsszV2O8COhF3wxbJDwux3DoLgRF5LxRDYRTe35rQrCCqzCOvZjE1KQhgwUvBc9hyu45nguye1wu4NULuBrluBHcOkvYEmUlg==</latexit>
eij
atom features
bond features
31. / 23
15
Graph Neural Networks (GNNs)
N O
C
C
C
C
H
H
H
H
H
N O
C
C
C
C
H
H
H
H
H
GNN Layer
GNN updates
features
• sum, mean or max
• attentive pooling
Aggregate
Permutation
invariant
operations
Message
Permutation
equivariant
operations
• nn.Linear
Bond features can be
used (typically in )
<latexit sha1_base64="rQGx6XMOvsCjXRp7b9gbmKjgm1M=">AAAC/HichVHNTttAEB4b2tI0lNBeKnGxiKiChKJNhVrUU1QuPaHwE0DCkbU2m3jD+kfrTVBquQ/AC3DgVNQeEOIKD9BLX4ADBx4AcUylXnroxHEVtah0LHu//Wa+8bc7dih4pAi50vSx8QcPH008zj3JTz6dKkw/24yCjnRY3QlEILdtGjHBfVZXXAm2HUpGPVuwLXtveZDf6jIZ8cDfUL2QNTza8nmTO1QhZRX2TduL3cTihilYU1Epg33DDKNsX/qdXsiZNm8FoehEVtw2TO4bpkeV61ARr2A+QZHLjVG9kaH2ELHEink7mc+ZkrdcNW8ViqRM0jDugkoGipBFLShcgAm7EIADHfCAgQ8KsQAKET47UAECIXINiJGTiHiaZ5BADrUdrGJYQZHdw28LdzsZ6+N+0DNK1Q7+ReArUWnAHLkkJ6RPvpFTckN+/rNXnPYYeOnhag+1LLSmDl6s//ivysNVgTtS3etZQROWUq8cvYcpMziFM9R3Pxz219+uzcUvyTG5Rf+fyBX5iifwu9+dL6ts7egePzZ6wRvDAVX+HsddsPmqXHldXlxdLFbfZaOagBmYhRLO4w1U4T3UoI79r7UxLa9N6h/1z/qpfjYs1bVM8xz+CP38F+WDvZo=</latexit>
hi
0
@hi,
M
j2Ni
(hi, hj, eij)
1
A
Update by “Message Passing”
<latexit sha1_base64="+I1ZH8a510AHL/VRK05INyOdDHc=">AAACi3ichVHLSsNAFL2Nr1qtrboR3BRLxVWZaFEpLooiuOzDPqAtJYnTdmheJGmhhv6ASzcu6kbBhfgBfoAbf8BFP0FcVnDjwps0IFqsN0zmzJl77pyZK+oyMy1CBj5uanpmds4/H1hYDC6FwssrBVNrGxLNS5qsGSVRMKnMVJq3mCXTkm5QQRFlWhRbR85+sUMNk2nqqdXVaVURGiqrM0mwkCpVRMVu9mqsFo6SOHEjMg54D0TBi7QWfoQKnIEGErRBAQoqWIhlEMDErww8ENCRq4KNnIGIufsUehBAbRuzKGYIyLbw38BV2WNVXDs1TVct4SkyDgOVEYiRF3JPhuSZPJBX8vlnLdut4Xjp4iyOtFSvhS7Wch//qhScLWh+qyZ6tqAO+65Xht51l3FuIY30nfOrYS6Zjdmb5Ja8of8bMiBPeAO18y7dZWi2P8GPiF7wxbBB/O92jIPCdpzfjScyiWjq0GuVH9ZhA7awH3uQghNIQ97twyX04ZoLcjtckjsYpXI+T7MKP4I7/gL6JJMZ</latexit>
hi
<latexit sha1_base64="iVE2zwSrY7uMFp5lULN8y4s7ZXg=">AAACjnichVHLSsNAFL2Nr1ofjboR3BRLxVWZSKkiiEU3XfZhH9CWksSpjs2LJC3U0B9wLy4ERcGF+AF+gBt/wEU/QVxWcOPCmzQgWqw3TObMmXvunJkrGQqzbEJ6AW5sfGJyKjgdmpmdmw/zC4tFS2+ZMi3IuqKbZUm0qMI0WrCZrdCyYVJRlRRakpr77n6pTU2L6dqB3TFoTRWPNNZgsmgjValKqkO7dYeddOt8lMSJF5FhIPggCn5kdP4RqnAIOsjQAhUoaGAjVkAEC78KCEDAQK4GDnImIubtU+hCCLUtzKKYISLbxP8Rrio+q+HarWl5ahlPUXCYqIxAjLyQe9Inz+SBvJLPP2s5Xg3XSwdnaaClRj18tpz/+Fel4mzD8bdqpGcbGrDleWXo3fAY9xbyQN8+vejnt3MxZ43ckjf0f0N65AlvoLXf5bsszV2O8COhF3wxbJDwux3DoLgRF5LxRDYRTe35rQrCCqzCOvZjE1KQhgwUvBc9hyu45nguye1wu4NULuBrluBHcOkvYEmUlg==</latexit>
eij
atom features
bond features
32. / 23
15
Graph Neural Networks (GNNs)
N O
C
C
C
C
H
H
H
H
H
N O
C
C
C
C
H
H
H
H
H
GNN Layer
GNN updates
features
• sum, mean or max
• attentive pooling
Aggregate
Permutation
invariant
operations
Update
Any
• nn.Linear
Message
Permutation
equivariant
operations
• nn.Linear
Bond features can be
used (typically in )
<latexit sha1_base64="rQGx6XMOvsCjXRp7b9gbmKjgm1M=">AAAC/HichVHNTttAEB4b2tI0lNBeKnGxiKiChKJNhVrUU1QuPaHwE0DCkbU2m3jD+kfrTVBquQ/AC3DgVNQeEOIKD9BLX4ADBx4AcUylXnroxHEVtah0LHu//Wa+8bc7dih4pAi50vSx8QcPH008zj3JTz6dKkw/24yCjnRY3QlEILdtGjHBfVZXXAm2HUpGPVuwLXtveZDf6jIZ8cDfUL2QNTza8nmTO1QhZRX2TduL3cTihilYU1Epg33DDKNsX/qdXsiZNm8FoehEVtw2TO4bpkeV61ARr2A+QZHLjVG9kaH2ELHEink7mc+ZkrdcNW8ViqRM0jDugkoGipBFLShcgAm7EIADHfCAgQ8KsQAKET47UAECIXINiJGTiHiaZ5BADrUdrGJYQZHdw28LdzsZ6+N+0DNK1Q7+ReArUWnAHLkkJ6RPvpFTckN+/rNXnPYYeOnhag+1LLSmDl6s//ivysNVgTtS3etZQROWUq8cvYcpMziFM9R3Pxz219+uzcUvyTG5Rf+fyBX5iifwu9+dL6ts7egePzZ6wRvDAVX+HsddsPmqXHldXlxdLFbfZaOagBmYhRLO4w1U4T3UoI79r7UxLa9N6h/1z/qpfjYs1bVM8xz+CP38F+WDvZo=</latexit>
hi
0
@hi,
M
j2Ni
(hi, hj, eij)
1
A
Update by “Message Passing”
<latexit sha1_base64="+I1ZH8a510AHL/VRK05INyOdDHc=">AAACi3ichVHLSsNAFL2Nr1qtrboR3BRLxVWZaFEpLooiuOzDPqAtJYnTdmheJGmhhv6ASzcu6kbBhfgBfoAbf8BFP0FcVnDjwps0IFqsN0zmzJl77pyZK+oyMy1CBj5uanpmds4/H1hYDC6FwssrBVNrGxLNS5qsGSVRMKnMVJq3mCXTkm5QQRFlWhRbR85+sUMNk2nqqdXVaVURGiqrM0mwkCpVRMVu9mqsFo6SOHEjMg54D0TBi7QWfoQKnIEGErRBAQoqWIhlEMDErww8ENCRq4KNnIGIufsUehBAbRuzKGYIyLbw38BV2WNVXDs1TVct4SkyDgOVEYiRF3JPhuSZPJBX8vlnLdut4Xjp4iyOtFSvhS7Wch//qhScLWh+qyZ6tqAO+65Xht51l3FuIY30nfOrYS6Zjdmb5Ja8of8bMiBPeAO18y7dZWi2P8GPiF7wxbBB/O92jIPCdpzfjScyiWjq0GuVH9ZhA7awH3uQghNIQ97twyX04ZoLcjtckjsYpXI+T7MKP4I7/gL6JJMZ</latexit>
hi
<latexit sha1_base64="iVE2zwSrY7uMFp5lULN8y4s7ZXg=">AAACjnichVHLSsNAFL2Nr1ofjboR3BRLxVWZSKkiiEU3XfZhH9CWksSpjs2LJC3U0B9wLy4ERcGF+AF+gBt/wEU/QVxWcOPCmzQgWqw3TObMmXvunJkrGQqzbEJ6AW5sfGJyKjgdmpmdmw/zC4tFS2+ZMi3IuqKbZUm0qMI0WrCZrdCyYVJRlRRakpr77n6pTU2L6dqB3TFoTRWPNNZgsmgjValKqkO7dYeddOt8lMSJF5FhIPggCn5kdP4RqnAIOsjQAhUoaGAjVkAEC78KCEDAQK4GDnImIubtU+hCCLUtzKKYISLbxP8Rrio+q+HarWl5ahlPUXCYqIxAjLyQe9Inz+SBvJLPP2s5Xg3XSwdnaaClRj18tpz/+Fel4mzD8bdqpGcbGrDleWXo3fAY9xbyQN8+vejnt3MxZ43ckjf0f0N65AlvoLXf5bsszV2O8COhF3wxbJDwux3DoLgRF5LxRDYRTe35rQrCCqzCOvZjE1KQhgwUvBc9hyu45nguye1wu4NULuBrluBHcOkvYEmUlg==</latexit>
eij
atom features
bond features
33. / 23
16
ChemProp
(Directed MPNN)
Use Case 1: Virtual Screening (QSAR/QSPR)
ExtraTrees
w/ ECFP6(1024)
Performance for unseen (test) data:
Standard ML GNN
Stokes et al, Cell (2020) https://doi.org/10.1016/j.cell.2020.01.021
Marchant, Nature (2020) https://doi.org/10.1038/d41586-020-00018-3
ChemProp (Yang et al, 2019)
from MIT MLPDS (Machine Learning
for Pharmaceutical Discovery
and Synthesis) Consortium
Disclaimer: This is just for a toy demo. This should be taken
as classification for ACTIVITY_OUTCOME (Active or Inactive)
95.079% (Active/Inactive) 95.604% (Active/Inactive)
• Regression for LogGI50 • Regression for LogGI50
• Classification accuracy • Classification accuracy
RMSE 0.6076
RMSE 0.7970
Activie/Inactive (Classification), LogGI50 (Regression)
35. / 23
18
Use Case 2: Quantum chemistry
input output
gdb_21014
1000 sec
Density Functional Theory (DFT)
B3LYP/6-31G(2df, p)
<latexit sha1_base64="JI//afsBt1AdIhSgVUbVSGVXtww=">AAACmnichVHLSsNAFD2Nr1pfVREEXQSL4qpMpagIgiiC4qZVq4KVksSxHZomIZkWavAH/AEXrhRcqB/gB7jxB1z0E8SlghsX3qYBUVFvmMyZM/fcOTNXd0zhScYaEaWtvaOzK9od6+nt6x+IDw7teHbVNXjOsE3b3dM1j5vC4jkppMn3HJdrFd3ku3p5pbm/W+OuJ2xrW9YdflDRipY4EoYmiSrER/IlTfprJ2o+4wl1UV0NQCGeYEkWhPoTpEKQQBgZO36HPA5hw0AVFXBYkIRNaPDo20cKDA5xB/CJcwmJYJ/jBDHSVimLU4ZGbJn+RVrth6xF62ZNL1AbdIpJwyWlikn2yK7ZC3tgt+yJvf9ayw9qNL3UadZbWu4UBk5Ht97+VVVolih9qv70LHGE+cCrIO9OwDRvYbT0teOzl62FzUl/il2yZ/J/wRrsnm5g1V6NqyzfPP/Dj05e6MWoQanv7fgJdmaSqdlkOptOLC2HrYpiDBOYpn7MYQlryCBH9X1c4Aa3yriyrKwrG61UJRJqhvEllO0PT6aXZA==</latexit>
Ĥ = E
by solving a one-electron Schrödinger
equation (Kohn–Sham equation)
Quantum chemical
calculations
36. / 23
18
Use Case 2: Quantum chemistry
input output
gdb_21014
1000 sec
Density Functional Theory (DFT)
B3LYP/6-31G(2df, p)
<latexit sha1_base64="JI//afsBt1AdIhSgVUbVSGVXtww=">AAACmnichVHLSsNAFD2Nr1pfVREEXQSL4qpMpagIgiiC4qZVq4KVksSxHZomIZkWavAH/AEXrhRcqB/gB7jxB1z0E8SlghsX3qYBUVFvmMyZM/fcOTNXd0zhScYaEaWtvaOzK9od6+nt6x+IDw7teHbVNXjOsE3b3dM1j5vC4jkppMn3HJdrFd3ku3p5pbm/W+OuJ2xrW9YdflDRipY4EoYmiSrER/IlTfprJ2o+4wl1UV0NQCGeYEkWhPoTpEKQQBgZO36HPA5hw0AVFXBYkIRNaPDo20cKDA5xB/CJcwmJYJ/jBDHSVimLU4ZGbJn+RVrth6xF62ZNL1AbdIpJwyWlikn2yK7ZC3tgt+yJvf9ayw9qNL3UadZbWu4UBk5Ht97+VVVolih9qv70LHGE+cCrIO9OwDRvYbT0teOzl62FzUl/il2yZ/J/wRrsnm5g1V6NqyzfPP/Dj05e6MWoQanv7fgJdmaSqdlkOptOLC2HrYpiDBOYpn7MYQlryCBH9X1c4Aa3yriyrKwrG61UJRJqhvEllO0PT6aXZA==</latexit>
Ĥ = E
by solving a one-electron Schrödinger
equation (Kohn–Sham equation)
ML
0.01 sec
≈
100,000 times faster!
Quantum chemical
calculations
37. / 23
19
input molecule H2O
Use Case 2: Quantum chemistry
gdb_3
0
1 2
graph (SchNet)
0 1
atom features
0 1 2
2
edges w/ cutoff (10Å)
0
bond features
0 1
1
0 2
2
1 2
edge_index
0.9620 0.9622 1.5133
latexit sha1_base64=WhwbZjIo+PbaHz7kU/YZBO38PiQ=AAACp3ichVG7SgNBFD2u72eiNoLNYlC0MEwkqAhC0MbO+EgMGFl211En2Re7m4Cu6cUfsLBSsBDBVnsbf8DCTxBLBRsLbzYLoqLeZXbOnLnnzpm5mmMIz2fssUlqbmlta+/o7Oru6e2LxfsH8p5dcXWe023Ddgua6nFDWDznC9/gBcflqqkZfEMrL9b3N6rc9YRtrfv7Dt8y1V1L7Ahd9YlS4iOuEohSTZ6bl4uHRc0M3Joi5Ek5giVilXiCJVkY8k+QikACUWTt+A2K2IYNHRWY4LDgEzagwqNvEykwOMRtISDOJSTCfY4aukhboSxOGSqxZfrv0mozYi1a12t6oVqnUwwaLilljLIHdsle2D27Yk/s/ddaQVij7mWfZq2h5Y4SOx5ae/tXZdLsY+9T9adnHzuYDb0K8u6ETP0WekNfPTh5WZtbHQ3G2Dl7Jv9n7JHd0Q2s6qt+scJXT//wo5EXejFqUOp7O36C/FQyNZ1Mr6QTmYWoVR0YxgjGqR8zyGAJWeSo/hGucYNbaUJalvJSoZEqNUWaQXwJSf0AKaGdRg==/latexit
rij := kri rjk
latexit sha1_base64=kMmroSd0z/qeaFfqTkrtDbgltFg=AAACinichVHLSsNAFL2Nr1qrrboR3ARLxVW5LUWrIhR14bIP+8BaShKnNTRNQpIWavEH3LkS7ErBhfgBfoAbf8BFP0FcVnDjwps0IFqsN0zmzJl77pyZK+qKbFqIPQ83Nj4xOeWd9s34Z+cCwfmFvKk1DYnlJE3RjKIomEyRVZazZEthRd1gQkNUWEGs79n7hRYzTFlTD622zsoNoabKVVkSLKIKRxXkd/hEJRjCCDrBD4OoC0LgRkoLPsIxnIAGEjShAQxUsAgrIIBJXwmigKATV4YOcQYh2dlncA4+0jYpi1GGQGyd/jValVxWpbVd03TUEp2i0DBIyUMYX/Ae+/iMD/iKn3/W6jg1bC9tmsWBlumVwMVS9uNfVYNmC06/VSM9W1CFhONVJu+6w9i3kAb61tlVP7uVCXdW8RbfyP8N9vCJbqC23qW7NMt0R/gRyQu9GDUo+rsdwyAfi0TXI/F0PJTcdVvlhWVYgTXqxwYk4QBSkHPqX8I1dDk/F+M2ue1BKudxNYvwI7j9Ly4OkVo=/latexit
Z0 = 8
latexit sha1_base64=QxdA1PVsyCLNE23CpTjm2APprYs=AAACinichVHLSsNAFL2Nr1qrrboR3ARLxVWZlOIToagLl33YB1YpSZzWoWkSkmmhFn/AnSvBrhRciB/gB7jxB1z0E8RlBTcuvEkDosV6w2TOnLnnzpm5iqkxmxPS9Qkjo2PjE/7JwFRweiYUnp3L20bDUmlONTTDKiqyTTWm0xxnXKNF06JyXdFoQantOvuFJrVsZugHvGXS47pc1VmFqTJHqnBYlsRtUSqHIyRG3BAHgeSBCHiRMsKPcAQnYIAKDagDBR04Yg1ksPErgQQETOSOoY2chYi5+xTOIYDaBmZRzJCRreG/iquSx+q4dmrarlrFUzQcFipFiJIXck965Jk8kFfy+WettlvD8dLCWelrqVkOXSxkP/5V1XHmcPqtGuqZQwXWXa8MvZsu49xC7eubZ1e97GYm2l4mt+QN/d+QLnnCG+jNd/UuTTOdIX4U9IIvhg2SfrdjEOTjMWk1lkgnIskdr1V+WIQlWMF+rEES9iEFObf+JVxDRwgKcWFD2OqnCj5PMw8/Qtj7AiFSkVQ=/latexit
Z1 = 1
latexit sha1_base64=PJMbQqlxF/BaTgtbHwfwEQN8SrI=AAACinichVHLSsNAFL2Nr1qrrboR3ARLxVWZlOIToagLl33YB1YpSZzWoWkSkmmhFn/AnSvBrhRciB/gB7jxB1z0E8RlBTcuvEkDosV6w2TOnLnnzpm5iqkxmxPS9Qkjo2PjE/7JwFRweiYUnp3L20bDUmlONTTDKiqyTTWm0xxnXKNF06JyXdFoQantOvuFJrVsZugHvGXS47pc1VmFqTJHqnBYjovbolQOR0iMuCEOAskDEfAiZYQf4QhOwAAVGlAHCjpwxBrIYONXAgkImMgdQxs5CxFz9ymcQwC1DcyimCEjW8N/FVclj9Vx7dS0XbWKp2g4LFSKECUv5J70yDN5IK/k889abbeG46WFs9LXUrMculjIfvyrquPM4fRbNdQzhwqsu14ZejddxrmF2tc3z6562c1MtL1Mbskb+r8hXfKEN9Cb7+pdmmY6Q/wo6AVfDBsk/W7HIMjHY9JqLJFORJI7Xqv8sAhLsIL9WIMk7EMKcm79S7iGjhAU4sKGsNVPFXyeZh5+hLD3BSN2kVU=/latexit
Z2 = 1
latexit sha1_base64=A/aHUcWCUQde6WVY9Df1h30Gjpk=AAACqXichVE9T9tQFD2Ylo/wkQALEotFSoUERDdtSFokpKgsjCQQiEiiyDYPsPCXbCdSiPgD/AEGJpAYqi7dKlhZ+gc65CegjlTqwsC1Y6kqiHAt+5533j3X572rOobu+USdPqn/zduBwaHh2Mjo2Hg8MTG57dkNVxMlzTZst6wqnjB0S5R83TdE2XGFYqqG2FGP1oL9naZwPd22tvyWI2qmcmDp+7qm+EzVE++qqtl2T+okr8qVJUrRx0xmUabU51xuOchEuWytnkgGKAj5OUhHIIkoNuzED1SxBxsaGjAhYMFnbECBx08FaRAc5mpoM+cy0sN9gRPEWNvgKsEVCrNH/D3gVSViLV4HPb1QrfFfDH5dVsqYo1/0le7pJ32jO3p4sVc77BF4aXFWu1rh1OOn05t/X1WZnH0c/lP19OxjH59Crzp7d0ImOIXW1TePz+43V4pz7fd0Sb/Z/wV16JZPYDX/aFcFUTzv4UdlL3xjPKD003E8B9sfUulsKlPIJPNfolENYQazmOd55JDHOjZQ4v6n+I5r3EgLUkEqS7vdUqkv0kzhv5C0R6Sdmbs=/latexit
r0 = [ 0.0344, 0.9775, 0.0076]
latexit sha1_base64=2hhG9gKLK2JIb9w0aH6ObiinRms=AAACpnichVHLSsNQED3GV3226kZwUy2KCykTqbUIgujGlfiqFWqpSbxqMC+StKDFteAPuHCl4ELErX6AG3/AhZ8gLhXcuHCSBkRFnZDMuefOmZx7R3UM3fOJHhukxqbmltZYW3tHZ1d3PNHTu+bZFVcTec02bHddVTxh6JbI+7pviHXHFYqpGqKg7s0F+4WqcD3dtlb9fUeUTGXH0rd1TfGZKicGN1Sz5h6W5eR0skhpymZyY5zGKRskkidK5UQqQEEkfwI5AilEsWgnbrGBLdjQUIEJAQs+YwMKPH6KkEFwmCuhxpzLSA/3BQ7RztoKVwmuUJjd4+8Or4oRa/E66OmFao3/YvDrsjKJYXqgS3qhe7qiJ3r/tVct7BF42ees1rXCKceP+1fe/lWZnH3sfqr+9OxjG7nQq87enZAJTqHV9dWDk5eVqeXh2gid0zP7P6NHuuMTWNVX7WJJLJ/+4UdlL3xjPCD5+zh+grXxtJxNZ5YyqZnZaFQxDGAIozyPScxgHovIc/8jXOMGt9KotCDlpUK9VGqINH34EtLmBwvLmR0=/latexit
r1 = [0.0648, 0.0206, 0.0015]
latexit sha1_base64=FqaYo7zlBtUvxlEmhgyAfxy/v8k=AAACpnichVHLSsNAFD3G97NVN4KbalFcSLjRYosgiG5cia9aoZaaxFGDeZGkBS2uBX/AhSsFFyJu7Qe48Qdc+AniUsGNC2/TgKioN0zmzJl77pyZq7mm4QdEjw1SY1NzS2tbe0dnV3dPLN7bt+47JU8XWd0xHW9DU31hGrbIBkZgig3XE6qlmSKn7c/X9nNl4fmGY68FB64oWOqubewYuhowVYwPbWpWxTsqTiRmEnmSM2klM67Ik0SZcZKJKF0oxpMh4kj8BEoEkohiyYlXsYltONBRggUBGwFjEyp8/vJQQHCZK6DCnMfICPcFjtDB2hJnCc5Qmd3n/y6v8hFr87pW0w/VOp9i8vBYmcAIPdAVvdA9XdMTvf9aqxLWqHk54Fmra4VbjJ0MrL79q7J4DrD3qfrTc4AdZEKvBnt3Q6Z2C72uLx+evqxOr4xURumCntn/OT3SHd/ALr/ql8ti5ewPPxp74RfjBinf2/ETrE/IypScWk4lZ+eiVrVhEMMY436kMYsFLCHL9Y9xg1tUpTFpUcpKuXqq1BBp+vElpK0PJf+ZKQ==/latexit
r2 = [0.8718, 1.3008, 0.0007]
SchNet (Schütt et al, 2017)
latexit sha1_base64=tzvRXqv2dAYIROlG6uwbJPf0mrw=AAAC9XichVHNShxBEK6d+BeNuiaXgJchi7IiLL0iieQk5pKT+Lcq2DL0jL27vfZMDz29a3SYF8gL5BByiKgQcvDoA3jxBRLwEYJHBS8erJ0dEqKoNcz011/VV/N1lxtKERlCznPWs67unt6+5/0DLwaHhvMjL1cj1dQer3hKKr3usohLEfCKEUby9VBz5ruSr7nbH9r5tRbXkVDBitkN+abPaoGoCo8ZpJx8SF0//pQ4wqaSVw3TWu3Yf7lJm4aRSDNFGjV9J27YVAQ29Zmpe0zG81iVYFFdFDNRY8KmakuZtAlVPq+xxIlFI6Fa1OpmwskXSImkYd8H5QwUIIsFlT8BClugwIMm+MAhAINYAoMInw0oA4EQuU2IkdOIRJrnkEA/aptYxbGCIbuN3xruNjI2wH27Z5SqPfyLxFej0oYx8ov8IJfkjPwkf8jNg73itEfbyy6ubkfLQ2f48+vl6ydVPq4G6v9Uj3o2UIWZ1KtA72HKtE/hdfStvS+Xy++XxuJxsk8u0P93ck5O8QRB68o7XORLXx/x46IXvDEcUPnuOO6D1alS+W1penG6MDuXjaoPRuENFHEe72AWPsICVLD/b7jJded6rB3rm3VgHXVKrVymeQX/hXV8C560vMU=/latexit
xi xi +
0
@
X
j2Ni
(xj) !ij
1
A
Message Passing with
residual connections
latexit sha1_base64=4oqceeOsg0RegmHmCOLOjG0SaOU=AAACi3ichVHNSgJRFD5Of2aZVpugjSRGKzmWVEgLKYKW/uQPqMjMdLXB+WNmlGzwBVq2aWGbghbRA/QAbXqBFj5CtDRo06LjOBAl2Rnu3O9+93znfvceQZcl00LsebiJyanpGe+sb27evxAILi7lTa1piCwnarJmFAXeZLKkspwlWTIr6gbjFUFmBaFxMNgvtJhhSpp6bLV1VlH4uirVJJG3iCqWBcU+61SxGgxjFJ0IjYKYC8LgRkoLPkIZTkADEZqgAAMVLMIy8GDSV4IYIOjEVcAmziAkOfsMOuAjbZOyGGXwxDboX6dVyWVVWg9qmo5apFNkGgYpQxDBF7zHPj7jA77i55+1bKfGwEubZmGoZXo1cLGS/fhXpdBswem3aqxnC2qw63iVyLvuMINbiEN96/yqn01kIvY63uIb+b/BHj7RDdTWu3iXZpnuGD8CeaEXowbFfrdjFOQ3o7HtaDwdDyf33VZ5YRXWYIP6sQNJOIIU5Jw+XEIXrjk/t8UluL1hKudxNcvwI7jDL6M0kvA=/latexit
x0
latexit sha1_base64=jqQfQ7TB6F1UPH31OFoS1eerzNw=AAACi3ichVHLSsNAFL2Nr1qtrboR3BRLxVWZaFEpLooiuOzDPqAtJYnTOjQvkrRYQ3/ApRsXdaPgQvwAP8CNP+CinyAuK7hx4U0aEC3WGyZz5sw9d87MFXWZmRYhfR83MTk1PeOfDczNBxdC4cWlgqm1DInmJU3WjJIomFRmKs1bzJJpSTeooIgyLYrNA2e/2KaGyTT12OrotKoIDZXVmSRYSJUqomKfdWt8LRwlceJGZBTwHoiCF2kt/AgVOAENJGiBAhRUsBDLIICJXxl4IKAjVwUbOQMRc/cpdCGA2hZmUcwQkG3iv4GrssequHZqmq5awlNkHAYqIxAjL+SeDMgzeSCv5PPPWrZbw/HSwVkcaqleC12s5D7+VSk4W3D6rRrr2YI67LpeGXrXXca5hTTUt8+vBrlkNmavk1vyhv5vSJ884Q3U9rt0l6HZ3hg/InrBF8MG8b/bMQoKm3F+O57IJKKpfa9VfliFNdjAfuxACo4gDXm3D5fQg2suyG1xSW5vmMr5PM0y/Aju8AulVJLx/latexit
x1
latexit sha1_base64=fLMd1ywDpT0cBkzaL9hTb77jq/8=AAACi3ichVHLSsNAFL2Nr1qtrboR3BRLxVWZ1KJSXBRFcNmHfUBbShLHGpoXybRYQ3/ApRsXdaPgQvwAP8CNP+CinyAuK7hx4U0aEC3WGyZz5sw9d87MFQ1FthghfR83MTk1PeOfDczNBxdC4cWloqW3TIkWJF3RzbIoWFSRNVpgMlNo2TCpoIoKLYnNfWe/1KamJevaEesYtKYKDU0+kSWBIVWuiqp91q0n6uEoiRM3IqOA90AUvMjo4UeowjHoIEELVKCgAUOsgAAWfhXggYCBXA1s5ExEsrtPoQsB1LYwi2KGgGwT/w1cVTxWw7VT03LVEp6i4DBRGYEYeSH3ZECeyQN5JZ9/1rLdGo6XDs7iUEuNeuhiJf/xr0rFmcHpt2qsZwYnsON6ldG74TLOLaShvn1+NcincjF7ndySN/R/Q/rkCW+gtd+luyzN9cb4EdELvhg2iP/djlFQTMT5rXgym4ym97xW+WEV1mAD+7ENaTiEDBTcPlxCD665ILfJpbjdYSrn8zTL8CO4gy+ndJLy/latexit
x2
latexit sha1_base64=YVJW2X9s34FCmpunsQgR00z8vXY=AAACjnichVHLSsNAFL2Nr1ofrboR3BRLxVW5kVJFEItuuuzDPqAtJYlTDc2LJK3U0B9wLy4ERcGF+AF+gBt/wEU/QVxWcOPC2zQgWqw3TObMmXvunJkrGops2YhdHzc2PjE55Z8OzMzOzQdDC4sFS2+aEstLuqKbJVGwmCJrLG/LtsJKhskEVVRYUWzs9/eLLWZasq4d2G2DVVXhSJPrsiTYRJUrouqcdGoO8p1aKIIxdCM8DHgPRMCLtB56hAocgg4SNEEFBhrYhBUQwKKvDDwgGMRVwSHOJCS7+ww6ECBtk7IYZQjENuh/RKuyx2q07te0XLVEpyg0TFKGIYoveI89fMYHfMXPP2s5bo2+lzbN4kDLjFrwbDn38a9KpdmG42/VSM821GHL9SqTd8Nl+reQBvrW6UUvt52NOmt4i2/k/wa7+EQ30Frv0l2GZS9H+BHJC70YNYj/3Y5hUNiI8YlYPBOPJPe8VvlhBVZhnfqxCUlIQRry7ouewxVccyEuwe1wu4NUzudpluBHcKkvk/uUNg==/latexit
w01
latexit sha1_base64=tYR7XnwyhyUSVK/6XFo1uz/tDtI=AAACjnichVFLSwJRFD5OL7OHVpugjSRGKzmKWASR1Malj3yAisxMVxucFzOjYYN/oH20CIqCFtEP6Ae06Q+08CdES4M2LTqOA1GSneHO/e53z3fud+8RdFkyLcSeh5uYnJqe8c765uYXFv2BpeWCqbUMkeVFTdaMksCbTJZUlrckS2Yl3WC8IsisKDQPBvvFNjNMSVMPrY7OqgrfUKW6JPIWUeWKoNgn3ZqNsW4tEMIIOhEcBVEXhMCNtBZ4hAocgQYitEABBipYhGXgwaSvDFFA0Imrgk2cQUhy9hl0wUfaFmUxyuCJbdK/Qauyy6q0HtQ0HbVIp8g0DFIGIYwveI99fMYHfMXPP2vZTo2Blw7NwlDL9Jr/bDX38a9KodmC42/VWM8W1GHb8SqRd91hBrcQh/r26UU/t5MN2xt4i2/k/wZ7+EQ3UNvv4l2GZS/H+BHIC70YNSj6ux2joBCLRBOReCYeSu67rfLCGqzDJvVjC5KQgjTknRc9hyu45gJcgtvl9oapnMfVrMCP4FJflhyUNw==/latexit
w02
latexit sha1_base64=Ak3wjfcp94hUo7DAHdEjDzlDdm0=AAACjnichVHLSsNAFL2Nr1ofjboR3BRLxVWZlFJFEItuuuzDPqAtJYnTOjQvkrRSQ3/AvbgQFAUX4gf4AW78ARf9BHFZwY0Lb9OAaLHeMJkzZ+65c2auZCjMsgnp+biJyanpGf9sYG5+YTHILy0XLL1lyjQv64puliTRogrTaN5mtkJLhklFVVJoUWoeDPaLbWpaTNcO7Y5Bq6rY0FidyaKNVLkiqc5Jt+YIsW6ND5MocSM0CgQPhMGLtM4/QgWOQAcZWqACBQ1sxAqIYOFXBgEIGMhVwUHORMTcfQpdCKC2hVkUM0Rkm/hv4KrssRquBzUtVy3jKQoOE5UhiJAXck/65Jk8kFfy+Wctx60x8NLBWRpqqVELnq3mPv5VqTjbcPytGuvZhjpsu14ZejdcZnALeahvn170czvZiLNBbskb+r8hPfKEN9Da7/JdhmYvx/iR0Au+GDZI+N2OUVCIRYVENJ6Jh5P7Xqv8sAbrsIn92IIkpCANefdFz+EKrjmeS3C73N4wlfN5mhX4EVzqC5g+lDg=/latexit
w12
40. / 23
20
pred vs true for SchNet (Schütt et al, 2017)
Use Case 2: Quantum chemistry
pred vs true for DimeNet (Klicpera et al, 2020)
Dipole Moment Energy U
HOMO
LUMO
Heat Capacity
Enthalpy H
Dipole Moment Energy U
HOMO
LUMO
Heat Capacity
Enthalpy H
41. / 23
21
SchNet (Schütt et al, 2017)
Use Case 2: Quantum chemistry
DimeNet (Klicpera et al, 2020)
Free Energy Free Energy
y_true
y_true
y_pred y_pred
ExtraTrees w/ ECFP6 LightGBM w/ ECFP6 3-Layer MLP w/ ECFP6
(without 3D geometry) (without 3D geometry) (without 3D geometry)
Free Energy Free Energy Free Energy
46. / 23
Potential Energy Surface
latexit sha1_base64=Wyp4gB8RxRsto5OIPkBWRdHaY3U=AAAChHichVG7SgNBFD1ZNcZ31EawCQbFQsJE4wMLCYpgaRIThSiyu07i4GZ32Z0EYvAHtFUsrBQsxA/wA2z8AYt8glhGsLHwZrMgGtS7zM6ZM/fcOTNXsw3hSsbqAaWjsyvYHerp7esfGBwKD4/kXKvs6DyrW4bl7Giqyw1h8qwU0uA7tsPVkmbwbe1orbm/XeGOKyxzS1ZtvldSi6YoCF2VRKXW98NRFmNeRNpB3AdR+LFphR+wiwNY0FFGCRwmJGEDKlz68oiDwSZuDzXiHELC2+c4QS9py5TFKUMl9oj+RVrlfdakdbOm66l1OsWg4ZAygkn2zO5Ygz2xe/bCPn6tVfNqNL1UadZaWm7vD52OZd7/VZVoljj8Uv3pWaKAJc+rIO+2xzRvobf0lePLRmY5PVmbYjfslfxfszp7pBuYlTf9NsXTV3/40cgLvRg1KP6zHe0gNxuLL8QSqUQ0ueq3KoRxTGCa+rGIJDawiSzV5zjDOS6UoDKjzCnzrVQl4GtG8S2UlU9p44/F/latexit
E
latexit sha1_base64=VQOOvVSP0lW6m0gNVQ8zA5pNsEc=AAAChnichVG7SgNBFD1ZXzE+ErURbIJBsQo3Eh9YBW0sTWJMQCXsrmMcsi92NwEN/oBgq4WVgoX4AX6AjT9gkU8QSwUbC282C6Ki3mV2zpy5586ZuZpjSM8nakeUnt6+/oHoYGxoeGQ0nhgb3/LshquLkm4btlvRVE8Y0hIlX/qGqDiuUE3NEGWtvtbZLzeF60nb2vQPHbFrqjVL7ktd9ZkqulVZTaQoTUEkf4JMCFIIY8NO3GEHe7ChowETAhZ8xgZUePxtIwOCw9wuWsy5jGSwL3CMGGsbnCU4Q2W2zv8ar7ZD1uJ1p6YXqHU+xeDhsjKJGXqkG3qhB7qlJ3r/tVYrqNHxcsiz1tUKpxo/mSy+/asyefZx8Kn607OPfSwHXiV7dwKmcwu9q28enb8UVwozrVm6omf2f0ltuucbWM1X/TovChd/+NHYC78YNyjzvR0/wdZ8OrOYzuazqdxq2KoopjCNOe7HEnJYxwZKXL+GU5zhXIkqaWVBWeqmKpFQM4EvoeQ+AL7mkM4=/latexit
ri
latexit sha1_base64=kTQzD4iiA9afGMFuEz1PlDAJN4Y=AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhgUIwV0caShzwSJGR3HXFlX9ldSJD4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlbOqp9VQhMWYG+FhEPdABF6kjdAjDnEEAzIa0MChwyGsQoRNXxlxMJjEVdAmziKkuPsc5wiQtkFZnDJEYuv0r9Gq7LE6rfs1bVct0ykqDYuUYUTZC7tnPfbMHtgr+/yzVtut0ffSolkaaLlZDV4s5z7+VWk0Ozj5Vo307OAY265XhbybLtO/hTzQN886vdxONtpeY7fsjfzfsC57ohvozXf5LsOz1yP8SOSFXowaFP/djmFQ2IjFt2KJTCKS2vVa5ccKVrFO/UgihX2kkaf6NVziCh3BL8SETSE5SBV8nmYJP0JIfQHBBpDP/latexit
rj
22
Chemical Reaction Design and Discovery
EQ1 EQ2
Chemical Reaction
47. / 23
Potential Energy Surface
latexit sha1_base64=Wyp4gB8RxRsto5OIPkBWRdHaY3U=AAAChHichVG7SgNBFD1ZNcZ31EawCQbFQsJE4wMLCYpgaRIThSiyu07i4GZ32Z0EYvAHtFUsrBQsxA/wA2z8AYt8glhGsLHwZrMgGtS7zM6ZM/fcOTNXsw3hSsbqAaWjsyvYHerp7esfGBwKD4/kXKvs6DyrW4bl7Giqyw1h8qwU0uA7tsPVkmbwbe1orbm/XeGOKyxzS1ZtvldSi6YoCF2VRKXW98NRFmNeRNpB3AdR+LFphR+wiwNY0FFGCRwmJGEDKlz68oiDwSZuDzXiHELC2+c4QS9py5TFKUMl9oj+RVrlfdakdbOm66l1OsWg4ZAygkn2zO5Ygz2xe/bCPn6tVfNqNL1UadZaWm7vD52OZd7/VZVoljj8Uv3pWaKAJc+rIO+2xzRvobf0lePLRmY5PVmbYjfslfxfszp7pBuYlTf9NsXTV3/40cgLvRg1KP6zHe0gNxuLL8QSqUQ0ueq3KoRxTGCa+rGIJDawiSzV5zjDOS6UoDKjzCnzrVQl4GtG8S2UlU9p44/F/latexit
E
latexit sha1_base64=VQOOvVSP0lW6m0gNVQ8zA5pNsEc=AAAChnichVG7SgNBFD1ZXzE+ErURbIJBsQo3Eh9YBW0sTWJMQCXsrmMcsi92NwEN/oBgq4WVgoX4AX6AjT9gkU8QSwUbC282C6Ki3mV2zpy5586ZuZpjSM8nakeUnt6+/oHoYGxoeGQ0nhgb3/LshquLkm4btlvRVE8Y0hIlX/qGqDiuUE3NEGWtvtbZLzeF60nb2vQPHbFrqjVL7ktd9ZkqulVZTaQoTUEkf4JMCFIIY8NO3GEHe7ChowETAhZ8xgZUePxtIwOCw9wuWsy5jGSwL3CMGGsbnCU4Q2W2zv8ar7ZD1uJ1p6YXqHU+xeDhsjKJGXqkG3qhB7qlJ3r/tVYrqNHxcsiz1tUKpxo/mSy+/asyefZx8Kn607OPfSwHXiV7dwKmcwu9q28enb8UVwozrVm6omf2f0ltuucbWM1X/TovChd/+NHYC78YNyjzvR0/wdZ8OrOYzuazqdxq2KoopjCNOe7HEnJYxwZKXL+GU5zhXIkqaWVBWeqmKpFQM4EvoeQ+AL7mkM4=/latexit
ri
latexit sha1_base64=kTQzD4iiA9afGMFuEz1PlDAJN4Y=AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhgUIwV0caShzwSJGR3HXFlX9ldSJD4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlbOqp9VQhMWYG+FhEPdABF6kjdAjDnEEAzIa0MChwyGsQoRNXxlxMJjEVdAmziKkuPsc5wiQtkFZnDJEYuv0r9Gq7LE6rfs1bVct0ykqDYuUYUTZC7tnPfbMHtgr+/yzVtut0ffSolkaaLlZDV4s5z7+VWk0Ozj5Vo307OAY265XhbybLtO/hTzQN886vdxONtpeY7fsjfzfsC57ohvozXf5LsOz1yP8SOSFXowaFP/djmFQ2IjFt2KJTCKS2vVa5ccKVrFO/UgihX2kkaf6NVziCh3BL8SETSE5SBV8nmYJP0JIfQHBBpDP/latexit
rj
22
Chemical Reaction Design and Discovery
EQ1 EQ2
Chemical Reaction
48. / 23
Potential Energy Surface
latexit sha1_base64=Wyp4gB8RxRsto5OIPkBWRdHaY3U=AAAChHichVG7SgNBFD1ZNcZ31EawCQbFQsJE4wMLCYpgaRIThSiyu07i4GZ32Z0EYvAHtFUsrBQsxA/wA2z8AYt8glhGsLHwZrMgGtS7zM6ZM/fcOTNXsw3hSsbqAaWjsyvYHerp7esfGBwKD4/kXKvs6DyrW4bl7Giqyw1h8qwU0uA7tsPVkmbwbe1orbm/XeGOKyxzS1ZtvldSi6YoCF2VRKXW98NRFmNeRNpB3AdR+LFphR+wiwNY0FFGCRwmJGEDKlz68oiDwSZuDzXiHELC2+c4QS9py5TFKUMl9oj+RVrlfdakdbOm66l1OsWg4ZAygkn2zO5Ygz2xe/bCPn6tVfNqNL1UadZaWm7vD52OZd7/VZVoljj8Uv3pWaKAJc+rIO+2xzRvobf0lePLRmY5PVmbYjfslfxfszp7pBuYlTf9NsXTV3/40cgLvRg1KP6zHe0gNxuLL8QSqUQ0ueq3KoRxTGCa+rGIJDawiSzV5zjDOS6UoDKjzCnzrVQl4GtG8S2UlU9p44/F/latexit
E
latexit sha1_base64=VQOOvVSP0lW6m0gNVQ8zA5pNsEc=AAAChnichVG7SgNBFD1ZXzE+ErURbIJBsQo3Eh9YBW0sTWJMQCXsrmMcsi92NwEN/oBgq4WVgoX4AX6AjT9gkU8QSwUbC282C6Ki3mV2zpy5586ZuZpjSM8nakeUnt6+/oHoYGxoeGQ0nhgb3/LshquLkm4btlvRVE8Y0hIlX/qGqDiuUE3NEGWtvtbZLzeF60nb2vQPHbFrqjVL7ktd9ZkqulVZTaQoTUEkf4JMCFIIY8NO3GEHe7ChowETAhZ8xgZUePxtIwOCw9wuWsy5jGSwL3CMGGsbnCU4Q2W2zv8ar7ZD1uJ1p6YXqHU+xeDhsjKJGXqkG3qhB7qlJ3r/tVYrqNHxcsiz1tUKpxo/mSy+/asyefZx8Kn607OPfSwHXiV7dwKmcwu9q28enb8UVwozrVm6omf2f0ltuucbWM1X/TovChd/+NHYC78YNyjzvR0/wdZ8OrOYzuazqdxq2KoopjCNOe7HEnJYxwZKXL+GU5zhXIkqaWVBWeqmKpFQM4EvoeQ+AL7mkM4=/latexit
ri
latexit sha1_base64=kTQzD4iiA9afGMFuEz1PlDAJN4Y=AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhgUIwV0caShzwSJGR3HXFlX9ldSJD4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlbOqp9VQhMWYG+FhEPdABF6kjdAjDnEEAzIa0MChwyGsQoRNXxlxMJjEVdAmziKkuPsc5wiQtkFZnDJEYuv0r9Gq7LE6rfs1bVct0ykqDYuUYUTZC7tnPfbMHtgr+/yzVtut0ffSolkaaLlZDV4s5z7+VWk0Ozj5Vo307OAY265XhbybLtO/hTzQN886vdxONtpeY7fsjfzfsC57ohvozXf5LsOz1yP8SOSFXowaFP/djmFQ2IjFt2KJTCKS2vVa5ccKVrFO/UgihX2kkaf6NVziCh3BL8SETSE5SBV8nmYJP0JIfQHBBpDP/latexit
rj
22
Chemical Reaction Design and Discovery
EQ1 EQ2
TS
Chemical Reaction
49. / 23
Potential Energy Surface
latexit sha1_base64=Wyp4gB8RxRsto5OIPkBWRdHaY3U=AAAChHichVG7SgNBFD1ZNcZ31EawCQbFQsJE4wMLCYpgaRIThSiyu07i4GZ32Z0EYvAHtFUsrBQsxA/wA2z8AYt8glhGsLHwZrMgGtS7zM6ZM/fcOTNXsw3hSsbqAaWjsyvYHerp7esfGBwKD4/kXKvs6DyrW4bl7Giqyw1h8qwU0uA7tsPVkmbwbe1orbm/XeGOKyxzS1ZtvldSi6YoCF2VRKXW98NRFmNeRNpB3AdR+LFphR+wiwNY0FFGCRwmJGEDKlz68oiDwSZuDzXiHELC2+c4QS9py5TFKUMl9oj+RVrlfdakdbOm66l1OsWg4ZAygkn2zO5Ygz2xe/bCPn6tVfNqNL1UadZaWm7vD52OZd7/VZVoljj8Uv3pWaKAJc+rIO+2xzRvobf0lePLRmY5PVmbYjfslfxfszp7pBuYlTf9NsXTV3/40cgLvRg1KP6zHe0gNxuLL8QSqUQ0ueq3KoRxTGCa+rGIJDawiSzV5zjDOS6UoDKjzCnzrVQl4GtG8S2UlU9p44/F/latexit
E
latexit sha1_base64=VQOOvVSP0lW6m0gNVQ8zA5pNsEc=AAAChnichVG7SgNBFD1ZXzE+ErURbIJBsQo3Eh9YBW0sTWJMQCXsrmMcsi92NwEN/oBgq4WVgoX4AX6AjT9gkU8QSwUbC282C6Ki3mV2zpy5586ZuZpjSM8nakeUnt6+/oHoYGxoeGQ0nhgb3/LshquLkm4btlvRVE8Y0hIlX/qGqDiuUE3NEGWtvtbZLzeF60nb2vQPHbFrqjVL7ktd9ZkqulVZTaQoTUEkf4JMCFIIY8NO3GEHe7ChowETAhZ8xgZUePxtIwOCw9wuWsy5jGSwL3CMGGsbnCU4Q2W2zv8ar7ZD1uJ1p6YXqHU+xeDhsjKJGXqkG3qhB7qlJ3r/tVYrqNHxcsiz1tUKpxo/mSy+/asyefZx8Kn607OPfSwHXiV7dwKmcwu9q28enb8UVwozrVm6omf2f0ltuucbWM1X/TovChd/+NHYC78YNyjzvR0/wdZ8OrOYzuazqdxq2KoopjCNOe7HEnJYxwZKXL+GU5zhXIkqaWVBWeqmKpFQM4EvoeQ+AL7mkM4=/latexit
ri
latexit sha1_base64=kTQzD4iiA9afGMFuEz1PlDAJN4Y=AAAChnichVG7TgJBFD2sL8QHqI2JDZFgrMhgUIwV0caShzwSJGR3HXFlX9ldSJD4Aya2UlhpYmH8AD/Axh+w4BOMJSY2Fl6WTYwS8W5m58yZe+6cmSuZqmI7jHV9wtj4xOSUfzowMzs3HwwtLBZso2HJPC8bqmGVJNHmqqLzvKM4Ki+ZFhc1SeVFqb7X3y82uWUrhn7gtExe0cSarhwrsugQlbOqp9VQhMWYG+FhEPdABF6kjdAjDnEEAzIa0MChwyGsQoRNXxlxMJjEVdAmziKkuPsc5wiQtkFZnDJEYuv0r9Gq7LE6rfs1bVct0ykqDYuUYUTZC7tnPfbMHtgr+/yzVtut0ffSolkaaLlZDV4s5z7+VWk0Ozj5Vo307OAY265XhbybLtO/hTzQN886vdxONtpeY7fsjfzfsC57ohvozXf5LsOz1yP8SOSFXowaFP/djmFQ2IjFt2KJTCKS2vVa5ccKVrFO/UgihX2kkaf6NVziCh3BL8SETSE5SBV8nmYJP0JIfQHBBpDP/latexit
rj
22
Chemical Reaction Design and Discovery
EQ1 EQ2
TS
Chemical Reaction
Energy
latexit sha1_base64=Wyp4gB8RxRsto5OIPkBWRdHaY3U=AAAChHichVG7SgNBFD1ZNcZ31EawCQbFQsJE4wMLCYpgaRIThSiyu07i4GZ32Z0EYvAHtFUsrBQsxA/wA2z8AYt8glhGsLHwZrMgGtS7zM6ZM/fcOTNXsw3hSsbqAaWjsyvYHerp7esfGBwKD4/kXKvs6DyrW4bl7Giqyw1h8qwU0uA7tsPVkmbwbe1orbm/XeGOKyxzS1ZtvldSi6YoCF2VRKXW98NRFmNeRNpB3AdR+LFphR+wiwNY0FFGCRwmJGEDKlz68oiDwSZuDzXiHELC2+c4QS9py5TFKUMl9oj+RVrlfdakdbOm66l1OsWg4ZAygkn2zO5Ygz2xe/bCPn6tVfNqNL1UadZaWm7vD52OZd7/VZVoljj8Uv3pWaKAJc+rIO+2xzRvobf0lePLRmY5PVmbYjfslfxfszp7pBuYlTf9NsXTV3/40cgLvRg1KP6zHe0gNxuLL8QSqUQ0ueq3KoRxTGCa+rGIJDawiSzV5zjDOS6UoDKjzCnzrVQl4GtG8S2UlU9p44/F/latexit
E
ML
• Acceleration by ML potential?
• Artificial force learning?
ML
Forces
• Scope/network expansion?
ML
• Fill any gap between theory and
experiments (reality) by data?
50. / 23
23
Machine Learning and Machine Discovery
• Machine Learning: many fascinating technical topics of my long-
standing interests on “ML with combinatorial structures” (such as GNNs)
• Machine Discovery: many long-standing important open problems
towards “AI for automating discovery”
Prior Info
Observational data
Reported facts
Textbook knowledge
Discovery
Representation
Model (Belief)
Intervention
Hypothesis
New Info
Prior Info
• Identify relevant variables
• Set design choices
• Set experiments
• Interpret results
Model (Belief)
Hypothesis
https://itakigawa.github.io/data/icred_med_202110.pdf
An exciting “real-world” test bench for ML researchers!
This Slide