The Five Tribes of Machine Learning, and What You Can Take from Each: There are five main schools of thought in machine learning, and each has its own master algorithm – a general-purpose learner that can in principle be applied to any domain. The symbolists have inverse deduction, the connectionists have backpropagation, the evolutionaries have genetic programming, the Bayesians have probabilistic inference, and the analogizers have support vector machines. What we really need, however, is a single algorithm combining the key features of all of them. In this talk I will describe my work toward this goal, including in particular Markov logic networks, and speculate on the new applications that such a universal learner will enable, and how society will change as a result.

1. And What You Can
Take from Each
Pedro Domingos
University of Washington

2. Evolution Experience
Culture

3. Evolution Experience
Culture Computers

4. Most of the knowledge in the world in the
future is going to be extracted by machines
and will reside in machines.
– Yann LeCun, Director of AI Research, Facebook

5. 1. Fill in gaps in existing knowledge
2. Emulate the brain
3. Simulate evolution
4. Systematically reduce uncertainty
5. Notice similarities between old and new

6. Tribe Origins Master Algorithm
Symbolists Logic, philosophy Inverse deduction
Connectionists Neuroscience Backpropagation
Evolutionaries Evolutionary biology Genetic programming
Bayesians Statistics Probabilistic inference
Analogizers Psychology Kernel machines

7. Tom Mitchell Steve Muggleton Ross Quinlan

8. Addition Subtraction
2
+ 2
―――
= ?
――
2
+ ?
―――
= 4
――

9. Deduction
Socrates is human
+ Humans are mortal .
―――――――――――
= ?
Induction
Socrates is human
+ ?
―――――――――――
= Socrates is mortal
―――――――――― ――――――――――

11. Yann LeCun Geoff Hinton Yoshua Bengio

16. John Koza
John Holland
Hod Lipson

20. David Heckerman Judea Pearl Michael Jordan

24. Peter Hart Vladimir Vapnik Douglas Hofstadter

28. Tribe Problem Solution
Symbolists Knowledge composition Inverse deduction
Connectionists Credit assignment Backpropagation
Evolutionaries Structure discovery Genetic programming
Bayesians Uncertainty Probabilistic inference
Analogizers Similarity Kernel machines

29. Tribe Problem Solution
Symbolists Knowledge composition Inverse deduction
Connectionists Credit assignment Backpropagation
Evolutionaries Structure discovery Genetic programming
Bayesians Uncertainty Probabilistic inference
Analogizers Similarity Kernel machines
But what we really need is
a single algorithm that solves all five!

30. Representation
Probabilistic logic (e.g., Markov logic networks)
Weighted formulas → Distribution over states
Evaluation
Posterior probability
User-defined objective function
Optimization
Formula discovery: Genetic programming
Weight learning: Backpropagation

31. Much remains to be done . . .
We need your ideas

32. Home Robots
Cancer Cures 360o Recommenders
World Wide Brains

33. If we used all our technology resources,
we could actually give people personalized
recommendations for every step of your life.
– Aneesh Chopra, former CTO of the U.S.