At DataSift we're constantly looking for ways to help customers build better insights from data. We're increasingly looking to data science techniques to achieve this. One area we're currently working on is moving beyond keywords when building filters and classifiers. If you're a member of DataSift FORUM you'll be aware that we recently released our Keyword Relationship Explorer tool. This tool was a first step towards our goal. Join us for our first in a series of webinars for the DataSift FORUM where you will: Understand how we built the model that sits behind the tool Learn how you can build your own model from your own data See how we're looking to build the concept of similarity into our platform and how it will transform the way customers will filter and classify data

1. Applying Data Science to
Move Beyond Keywords
for Social Analysis

2. Richard Caudle
Director
Developer Relations
Claudio Weeraratne
Director
Product Management

3. DATASIFT
FORUM

4. RUN ON THE
BANKS?

5. RUN ON THE
BANKS?

6. AMBIGUITY OF NATURAL LANGUAGE

7. RUN ON THE
BANKS?

8. MOVING BEYOND
KEYWORDS
ba
nk
similarity x
wit
h-
dr
aw
similarity y
AND
interaction.content any "rbs,lloyds,hsbc,barclays"
AND interaction.content any "withdraw,close,cashpoint,atm"

9. CONCEPT MODELING

10. KEYWORD
RELATIONSHIP
S

11. CONCEPT
MODEL
r
b
s
VECTORSPACE
#rbs
runningbacks
#hsbc

12. OUR APPROACH
• Produce a vector space where words are grouped by their
context
• Context of a word is given by surrounding words
• Perform unsupervised machine learning to learn topics
• word2vec is a well known implementation
• gensim is a Python library that simplifies word2vec usage
• Resulting model is queryable for similarity (of word vectors)
• Language-agnostic solution

13. LEARNING
SIMILARITY
Learn to predict a word from surrounding words
"I'm heading to #rbs to close my
account"
rbs
account
close
rbs
account
close
hsbc
barclays
withdraw
balance
cash money
(1000's posts)
CONCEPT 'BANK'
NEURALNETWORK

14. LEARNING
SIMILARITY

15. DEMO

16. IMPROVED FILTERING &
CLASSIFICATION
interaction.content similar "bank,hsbc:0.7"
AND interaction.content similar "withdraw:0.8"
interaction.content any "rbs,lloyds,hsbc,barclays"
AND interaction.content any "withdraw,close,cashpoint,atm"
CONCISE
INTUITIVE
MAINTAINABLE
UP-TO-DATE
HIGHER COVERAGE
ACCURACY

17. IMPROVING OUR
PLATFORM
• Further validation of approach
• Operationalization of model production
• Creation new models for different audiences
• Automated updating of models
• Implementation of 'similarity' in CSDL

18. Q&A

19. LEARN MORE
datasift.com/forum

20. THANK YOU