Are you tapping into automation for keyword research? If not, why not? When it comes to SEO, automation is awesome. For starters, it can help free up a lot of time that is normally spent on menial tasks. What’s more, it can also aid deep analysis, and even facilitate innovation. If you are still doing keyword research manually, this is a must-attend session. Paul will show you how to get started with automated keyword research, using some easy-to-use tools. You’ll see first-hand how they can help you uncover valuable insights automatically. Overall, you will walk away with an immediately actionable plan to start automating your keyword research today.

1. HOW TO AUTOMATE YOUR
KEYWORD RESEARCH
Paul Shapiro – Organic Search Director at Catalyst
SearchLove Boston – May 4th, 2016

2. @fighto 2
Without Automation

3. @fighto 3
With Automation

4. @fighto
WHO AM I?
4

5. @fighto 5
Tool Data Sources Visualization
+ +

6. @fighto
WHAT YOU WILL LEARN
• The problems with the standard keyword research process
• How you can save time on keyword research with automation
• How to automate keyword research with a powerful, but easy-to-use
tool
• How APIs work and how to leverage them
• What data sources are useful for keyword research automation
• How to use data visualization to make keyword research decisions
6

7. MANUAL KEYWORD
RESEARCH IS
OFTEN BLAND &
TIME CONSUMING
7

8. @fighto 8

9. @fighto 9
Most Keyword Research Looks Like This

10. @fighto 10
That’s just a
Google Keyword
Planner Export

11. @fighto
TYPICAL TIME INVESTMENT FOR KEYWORD RESEARCH
5
8
12
21 21
6
10
18
22
28
0
5
10
15
20
25
30
Micro (0-49 pages) Small (50-99 pages) Medium (100-249 pages) Large (250-499 pages) Extra Large (>500 pages)
Hours to Complete Keyword Research by Site Size
Average (Low End) Average (High End)
11

12. @fighto 12
AIN’T NOBODY GOT TIME FOR MANUAL
KEYWORD RESEARCH!

13. @fighto
SIZE OF THE DATA SET
13
vs.

14. @fighto 14
We can do better!

15. WE CAN SAVE TIME WITH
AUTOMATION

16. @fighto
TYPICAL TIME INVESTMENT FOR KEYWORD RESEARCH
5
8
12
21 21
6
10
18
22
28
0
5
10
15
20
25
30
Micro (0-49 pages) Small (50-99 pages) Medium (100-249 pages) Large (250-499 pages) Extra Large (>500 pages)
Hours to Complete Keyword Research by Site Size
Average (Low End) Average (High End)
16

17. @fighto 17
If we do keyword research
once per month, we will save
around 24 to 45 full days
over a 5-year period using
automation.
http://xkcd.com/1205/

18. WE CAN DO BETTER THAN AN
EXCEL SPREADSHEET

19. @fighto
THAT’S MORE LIKE IT!
19
h/t @victorpan

20. @fighto
HOW TO USE FOR KEYWORD SELECTION / TARGETING
20
This top-right quadrant contains
keywords with:
• Low competition
• Good growth
Larger bubbles show higher
search volumes.
You can alternatively use
current rank on the x-axis to
signal organic market share like
a traditional growth-share
matrix.

21. @fighto 21
That’s much better!
actionability.

22. MY KEYWORD RESEARCH
AUTOMATION PROCESS

23. @fighto 23
https://www.knime.org
It starts with a tool…

24. @fighto
WHY KNIME?
• Singular tool for data collection,
manipulation, analysis, and reporting
• Easy to use and no development skills
necessary
• If you are a developer, it provides faster
prototyping
• Large library of built-in “nodes”
• Free/Open Source
• Run on Windows/Mac/Linux
• Infinitely expandable
24

25. THE BASICS OF KNIME

26. @fighto
WHAT’S A NODE?
• Nodes are prebuilt, drag and drop modules designed perform a singular task
• Nodes are strung together like a chain to accomplish larger, more complex tasks
• Nodes can be grouped together into “meta-nodes”, which can be configured in unison
26

27. @fighto
HOW DO YOU ADD NODES & HOW DO THEY CONNECT?
How do you add nodes to your “workflow”?
How do you string nodes together?
27

28. @fighto
HOW DO YOU CONFIGURE & RUN NODES?
Configuring Nodes
Running Workflows
28
OR

29. APIs 101
29

30. @fighto
EASY & APPLICABLE API: GOOGLE AUTOSUGGEST
http://suggestqueries.google.com/complete/search?output=toolbar&hl=en&q=ninja%20turtles
30
Simple API using “GET Request”
This is a variable
that changes

31. @fighto 31
Output = XML

32. A KNIME-API EXAMPLE
32

33. @fighto
STEP 1: LIST OF KEYWORDS
33

34. @fighto
STEP 2: ENCODE SPACES IN KEYWORD PHRASES
34
Space
Character

35. @fighto
STEP 3: CONCATENATE API URL + KEYWORDS
35

36. @fighto
STEP 4: DOWNLOAD API OUTPUT
36

37. @fighto
STEP 6: LOOP THROUGH EACH KEYWORD
37

38. @fighto
STEP 7: PARSE THE XML OUTPUT FORM THE API
38

39. @fighto 39
Auto-Complete
Suggestions

40. @fighto 40

41. @fighto
FYI: I’M NOT A KNIME SALESMAN
41

42. DATA INPUTS / SOURCES

43. @fighto 43
# of primary data sources to aggregate

44. @fighto 44
http://searchwilderness.com/gwmt-data-python/
Google Search Analytics – Historical MySQL
Database via Python Script

45. @fighto 45
AdWords - Search Query Report API or CSV

46. @fighto 46
SEMRush – API (if <100,000)
or Domain vs. Domain:
Competitor Keywords (Advanced Mode)
https://www.semrush.com/info/domain_vs_domain

47. @fighto 47
GrepWords API – For Google Keyword Planner
related query suggestions & search volumes
http://grepwords.com/

48. @fighto 48
Google Autosuggest API
or Emulation with Selenium

49. @fighto 49
Rank Checking/Full SERP API - GetSTAT
https://getstat.com/

50. @fighto 50
MarketMuse – Semantic Keyword
Recommendations
https://www.marketmuse.com/
Alternative: http://pshapi.ro/SemanticKWR

51. @fighto 51
Moz API + Moz Nodes
https://tech.knime.org/book/mmi-labs-nodes
https://moz.com/products/mozscape/access

52. @fighto 52
Google Trends – Unofficial API
https://github.com/GeneralMills/pytrends
Script soon to follow: http://pshapi.ro/slopetrends

53. @fighto 53
More data sources:
http://pshapi.ro/kwmetrics

54. THE DATA-TO-KNIME
PROCESS
54

55. @fighto
Filtering +
Data
Manipula-
tions
ONE BIG KEYWORD LIST
Seed Keywords
- List
- GWMT
- SEMRush Comp. KWs
- SQR Keywords
Keyword Planner
Suggestions (via
GrepWords)
Google Autocomplete
Semantic Keyword
Recommendations
(via MarketMuse)
Google Autocomplete
SEMRush
Domain vs. Domain
Keywords
Google Autocomplete
55

56. @fighto
STRING ‘EM ALL TOGETHER AND THEN…
56

57. @fighto
DATA MANIPULATIONS / CALCULATIONS
• Get top 10 results from rank checking API (i.e., GetSTAT)
• Use Moz API nodes and find average PA to assess
competiveness.
• Optionally, use SEMRush’s Keyword Difficulty API
Organic
Competition
Search Volume
Keyword Trends
57

58. @fighto
DATA MANIPULATIONS / CALCULATIONS
• Get top 10 results from rank checking API (i.e., GetSTAT)
• Use Moz API nodes and find average PA to assess
competiveness.
• Optionally, use SEMRush’s Keyword Difficulty API
Organic
Competition
• Get Search Volumes via SEMRush API or via GrepWords APISearch Volume
Keyword Trends
58

59. @fighto
DATA MANIPULATIONS / CALCULATIONS
• Get top 10 results from rank checking API (i.e., GetSTAT)
• Use Moz API nodes and find average PA to assess
competiveness.
• Optionally, use SEMRush’s Keyword Difficulty API
Organic
Competition
• Get Search Volumes via SEMRush API or via GrepWords APISearch Volume
• Use 2 Years of Google Trends data to calculate slope and
determine growing/declining keywordsKeyword Trends
59

60. @fighto 60

61. @fighto 61
Reusable
5-Minutes

62. @fighto 62
You can use this data to
filter keywords and reduce
massive list... OR, you can
leverage data visualization!

63. VISUALIZING KEYWORD
RESEARCH

64. @fighto 64
We have a
few options…

65. @fighto
OPTION: EXCEL + BUBBLE CHART
65

66. @fighto
OPTION: TABLEAU
66

67. @fighto
OPTION: PYTHON + BOKEH CHARTS IN KNIME
67

68. @fighto
THEY’RE INTERACTIVE…
68

69. @fighto 69
Sample Python Code:
http://pshapi.ro/SLBokeh

70. @fighto
SO WHAT…
• Free up time spend doing mundane, repetitive work and spend more doing impactful, strategic work
• Free up staffing resources and utilize them in other ways
• Extract and analyze a larger pool of data for greater data pool
70
From this… To this…

71. @fighto
SUMMARY
• You should rethink how your currently do keyword research.
• Keyword research is often bland, time consuming, and not as actionable as it can be.
• KNIME provides an easy-to-use interface for automating your keyword research
• APIs are easier to use than they sound; Use them
• There are great data sources you can leverage via APIs/built-in KNIME nodes
• Visualize your keywords using a growth-share matrix like-graph to make keyword usage
decisions
• Emojis are awesome
71

72. @fighto
RESOURCES: DATA SOURCES > KNIME
72
Data Source KNIME Node(s)
Google Search Analytics Database Table Connector/Reader
AdWords Search Query Report XLS Reader
SEMRush Domain vs. Domain (or API if less
than 100,000 keyword)
XLS/CSV Reader or GET Resource & Read
REST Representation
Google Keyword Planner/GrepWords GET Resource & Read REST Representation
Google Autosuggest
GET Resource & Read REST Representation
(or Selenium nodes)
Rank Checking (via GetSTAT) GET Resource & Read REST Representation
Semantic Keywords/MarketMuse POST Resource
MOZ Moz API Connector/URL Metrics nodes
Google Trends TBD – Either CSV or Python Script nodes

73. @fighto
ADDITIONAL RESOURCES
• KNIME Version to install: “KNIME Analytics Platform + all free extensions”
• To install KNIME Add-ons, Choose Help->Install New Software
• Moz nodes
• Should come with Nightly Builds. If not can add http://update.knime.org/community-
contributions/3.1 as a repository.
• Add-on Selenium nodes
• http://seleniumnodes.com/
• Search Engine Land article addressing additional data inputs: http://pshapi.ro/kwmetrics
• Information about Growth-share matrices:
https://en.wikipedia.org/wiki/Growth%E2%80%93share_matrix
• Sanple Python + Bokeh script for an interactive keyword research visualization in KNIME:
http://pshapi.ro/SLBokeh
73

74. @fighto
Questions?
74

75. THANK YOU!
@fighto
http://searchwilderness.com
http://catalystdigital.com