A presentation discussing the use of command line automation to save time on repetitive SEO tasks. Covers basic command line usage and the use of PowerShell with a number of useful scripts that you can start using today. Presented at BrightonSEO in April 2019
2. @Fearless_Shultz #brightonSEO
@Fearless_Shultz #brightonSEO
What the command line
is and how it helps us to
work more efficiently
Basic examples of how
time and effort can be
saved by using CLI Scripts
Chaining multiple scripts /
modules together to create
automated solutions
What I’m Going to Talk About.
27. @Fearless_Shultz #brightonSEO
Basic Command Line Examples
Create a new
master file
Open each
individual csv
Select and
copy and
paste the data
Double check
everything has
been copied
39. @Fearless_Shultz #brightonSEO
Basic Command Line Examples
Download
images via FTP
Upload in
batches to an
image
processing
service
Tweak image
settings as
required
Download
optimised images
50. @Fearless_Shultz #brightonSEO
Basic Command Line Examples
Install Lighthouse
extension
Open each URL
in a Browser
Select options
and run
performance
test
Once each
report has
finished running,
save as a HTML
file
55. @Fearless_Shultz #brightonSEO
Basic Command Line Examples
• This does require some additional modules installing but it is generally very quick and
easy to get set up. Full instructions here https://edit.co.uk/blog/save-time-by-batch-
processing-lighthouse-audit-files-from-the-command-line/
62. @Fearless_Shultz #brightonSEO
Introducing PowerShell
Scripting Panel
Command Line
cmdlet Library
PowerShell is a task-based command-line shell and scripting language built on .NET. PowerShell helps
system administrators and power-users rapidly automate tasks that manage operating systems (Linux,
macOS, and Windows) and processes
63. @Fearless_Shultz #brightonSEO
Introducing PowerShell - cmdlets
A cmdlet (pronounced "command-let") is a lightweight Windows PowerShell script that performs a
single function.
Import-CSV Invoke-RestMethodExport-CSV
64. @Fearless_Shultz #brightonSEO
Introducing PowerShell – Import-Csv
The Import-CSV cmdlet turns a
CSV file into an object which
has properties which can easily
be referenced with filters
including:
-lt (Less than)
-gt (Greater than
-eq (Equal to)
-like (Like)
71. @Fearless_Shultz #brightonSEO
Introducing PowerShell – Import-Csv
Open redirect
report from SF
in Excel
Create filter to
find any rows
with a redirect
count of more
than one.
Create a new
column to
contain the
redirect
Use a
concatenation
formula to
prepend the
rule to the first
and last URLs
Remove all the
other columns
which aren’t
required and
save.
74. @Fearless_Shultz #brightonSEO
Introducing PowerShell – Invoke-RestMethod
Invoke-RestMethod consumes
REST APIs and parses the result
and where JSON is returned
the data is converted into a
PowerShell object that can be
easily queried.
86. @Fearless_Shultz #brightonSEO
Confidential
@Fearless_Shultz #brightonSEO
What is the Command Line?
cd "C:Program Files (x86)Screaming Frog SEO Spider"
ScreamingFrogSEOSpiderCli.exe ^
--crawl https://edit.co.uk ^
--headless ^
--save-crawl ^
--config “C:image-crawl.seospiderconfig" ^
--output-folder “C:scriptsScreamingFrog" ^
--export-tabs “Internal:ALL"
Chaining-lighthouse.bat
87. @Fearless_Shultz #brightonSEO
Confidential
@Fearless_Shultz #brightonSEO
What is the Command Line?
@echo off
cd C:scriptsChainingLighthouselighthouse-reports
for /f "delims=" %%a in
(C:scriptsChainingLighthousetest-urls.txt) DO (
ECHO Line is: %%a
lighthouse --quiet --chrome-flags="--headless" %%a
)
Test.bat
91. @Fearless_Shultz #brightonSEO
CLI Scripting Resources
Title Link
A beginners guide to batch scripting in
Windows http://steve-jansen.github.io/guides/windows-batch-scripting/
Distilled Beginners guide to command line for
SEO https://www.distilled.net/blog/seo/the-beginners-guide-to-using-the-command-line-for-seo/
SEO Strategies to impress your boss https://elite-strategies.com/impress-seo-boss-linux-command-line-tricks/
Beginners guide to Terminal on Mac https://www.makeuseof.com/tag/beginners-guide-mac-terminal/
Tom Pool CLI Hacks presentation http://bit.ly/tom-pool-command-line-hacks-presentation
Tom Pool CLI Cheat Sheet http://bit.ly/command-line-cheat-sheet
Tom Pool Podcast http://bit.ly/tom-pool-presentation-podcast
64 Resources for learning to code online https://skillcrush.com/2016/03/15/64-online-resources-to-learn-to-code-for-free/
The 100 days of code project https://www.100daysofcode.com/
PowerShell download packages https://github.com/PowerShell/PowerShell
Beginners guide to PowerShell https://blog.netwrix.com/2018/02/21/windows-powershell-scripting-tutorial-for-beginners/
Guide to using Arrays in Poweshell https://4sysops.com/archives/arrays-in-powershell-create-change-read-sort-delete/
PowerShell Foreach-Object https://www.youtube.com/watch?v=BoutUXsBGcU
PowerShell out-File https://stackoverflow.com/questions/19450616/export-csv-exports-length-but-not-name
92. @Fearless_Shultz #brightonSEO
CLI Scripting Resources
Title Link
CSV kit for PowerShell https://searchenginewatch.com/2019/02/06/technical-seo-2019-statelessness/
Loop through files in directory in powershell
https://stackoverflow.com/questions/18847145/loop-through-files-in-a-directory-using-
powershell
Understanding invoke-rest method https://4sysops.com/archives/understanding-the-invoke-restmethod-powershell-cmdlet
Working With Csvs in PowerShell https://www.petri.com/powershell-import-csv-cmdlet-parse-comma-delimited-csv-text-file
Video example of connecting to REST Method https://www.youtube.com/watch?v=7mEmQgGowMY
PowerShell add-ons https://4sysops.com/archives/my-favorite-windows-powershell-ise-add-ons/
Powershell substrings
https://4sysops.com/archives/strings-in-powershell-replace-compare-concatenate-split-
substring/
Screaming Frog Command Scheduler
https://www.greenlanemarketing.com/blog/schedule-your-screaming-frog-crawls-with-
punctual-frog/
Screaming Frog Command Line
Documentation https://www.screamingfrog.co.uk/seo-spider/user-guide/general/#commandlineoptions
Running Screaming Frog CLI On a Server https://www.databulle.com/blog/seo/screaming-frog-headless.html
GTMetrix API https://gtmetrix.com/api/
Cygwin to run Linux and Mac commands on
Windows https://www.cygwin.com/
Huge searchable DB of REST Apis https://www.programmableweb.com/apis/directory
Regex to strip out domain from URL (For
Redirects) https://stackoverflow.com/questions/4791700/regex-needed-to-strip-out-domain-name
Content and sentiment analysis https://www.pveller.com/content-work-automation-with-text-analytics/