data science at the new york times

data science @ The New York Times
chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
chris.wiggins@hackNY.org
@chrishwiggins
data natives, berlin, 2020-11-18

data science: mindset & toolset
- drew conway, 2010

data science: mindset & toolset
develop + deploy
machine learning solutions
to
newsroom + business problems

data science @ The New York Times:

data science @ The New York Timesdata science @ The New York Times

news: 20th century
church state

every publisher is now a startup
2014

ad -v- subs crossover in 2011;
post-election data not shown
2017

news: 21st century
church state
data

learnings
- descriptive modeling
- predictive modeling
- prescriptive modeling

(actually ML, shhhh…)
- (unsupervised learning)
- (supervised learning)
- (reinforcement learning)

(actually ML, shhhh…)
2012; h/t michael littman

learnings

UPDATE COPYReaderscope
In the course of our global expansion, we
realized we needed to have much more
sophisticated, real-time insight into what’s
happening across our site.  
Who is reading what? And where?

LOCATIONS FAQAUDIENCE SEGMENTSTOPICS
FAQ or Intro Information
Lorem ipsum dolor sit amet, consectetur
adipiscing elit, sed do eiusmod tempor incididunt
ut quislabore et dolore magna aliqua. enim ad
minim veniam, quis quis aliqua ullamconostrud
exercitation ullamco laboris nisi ut aliquip ex ea
commodo consequat.
Duis aute irure dolor in reprehenderit in
voluptate velit esse cillum dolore eu fugiat
nulla pariatur. Excepteur sint occaecat non
proident, sunt in culpa qui officia deserunt
mollit anim id est laborum. aliquip ex ea
commodo consequat.
nulla pariatur. Excepteur sint occaecat
cupidatat non proident, sunt in culpa qui
officia deserunt mollit anim id est laborum.
aliquip ex ea commodo consequat.aliquip ex
Privacy Policy Terms of Service © The New York Times CompanyNYTimes.com Send Us Feedback
Searchex. High-tech Lifestyle, Parents, Media - Comedy Films
Audience Segment
Search by
nytreaderscope
Illustration by Clara Nguyen
commodo consequat.
commodo consequat.
Audience Segment
Search by
nytreaderscope
f
f
f
Tool: Readerscope
AUDIENCE INSIGHTS ENGINE

commodo consequat.
commodo consequat.
Audience Segment
Search by
nytreaderscope
C-Suite
C-Suite, Executives and BDMs - Entertainment
C-Suite, Executives and BDMs - Media
C-Suite|
C-Suite
C-Suite, Executives and BDMs - Entertainment
C-Suite, Executives and BDMs - Media
commodo consequat.
commodo consequat.
Audience Segment
Search by
nytreaderscope
f
Tool: Readerscope
AUDIENCE INSIGHTS ENGINE

predictive modeling, e.g.,
“the funnel”

interpretable predictive modeling
supercoolstuff

optimization & learning, e.g.,
“How The New York Times Works “popular mechanics, 2015

optimization & prediction, e.g.,
(some models)
(somemoneys)
“newsvendor problem,” literally (+prediction+experiment)

deep prediction w/ cloud AI
ming zhao: convolutional neural nets, in google
cloud platform, to help “tone” images for print

things: cloud AI
thousands of images, daily
deep prediction w/ cloud AI

We conducted
user research
to gather
millions of
observations
about how
different
articles made
people feel.
DATA COLLECTION
When reading this article, did you feel…
Anger Sadness Happiness Despair
Hurt No Emotion Jealousy Frustration
Anxiety Hope Hate Interest
Guilt Contentment Contempt Love
Compassion Shame Amusement Stress
Irritation Fear Boredom Surprise
Confusion Disgust Irony Pride
Disappointment

Adventur
ous
98
Interest
42
Happiness
96
Self 
Confident
39
Love
97
*units based on 100th percentile
Hate
Inspired
100
Amused
100
Sadness
27

Sources: Google DFP, NYT Ad Performance Data, Sizmek
April May June July August September October November December
Perspective Targeting Impression Volume By Month
Throughout the year, NYT
began running more and more
perspective targeting
campaigns every month.
And performance kept
breaking boundaries and
setting new benchmarks for
success.
A Record First Year

learnings
…two examples

w/“audience development” team

leverage methods which are predictive yet performant
w/“audience development” team

driving question: which content
should we promote, where and when?

NB: data informed, not data-driven

learnings
… recommendation as prescription

2018: algos for *highly editorially curated* content pools
- smarter living
- midterms
- editors picks

2019-now: all of the above,
plus:
- For You Tab
- stay tuned…

slow: Randomized controlled trial

slow: Randomized controlled trial
time ——->
banditRCT

old (1933) idea: do the best you can

Lihong Li (YHOO->MSFT->GOOG), 2011
thompson sampling & “bandits”

common requirements in
data science:

common requirements in
data science:
1. people
2. ideas
3. things
cf. John Boyd, USAF

monica rogati, Aug 1 2017 hackernoon.com
things: de>da>ds/ml/ai

Reporting
Learning
Test
Optimizing
Exploredescriptive:
predictive:
prescriptive:

watch this space: NYT+AI
physics
math/fin p chem app math
cog sciEE

people.. so far (we’re hiring!!!!)
astrophys math/fin
pure mathapp math
cog sciEE
biophysseismology neuro
physics
mech eng CS mol bio ?CS

chris.wiggins@columbia.edu
chris.wiggins@nytimes.com
chris.wiggins@hackNY.org
@chrishwiggins
more info: data-ppf.github.io

data science at the new york times

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie data science at the new york times

Ähnlich wie data science at the new york times (20)

Mehr von chris wiggins

Mehr von chris wiggins (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

data science at the new york times