We study the news reading behaviour of several hundred thousand users on 65 highly-visited news sites. We focus on a specific phenomenon: users reading several articles related to a particular news development, which we call story-focused reading. Our goal is to understand the effect of story-focused reading on user engagement and how news sites can support this phenomenon. We found that most users focus on stories that interest them and that even casual news readers engage in story-focused reading. During story-focused reading, users spend more time reading and a larger number of news sites are involved. In addition, readers employ different strategies to find articles related to a story. We also analyse how news sites promote story-focused reading, by looking at how they link their articles to related content published by them, or by other sources. The results show that providing links to related content leads to a higher engagement of the users, and that this is the case even for links to external sites. We also show that the performance of links can be affected by their type, their position, and how many of them are present within an article. This work co-authored with J. Lehmann, C. Castillo and R. Baeza-Yates has been published in the Journal of The Association For Information Science And Technology (JASIST), available online in May 2016. The work was presented at the Yahoo TechPulse Annual conference in December 2016.

1. 1
Story-focused Reading in Online
News and its Potential for User
Engagement
Janette Lehmann, Carlos Castillo, Mounia Lalmas, Ricardo Baeza-Yates

2. Story-focused news reading
2
• While reading news, users may become interested in a particular news item, and
want to find more about it:
– to obtain various angles on the story
– to overcome media bias
– to confirm the veracity of what they are reading
– …
Users read multiple articles about a particular news development or event.

3. Story-focused news reading
3
• Previous studies found that
– many users visit established news outlets to confirm a story, no matter
from which source the information initially came from (The Poynter Institute)
– some users find links to related information on a news article page
important (Pew Research Center)
Users read multiple articles about a particular news development or event.

4. Story-focused news reading
4
• News sites recognise that users want to further inform themselves so they:
– provide information on different aspects or components of a story
– link to other articles published by them and other sites
Our goal is to understand the effect of story-focused news reading on user
engagement and provide insights into how news sites can support it.
Users read multiple articles about a particular news development or event.

5. Questions
5
• Does story-focused news reading exist and to what extent?
• What are the characteristics of story-focused news reading?
• How do links on news articles promote story-focused news reading?
• How does story-focused news reading impact user engagement?
Users read multiple articles about a particular news development or event.

6. One-month Yahoo Toolbar data
6
• A news story is a collection of articles related to same news event
(TD-IDF term weight in article and cosine similarity with threshold 0.4)
• On average, each story has 14 articles (median 8), is covered by 7 news sites
(median 5), and has 2,482 users per story (median 758)
65 popular news sites publishing articles in English; 4.9M news reading
sessions; covering 2,536 stories comprising 25,703 news articles
• Percentage of users engaging at least once in story-focused reading is 16%
• Avid news readers more likely to engage in story-focused reading:
– 64% of users with at least 15 reading sessions have at least one story-focused session
– But being an active news reader does not imply engaging in story-focused reading

7. Questions
7
• Does story-focused news reading exist and to what extent?
• What are the characteristics of story-focused news reading?
• How do links on news articles promote story-focused news reading?
• How does story-focused news reading impact user engagement?
Users read multiple articles about a particular news development or event.

8. Story-focus news reading exist
• shuffle test: alternative dataset with same
distribution of session length, but with
random articles
• probability of story-focused reading is x4
larger and probability of multi-site is x2
larger in actual dataset than shuffle dataset
8
●
● ● ●
●
●
●
●
●
●
● ●
●
●
●
●
●
0.0
0.3
0.6
0.9
10−4
10−3
10−2
10−1
Prob. of story-focused reading
Density
Actual data
Shuffled data
●
●
●
● ● ●
● ● ● ●
●
0
1
2
3
0.25 0.50 0.75
Prob. of multi-provider reading
Density
Actual data
Shuffled data
●
story-focused reading does not depend on
popularity of a story, its number of articles, or
number of news sites that cover it

9. Questions
9
• Does story-focused news reading exist and to what extent?
• What are the characteristics of story-focused news reading?
• How do links on news articles promote story-focused news reading?
• How does story-focused news reading impact user engagement?
Users read multiple articles about a particular news development or event.

10. Story-focused vs non story-focused reading
10
%
sessions
num. of
articles
%
focused
duration (minutes) number of sites
non-foc. foc. non-foc. foc.
74.93
17.31
4.95
1.66
0.63
0.27
0.12
0.13
1
2
3
4
5
6
7
7>
15.16
29.00
41.33
51.02
60.85
67.13
78.15
3.09
5.77
8.57
11.24
13.60
15.92
18.61
21.29
6.66
9.23
12.92
15.96
18.35
21.59
27.91
1.00
1.20
1.37
1.52
1.65
1.81
1.91
2.12
1.26
1.48
1.67
1.83
1.98
2.14
2.55
• users spend more time reading news when focusing on a specific story
• more sites are accessed when users engage in story-focused reading
when session
length increases
(more articles are
read) à probability
that session is
story-focused
increases

11. Depth of story-focused reading
11
session
%
number of articles duration (minutes) number
of sites
in-story out-story total per-article
85.03
11.48
2.43
0.69
0.23
0.08
0.06
2
3
4
5
6
7
>7
0.89
1.09
1.45
1.67
1.80
2.67
3.05
6.67
10.48
14.29
18.23
20.09
23.09
25.03
3.34
3.49
3.57
3.65
3.35
3.30
2.79
1.27
1.53
1.79
2.05
2.31
2.36
3.19
Per-article dwell time increases for sessions with <= 5 articles:
users spend time reading articles they are accessing
Per-article dwell time decreases for sessions > 5 articles:
users are skimming articles, as articles contain increasingly more
redundant information
number out-story
articles increases
as session depth
increases
deeper story-
focused sessions
are longer à
involve a larger
number of sites +
and mostly made of
in-story articles

12. Questions
12
• Does story-focused news reading exist and to what extent?
• What are the characteristics of story-focused news reading?
• How do links on news articles promote story-focused news reading?
• How does story-focused news reading impact user engagement?
Users read multiple articles about a particular news development or event.

13. Links
• Type of link, Position of link, Number of link
• Performance metrics:
• Popularity: percentage of links of a particular group
• Performance: how often the links in the group are clicked (percentage)
13
2 3 4 5 6 7 >7
0
20
40
60
80
100
Number of in-story news articles
%ofupstreamtraffic
Internal Article (Int) Non-Article (Int) External
• 75.45% of articles have
inline links
• 6.4% of links in article are
inline links
inline links

14. Link performance: Type of links
• Internal links within articles promotes story-focused reading
• Users tend to click on links that bring them to other news articles within same site
14
-20
0
20
40
60
80
100
Internal Article (Int) Non-Article (Int) External
% Articles with this type of links Popularity (%) of this type of links
Performance (%) of this type of links
apart for multimedia, no
external links with
performance comparable
to internal links to news
articles

15. Link performance: Position of links
15
−0.2
0.0
0.2
[0,0.1[ [0.3,0.4[ [0.6,0.7[ [0.9,1]
Position in article text
Linkperformance
Linkpopularity
Link popularity● Link performance
●
●
●
●
● ●
●
●
●
●
●
●
●
● ● ● ● ●
●
●
10%
20%
30%
best performance with
links at end of articles
good performance for
links in upper 20% and
40% of article
(multimedia)
low performance for
links at top articles

16. Link performance: Number of links
• < 10 inline links may be wasting an
opportunity, as users may be enticed to
click to access related content
• > 10 and < 29 links does not result in more
clicks, but spreads the clicks more
• > 29 links may actually harm user
experience
16
0
5
10
15
20
[0,2] [9,11] [18,20] [27,29] [36,38]
Numberofclicks
Number of inline links in article
0.0
0.5
1.0
1.5
[0,2] [9,11] [18,20] [27,29] [36,38]
Number of inline links in article
Avg.clicksperlink
is 10 links per-article the “sweet spot”?

17. Questions
17
• Does story-focused news reading exist and to what extent?
• What are the characteristics of story-focused news reading?
• How do links on news articles promote story-focused news reading?
• How does story-focused news reading impact user engagement?
Users read multiple articles about a particular news development or event.

18. User engagement
• 50 news sites, 57K users,1M sessions with users with at least one non-story-
focused reading session and one story-focused reading session
• Engagement metrics:
– Dwell time: time spend on news site
– Absence time: % of sessions with absence time < 12 hours on news site
18
• story-focused reading vs non story-focused reading
• internal story-focused reading vs external story-focused reading

19. 19
User engagement: Dwell Time
internal story-focused
sessions:
• dwell time higher compared to
non story-focused sessions
• average increase in dwell
time is 50%
external story-focused
sessions:
• no observable effect on dwell
time
News provider
Dwelltimepersession
Non-focused Focused Ext-focused

20. User engagement: Absence Time
20
internal story-focused sessions:
• for 78% of sites, probability that
users return within 12 hours
increases by 68%
external story-focused sessions:
• for 70% of sites, probability that
users return within 12 hours
increases by 76% News provider
p(absence12h)
Non-focused Focused Ext-focused

21. Story-focused news reading exist …………and
21
• Internal inks within article promotes story-focused reading and keeps users
engaged
– longer period of engagement (reading sessions are longer) and earlier re-
engagement (shorter absence time).
• External links does not have negative effect on user engagement
– period of engagement remains the same (reading sessions are the same),
and re-engagement begins even sooner (shorter absence time).
• Important to use links properly, in terms of quantity, type and position.

22. Future work
22
• We need to account for the novelty and more importantly the quality of the related
content
• The online news ecosystem has changed, both in terms of how news sites
provides news and how users inform themselves