Presentation at #altmetrics14, #WebSci14, Bloomington, IN altmetrics.org/altmetrics14/

1. Measuring Twitter activity of
arXiv e-prints and published papers
Stefanie Haustein, Timothy D. Bowman, Benoît Macaluso,
Cassidy R. Sugimoto & Vincent Larivière
Canada Research Chair
on the Transformations of Scholarly Communication
École de bibliothéconomie et des sciences de l’information
@stefhaustein

2. Introduction
• increase of Twitter use
• 230 million active users, 500 million tweets per day
• 39% increase of users from 09/2012 to 09/20131
• 16% of US, 3% of world population in 20131
• 19% of US internet users 01/20142
• uptake by researchers
• 1 in 40 university faculty member in US and UK
have Twitter account (Priem & Costello, 2010)
• 9% of researchers use Twitter for work (Rowlands et al., 2011)
• 15% of German university faculty members, 70% of which at least
occasionally in professional context (Pscheida et al., 2013)
1 Twitter statistics calculated based on data from: http://www.sec.gov/Archives/edgar/data/1418091/000119312513400028/d564001ds1a.htm and
http://www.census.gov/population/international/data/
2 Pew Research Center’s Internet Project surveys, 2010-2014
http://www.pewresearch.org/fact-tank/2014/06/11/can-twitter-survive-in-a-facebook-world-the-key-is-being-different/

3. Introduction
• social media activity around scholarly articles grows
5% to 10% per month (Adie & Roe, 2013)
• scholarly documents on Twitter
1.6% of WoS papers with DOIs 2005-2011 (Zahedi, Costas & Wouters, 2014)
13.3% of WoS papers with DOIs 07-12/2011 (Costas, Zahedi & Wouters, 2014)
20.4% of PubMed/WoS 2012 (Haustein et al., 2014)
21.5% of WoS papers with DOIs 2012 (Costas, Haustein & Larivière, in prep.)
• tweeting peaks shortly after publication
• of the published version in the journal of record (Eysenbach, 2011)
• of the e-print (Shuai, Pepe & Bollen, 2012)

4. Research questions
Twitter impact of e-prints and published versions

5. Research questions
Twitter impact of e-prints and published versions
• How many times are e-prints and published versions
tweeted?
• When does the Twitter activity occur?
• Do Twitter audiences differ between the two versions?
• To what extent are both activities picked up by Altmetric.com?
! Should both kinds of activities been taken into account or are
they equivalent?

6. Methods: direct match arXiv & Altmetric.com

7. Results: direct match arXiv & Altmetric.com

8. Methods: indirect match through WoS
exact&and&fuzzy&
• DOI&
• 0tles&
• author&names&
• abstracts&
• 0tle&length&

9. Results: indirect match through WoS
5.8%&documents&&
w/&>1&Altmetric&id&
+1,507&papers&
w/&more&tweets&
+4,600&TwiKer&counts&
+716&tweeted&
papers&

10. Results: indirect match through WoS

11. Preliminary findings
• presence of DOIs in arXiv metadata influences ability to
match Twitter activity of e-print and published paper
! enriching arXiv metadata with DOIs eliminates
potential biases
• some improvement through indirect match, but actual
differences not as large as expected
• +3.2% tweeted documents, +7.7% tweets
• +7.6% papers with increase in Twitter activity
! Altmetric.com finds most Twitter activity of papers with
arXiv e-print
! most tweets refer to arXiv id, not DOI

12. Preliminary findings
• differences between arXiv primary categories
• high Twitter rate in Quantitative Biology (8.4) and Computer
Science (5.0)
• high Twitter coverage in HEP Experiment (81.1%), HEP Theory
(72.7%), HEP Phenomenology (69.6%), HEP Lattice (69.2%)
• particular high coverage (44.9%) if compared to other
studies
! high presence of automated Twitter accounts!
bots and cyborgs triggered by arXiv feeds:
@hep_th
@hep_ph
@hep_ex
@hep_lat

13. Outlook: automated tweets
• search for “arXiv” in Twitter user name and description:
• many more automated accounts possible:
• journals
• publishers
! not equally distributed
! distribution instead of impact
account'type& number'(%)'
of'accounts&
tweets& mean'
followers&
mean'
following&
%'of'50,068'
tweets&
mean'Truthy'
BotOrNot'score&
arXiv&feed&(bot)& 43&(84.3%)& 87,389& 34.9& 0.6& 8.8%& 33%&
topic&feed&(bot)& 4&(7.8%)& 10,040& 527.0& 491.5& 0.1%& 40%&
selec0ve& 4&(7.8%)& 3,081& 361.8& 50.5& 1.0%& 46%&
'& 51'(100%)& 100,510& 99.1& 43.0& 9.9%& 33%&
• societies / associations
• institutions
• authors

14. Outlook: automated tweets
25% 100%
27%
• distinguishing type of tweet based on content
e.g., similarity with article title (%)
engagement distribution
100%
86%

15. Outlook: next steps
• detecting bots and cyborgs
• How much of Twitter activity in scholarly communication
do they account for?
• How are their tweets distributed?
• distinguishing between distribution and engagement
• answering our original research questions

16. Stefanie Haustein
Thank you for your attention!
Questions?
stefanie.haustein@umontreal.ca
@stefhaustein
Thanks to Euan Adie and
for access to their Twitter data!

17. References
Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Learned
Publishing, 26(1), 11-17.
Costas, R., Zahedi, Z. & Wouters, P. (2014). Do altmetrics correlate with citations? Extensive comparison of altmetric
indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Sciences and
Technology. arxiv: 1401.4321
Eysenbach, G. (2011) Can tweets predict citations? Metrics of social impact based on twitter and correlation with
traditional metrics of scientific impact. Journal of Medical Internet Research, 13, e123. doi: 10.2196/jmir.2012
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014b). Tweeting Biomedicine: An Analysis of
Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Sciences and
Technology, 65(4), 656-669. doi: 10.1002/asi.23101
Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the 73th Annual Meeting of
the American Society for Information Science and Technology, Pittsburgh, USA.
Pscheida, D., Albrecht, S., Herbst, Minet, C. & Köhler, T. (2013). Nutzung von Social Media und onlinebasierten
Anwendungen in der Wissenschaft. Erste Ergebnisse des Science 2.0-Survey 2013 des Leibniz-Forschungsverbunds
„Science 2.0“ available from: http://www.qucosa.de/fileadmin/data/qucosa/documents/13296/
Science20_Datenreport_2013_PDF_A.pdf
Rowlands, I., Nicholas, D., Russell, B., Canty, N., & Watkinson, A. (2011). Social media use in the research workflow.
Learned Publishing, 24, 183–195.
Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints: article
downloads, Twitter mentions, and citations. PLOS ONE, 7(11), e47523. doi:10.1371/journal.pone.0047523
Zahedi, Z., Costas, R. & Wouters, P. (2014). How well developed are altmetrics? cross-disciplinary analysis of the
presence of 'alternative metrics' in scientific publications. Scientometrics. doi: 10.1007/s11192-014-1264-0