4. What are “altmetrics”?
o “alternative metrics”
o new ways of measuring different, non-traditional
forms of impact, potentially of non-traditional
outputs.
o “alternative to only using citations”, not
“alternative to citations”.
o complementary to traditional citation-based
analysis.
5. Every researcher is a communicator
Within academia
Presentations and seminars
Funding and ethics applications
Academic books
Journal articles and posters
Term papers and essays
Meetings and conferences
Correspondence
Within society
Speaking at public events
Books for general audiences
Press
Social media
Blogs
Policy documents
We should measure both
6. New perspectives of impact
SOCIETAL IMPACT
ACADEMIC IMPACT
Journal Impact Factor
Citation counts
Traditional metrics
+
Download counts
Page views
Mentions in news reports
Mentions in social media
Mentions in blogs
Reference manager readers
… etc.
Alternative metrics
“altmetrics”
7. One available tool.
We are used a lot by
publishers, now some
institutions too.
We serve ± 2.5 million
requests a day.
12. Altmetrics
tools
don’t
(yet)
provide
good
metrics
for
impact
BUT
They
can
help
you
find
evidence
of
impact,
successes
13. Evidence of public outreach?
An article on the ecological
impacts of the Fukushima
nuclear accident.
• > 1,859 twitter accounts
shared, combined follower
count of 2.5M.
• 68% of tweets sent from
Japan.
Figure 1
• 77% of tweets from members
of the public.
2012, Scientific Reports 2, 570
18. PLOS
ALM
Reports:
An
exploratory
review
Engagement/Influence
beyond
citaEons
19. PLOS
ALM
Reports:
An
exploratory
review
Engagement/Influence
beyond
citaEons
q No
cita3ons
in
3
months
since
publica3on.
q However
TwiJer
men3ons
70x
the
average
ar3cle
in
our
dataset.
q
4x
the
average
for
PLOS
Medicine
ar3cles
in
2013.
20. Engagement/Influence
beyond
citaEons
MEP
Centre
for
Bioethics
MEP
Professor
of
EBM
Journal
editor
Health
journalist
NGO
Health,
PopulaEon
&
NutriEon
@
The
World
Bank
21. Monitoring
progress:
WT’s
key
indicators
Outcomes
Key
indicators
of
progress
Discoveries
1.
2.
significant
advances
in
the
genera3on
of
new
knowledge
contribute
to
discoveries
with
tangible
impacts
on
health
ApplicaEons
Engagement
3.
contribute
to
the
development
of
enabling
technologies,
products
and
devices
uptake
of
research
into
policy
and
prac3ce
Research
leaders
Research
environment
Influence
4.
5.
6.
7.
8.
enhanced
level
of
informed
debate
in
biomedicine
significant
engagement
of
key
audiences
&
increased
reach
develop
a
cadre
of
research
leaders
evidence
of
significant
career
progression
among
those
we
support
10.
key
contribu3ons
to
the
crea3on,
development
and
maintenance
of
major
research
resources
contribu3ons
to
the
growth
of
centres
of
excellence
11.
12.
significant
impact
on
science
funding
&
policy
developments
significant
impact
on
global
research
priori3es
and
processes
9.
25. In general, altmetrics numbers…
X
Don’t represent the quality of
research.
X
Don’t indicate the quality of
individual researchers.
X
Don’t tell the whole story –
always look for qualitative
data as well
29. Problems
• 30
–
40%
of
recent
biomedical
papers
will
have
Altmetric
aJen3on.
But
<
10%
in
social
sciences.
30. Problems
• 30
–
40%
of
recent
biomedical
papers
will
have
Altmetric
aJen3on.
But
<
10%
in
social
sciences.
• Tools
have
subtle
bias:
data
sources
are
mainly
those
popular
in
US,
Europe
33. Disciplinary differences
and other biases
Exploring social media metrics in scholarly context
Stefanie Haustein
stefanie.haustein@umontreal.ca
@stefhaustein
35. Altmetrics: definitions
• term coined by Jason Priem
• introduced as a better filter
than and alternative to
citations and peer-review
http://altmetrics.org/manifesto/
• “…altmetrics is a good idea,
but a bad name”
“…we would like to propose
the term influmetrics”
Rousseau & Ye (2013)
• rather complementary than
•
alternative to citations
social media metrics
36. Altmetrics: definitions
• ultimate goals
• similar to but more timely than citations
Ø predicting scientific impact
• different, broader impact than captured by citations
Ø measuring societal impact
• impact of various outputs
Ø “value all research products”
Piwowar (2013)
37. Altmetrics: definitions
• Altmetrics are “representing very different things”
(Lin & Fenner, 2013)
• unclear what exactly they measure:
•
•
•
•
scientific impact
social impact
“buzz”
all of the above?
40. Altmetrics: definitions
• complex to define and classify tools and motivations
• scientific and non-scientific audiences cannot be
determined on the platform used
• level of engagement differs not only between
platforms but also within:
saving paper to Mendeley library vs. tweeting about it
saving vs. reading
retweeting link vs. discussing content
Ø differentiation between audiences and engagement
needed to determine meaning of metrics
41. Bibliometrics: in retrospect
• when Garfield created SCI, sociologists of science
analyzed meaning of publications and citations
(Merton, Zuckerman, Cole & Cole, etc.)
• sociological research
• What is it to publish a paper?
• What are the reasons to cite?
• empirical bibliometric research
• disciplinary differences in publication
and citation behavior
• delay and obsolescence patterns
42. Bibliometrics: in retrospect
• empirical studies helped sociologists to understand
structure and norms of science
• for bibliometricians, studies provided a theoretical
framework and legitimation to use citation analysis
in research evaluation
• knowledge about disciplinary differences and
obsolescence patterns helped to normalize statistics
and create more appropriate indicators
43. Bibliometrics: in retrospect
• similar to development of SCI in the 1960s, social
media metrics have to be analyzed:
• qualitative studies to analyze who, how and why
people use various social media platforms
• large-scale quantitative studies to determine
differences and biases in terms of disciplines, topics,
document types, publications years, publication types
and sources, author age and affiliation, etc.
Ø to find out what various social media metrics mean
and what they can be used for
51. Altmetrics: disciplinary biases
x-axis:
coverage of
specialty on
platform
y-axis:
correlation
between social
media counts
and citations
bubble size:
intensity of use
based on mean
social media
count rate
52. Altmetrics: subject bias
General Biomedical Research papers 2011
Scatterplot of number of citations and number of tweets (A, ρ=0.181**) and Mendeley readers (B, ρ=0.677**),
bubble size represents number of Mendeley readers (A) and tweets (B). The respective three most tweeted (A)
and read (B) papers are labeled showing the first author.
53. Altmetrics: subject bias
Top 10 tweeted documents:
catastrophe & topical / web & social media / curious story
scientific discovery / health implication / scholarly community
Article
Journal
C
T
Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients
is associated with exposure to low-dose irradiation
PNAS
9
963
Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the
Fukushima nuclear accident
PNAS
30
639
Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at
Our Fingertips
Science
11
558
Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator
Journal of Physical
Chemistry A
--
549
Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes
Cutis
--
477
Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life
expectancy: a prospective cohort study
Lancet
51
419
Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations
Journal of Sexual
Medicine
--
392
Newman & Feldman (2011). Copyright and Open Access at the Bedside
New England
Journal of Medicine
3
332
Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells
Science
5
323
Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA
receptor expression in a mouse via the vagus nerve
PNAS
31
297
54. Altmetrics: future
• before applying social media counts in information
retrieval and research evaluation, we need:
Ø to understand and define meaning of various
social media metrics
Ø to identify different biases
Ø to differentiate between audiences and
level of engagement
Ø more transparency and reliability in data aggregation
55. References
Bar-Ilan, J. (2011). Articles tagged by 'bibliometrics' on Mendeley and CiteULike. Paper presented at the Metrics 2011 Symposium on
Informetric and Scientometric Research, New Orleans, Louisiana.
Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars' visibility on the social web.
In Proceedings of the 17th International Conference on Science and Technology Indicators, (pp. 98-109).
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press).
Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information
Science and Technology.
Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of
tweeting and scientific publication behavior. Aslib Proceedings.
jasonpriem (2010, September 28).
I like the term #articlelevelmetrics, but it fails to imply *diversity* of measures. Lately, I'm liking #altmetrics. [Twitter post].
Li, X. & Thelwall, M. (2012). F1000, Mendeley and Traditional Bibliometric Indicators. In Proceedings of the 17th International Conference
on Science and Technology Indicators, (pp. 541-551).
Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2),
461-471.
Lin, J. & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards
Quarterly, 25(2), 20-26.
Mohammadi, E., & Thelwall, M. (in press). Mendeley readership altmetrics for the social sciences and humanities: Research evaluation
and knowledge flows. Journal of the American Society for Information Science and Technology.
Piwowar, H. (2013). Value all research products. Nature, 493(7431), 159.
Priem, J., Piwowar, H., & Hemminger, B.M. (2012). Altmetrics in the wild: Using social media to explore scholarly impact. arXiv.
Priem, J., Taraborelli, D., Groth, P. & Neylon, C. (2010). Alt-Metrics: A Manifesto.
Rousseau, R., & Ye, F.Y. (2013). A multi-metric approach for research evaluation. Chinese Science Bulletin, 58(26), 3288-3290.
Zahedi, Z., Costas, R., & Wouters, P. (2013).
What is the impact of the publications read by the different Mendeley users? Could they help to identify alternative types of impact?
56. Thank you for your attention!
Questions?
Stefanie Haustein
stefanie.haustein@umontreal.ca
@stefhaustein
57. Scholarly
connecEons
Cita3ons,
social
media,
ORCID
and
authorship
networks
Mike
Taylor
Research
Specialist
hJp://orcid.org/0000-‐0002-‐8534-‐5985
@herrison
58. The
number
of
possible
connec3ons
between
researchers
and
ar3cles,
researchers
and
researchers,
and
ar3cles
and
ar3cles
is
accelera3ng
drama3cally.
Although
bibliometrics
has
been
studied
for
50
years,
the
study
of
these
new
connec3ons
has
only
been
undertaken
recently.
As
infrastructure
is
built
to
accommodate
this
massively
connected
world,
so
research
becomes
enabled
and
desirable.
Part
1
–
formal
links
Part
2
–
informal,
ad
hoc
links
67. End
of
part
1
Known
facts:
A
person
writes
an
ar3cle
A
person
reads
&
cites
other
ar3cles
A
person
writes
an
ar3cle
with
another
person
-‐
Bibliometrics
QuesEons
about
facts:
Not
everyone
did
enough
to
be
an
‘author’
-‐
Ethics,
acknowledgement
statements,
contributorship
Problems
about
facts:
Some3mes
people
in
the
same
field
have
the
same
name
Some3mes
people
with
the
same
name
write
the
same
paper
Some3mes
people
with
the
same
name
get
credit
for
papers
they
didn’t
write
-‐ ORCID,
over
300,000
ORCIDs
aler
a
year,
eg,
Elsevier's
editorial
system
has
over
100,000
ar3cles
in
produc3on
with
ORCIDs
69. A
person
does
X
with
an
ar3cle
pins
tweets
Writes
a
blog
Saves
on
delicious
Re-‐uses
Facebooks
Saves
on
Mendeley
Writes
a
newspaper
ar3cle
70. Different
kinds
of
outputs
Saves
on
delicious
pins
Facebooks
tweets
Re-‐uses
Saves
on
Zotero
/
Mendeley
/
Citeulike
/
biblio
tool
Writes
a
newspaper
ar3cle
Writes
a
blog
76. End
of
part
two
Known
facts:
There
are
more
connec3ons
now
than
have
ever
been
Of
more
types
than
ever
Crea3on
is
ad
hoc,
post
hoc,
technocra3c,
automa3c,
pragma3c,
real-‐3me…
We
can
count
things
we
don’t
understand
Emergent
thoughts:
An
ORCID
can
be
seen
as
a
document
about
a
person
Links
between
documents
can
be
formed
with
no
human
cura3on
(seman3c
web)
Altmetrics
gives
us
a
view
into
the
world
of
connecEons,
as
a
very
limited
starEng
point:
We
need
research
into
meaning
and
correla3on
before
we
can
make
conclusions
–
researchers
Issues
of
iden3ty,
privacy,
seman3cs,
authorship
/
contributorship,
cura3on
are
all
in3mately
bound
with
altmetrics
77. Appendix:
seven
use
cases
for
altmetrics
1. Predic3on
of
ul3mate
cita3on
2. Measuring
/
recognizing
component
re-‐use
/
preparatory
work
/
reproducibility
3. Hidden
impact
(impact
without
cita3on)
4. Real-‐3me
filtering
/
real-‐3me
evalua3on
(sigint)
5. Plaporm
/
publisher
/
ins3tu3on
comparison
6. Measuring
social
reach
/
es3ma3ng
social
impact
7. Altmetrics
is
of
interest
by
itself
78. !
!
!
NISO Webinar:
New Perspectives on Assessment How Altmetrics
Measure Scholarly Impact
!
!
Questions?!
All questions will be posted with presenter answers on the
NISO website following the webinar:!
!
http://www.niso.org/news/events/2013/webinars/altmetrics
NISO Webinar • November 13, 2013
79. THANK YOU
Thank you for joining us today.
Please take a moment to fill out the brief online survey.
We look forward to hearing from you!