Slides from my successful dissertation defense. The research focused on the role of technologies in supporting participation and organizing processes in citizen science projects, and the impacts of these processes on scientific outcomes.
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Crowdsourcing Scientific Work: A Comparative Study of Technologies, Processes, and Outcomes in Citizen Science
1. Crowdsourcing
Scientific
Work
A
Comparative
Study
of
Technologies,
Processes,
and
Outcomes
in
Citizen
Science
Andrea
Wiggins
11
April,
2012
Kevin
Crowston
(Advisor)
Geof
Bowker
(External
Reader)
Rick
Bonney
Murali
Venkatesh
(Internal
Reader)
Jian
Qin
John
Burdick
(Chair)
Steve
Sawyer
Tuesday, May 15, 12
2. Citizen
Science
• Projects
involving
the
public
with
scientists
in
collaborative
research.
-‐ Crowdsourcing
scientific
work
of
data
collection
and
processing
-‐ Increasingly
ICT-‐mediated
• As
citizen
science
gains
in
popularity,
scientists
need
a
better
understanding
of
how
design
and
management
influence
scientific
outcomes,
particularly
for
ICT-‐enabled
participation.
• Research
goals
-‐ Describe
the
phenomenon
of
citizen
science.
-‐ Develop
an
empirically-‐grounded
framework
that
describes
the
conditions,
processes,
and
products
of
citizen
science
projects.
Tuesday, May 15, 12
3. Related
Research
• Public
participation
in
science
-‐ Purposes
and
forms
of
engagement public
participation
-‐ Informal
science
education,
policy,
STS in science
cro r
tee g
•Irwin;
Bonney
et
al;
Cooper
et
al;
Wilderman so wd-
urc n n
olu itori
ing v n
mo
• Scientific
collaboration *
infrastructure
online scientific
cyber-
communities collaboration
-‐ Broader
context
of
practice
•Sonnenwald;
Finholt;
Lawrence
et
al
• Online
communities
* = citizen science
-‐ Participation
in
virtual
environments
•Crowston;
Haythornthwaite;
Preece
&
Shneiderman
Tuesday, May 15, 12
4. Research
Questions
How
do
virtuality
and
technology
alter
organizing
in
citizen
science?
How
do
virtuality
and
technology
shape
participation
in
citizen
science?
How
do
organizing
and
participation
influence
scientific
outcomes
in
citizen
science?
Tuesday, May 15, 12
6. Mountain
Watch
• Monitoring
alpine
climate
change
-‐ Participation
involves:
•Finding
monitoring
plots
•Identifying
target
plants
and
their
phenophases
•Recording
observations
on
paper
•Dropping
off
data
sheet
at
facilities
or
entering
online
-‐ Started
in
2004
by
the
Appalachian
Mountain
Club
•Primarily
in
White
Mountains
of
New
Hampshire
•Combines
citizen
science
with
other
research
efforts
•Intensive
study
of
factors
influencing
data
quality
Tuesday, May 15, 12
7. Great
Sunflower
Project
• Collecting
data
on
pollinator
service
(bees!)
-‐ Participation
involves:
•Planting
sunflowers
•Creating
garden
description
on
Drupal
website
•Recording
15-‐minute
observation
samples
on
data
sheet
•Online
data
entry
-‐ Started
in
2008
by
a
single
academic
researcher
•Collects
data
across
North
America
•Very
successful
in
attracting
volunteer
interest
Tuesday, May 15, 12
8. eBird
• Collecting
bird
abundance
and
distribution
data
-‐ Participation
involves:
•Choosing
observation
methods
•Recording
bird
observations
•Entering
observations
and
metadata
online
-‐ Launched
in
2002
by
Cornell
Lab
of
Ornithology
(with
National
Audubon
Society)
•World’s
largest
biodiversity
data
set
•Receives
between
2.5M
-‐
3M
observations/month
•Data
used
in
both
research
and
decision-‐making
for
policy
and
land
management
Tuesday, May 15, 12
9. Comparative
Case
Selection
Criterion Mountain
Watch Great
Sunflower eBird
Conservation,
Research,
education,
Mission Research,
education
education,
recreation conservation
Purpose
Scientific
Climate
change
effects
Bird
abundance
&
Plant-‐bee
relationships
interests on
alpine
habitats distribution
Intended
Hikers Gardeners Birders
Community
Institutions Single
nonprofit Academic Nonprofit
partnership
Environment
Resources 1.5
FTE,
$15K 0.5
FTE,
$13K 4.5
FTE,
$300K
Paper Structured
data
sheet Structured
data
sheet Variable
&
optional
Organization
website
Open
source
CMS
Purpose-‐built
software
Technologies Digital section website system
Data
access Limited Very
limited Extensive
Tuesday, May 15, 12
11. Data
Collection
• Semi-‐structured
interviews
with
project
organizers
-‐ Sampled
for
maximum
diversity
of
roles
and
perspectives,
with
individuals
from
7
organizations
-‐ Some
longitudinal
interviews,
additional
informal
interviews
• Participant
observation
-‐ 300+
hours
of
birding,
3
years
of
sunflowers,
6
days
in
the
White
Mountains
-‐ Listservs,
forums,
beta
testing
interfaces
&
mobile
application
-‐ Extensive
involvement
in
citizen
science
organizer
community
• Secondary
data,
documents,
&
artifacts
Tuesday, May 15, 12
12. Analysis
• Concurrent
with
data
collection
and
theory
development
-‐ Iterative
deductive
and
inductive
coding Commitment
Sustainability Satisfaction
-‐ Rich
process
models Scientific Contributions
Interests
Individual
-‐ Concept
diagrams
Community
Development
Resources
Scientific
• Research
Quality
Institutions
Knowledge
Mission Broader
Impacts
-‐ Interviewees
reviewed
transcripts
Technologies
Science
Skills
-‐ Key
informants
reviewed
case
chapters Biography
Design
Organizing
Networks
-‐ Expert
and
peer
review
of
findings Personal Interests Participation
-‐ Audit
trail,
ongoing
memos
-‐ Data
triangulation
Tuesday, May 15, 12
14. Theoretical
Framework
• Iteratively
developed
Organizational
-‐ Initial
version
based
on
Emergent
States
Community
literature,
used
to
guide
study Sustainability
Individual
-‐ 16
versions
over
3
years Emergent
States
Organizational Commitment
Inputs Individual Individual
• Inputs-‐Moderators-‐Outputs-‐
Inputs Roles Organizational
Outputs
Task Design Outputs
Demographics Contributions Knowledge
Technology
Inputs
structure Design
Organization
Skills
Motivation
Individual
Processes
Satisfaction
Learning
Communication
Innovation
Design
Joining
• Example
of
a
relevant
flow: Contributing
Organizational
-‐ Design
&
Organizing
-‐>
Processes
Scientific
Participation
-‐>
Research
Volunteer
Contributions
-‐>
Management
Data
Management
Scientific
Knowledge
Tuesday, May 15, 12
15. Theoretical
Framework
Environment
• Iteratively
developed
-‐ Initial
version
based
on
Inputs States Products
literature,
used
to
guide
study Project Inputs
Sustainability
Scientific Outcomes
Interests Commitment Scientific
-‐ 16
versions
over
3
years Community Satisfaction Knowledge
Broader
Resources
Impacts
Institutions
• Inputs-‐Moderators-‐Outputs-‐ Mission
Technologies
Inputs
structure
• Example
of
a
relevant
flow: Individual
Inputs
Outputs
Processes Contributions
Skills
-‐ Design
&
Organizing
-‐>
Science Individual
Biography
Development
Networks Design
Participation
-‐>
Personal
Interests
Organizing
Contributions
-‐>
Participation
Scientific
Knowledge
Tuesday, May 15, 12
16. Theoretical
Framework
Environment
• Iteratively
developed
-‐ Initial
version
based
on
Inputs States Products
literature,
used
to
guide
study Project Inputs
Sustainability
Scientific Outcomes
Interests Commitment Scientific
-‐ 16
versions
over
3
years Community Satisfaction Knowledge
Broader
Resources
Impacts
Institutions
• Inputs-‐Moderators-‐Outputs-‐ Mission
Technologies
Inputs
structure
• Example
of
a
relevant
flow: Individual
Inputs
Outputs
Processes Contributions
Skills
-‐ Design
&
Organizing
-‐>
Science Individual
Biography
Development
Networks Design
Participation
-‐>
Personal
Interests
Organizing
Contributions
-‐>
Participation
Scientific
Knowledge
Tuesday, May 15, 12
17. Theoretical
Framework
Environment
• Iteratively
developed
-‐ Initial
version
based
on
Inputs States Products
literature,
used
to
guide
study Project Inputs
Sustainability
Scientific Outcomes
Interests Commitment Scientific
-‐ 16
versions
over
3
years Community Satisfaction Knowledge
Broader
Resources
Impacts
Institutions
• Inputs-‐Moderators-‐Outputs-‐ Mission
Technologies
Inputs
structure
• Example
of
a
relevant
flow: Individual
Inputs
Outputs
Processes Contributions
Skills
-‐ Design
&
Organizing
-‐>
Science Individual
Biography
Development
Networks Design
Participation
-‐>
Personal
Interests
Organizing
Contributions
-‐>
Participation
Scientific
Knowledge
Tuesday, May 15, 12
18. Theoretical
Framework
Environment
• Iteratively
developed
-‐ Initial
version
based
on
Inputs States Products
literature,
used
to
guide
study Project Inputs
Sustainability
Scientific Outcomes
Interests Commitment Scientific
-‐ 16
versions
over
3
years Community Satisfaction Knowledge
Broader
Resources
Impacts
Institutions
• Inputs-‐Moderators-‐Outputs-‐ Mission
Technologies
Inputs
structure
• Example
of
a
relevant
flow: Individual
Inputs
Outputs
Processes Contributions
Skills
-‐ Design
&
Organizing
-‐>
Science Individual
Biography
Development
Networks Design
Participation
-‐>
Personal
Interests
Organizing
Contributions
-‐>
Participation
Scientific
Knowledge
Tuesday, May 15, 12
19. Theoretical
Framework
Environment
• Iteratively
developed
-‐ Initial
version
based
on
Inputs States Products
literature,
used
to
guide
study Project Inputs
Sustainability
Scientific Outcomes
Interests Commitment Scientific
-‐ 16
versions
over
3
years Community Satisfaction Knowledge
Broader
Resources
Impacts
Institutions
• Inputs-‐Moderators-‐Outputs-‐ Mission
Technologies
Inputs
structure
• Example
of
a
relevant
flow: Individual
Inputs
Outputs
Processes Contributions
Skills
-‐ Design
&
Organizing
-‐>
Science Individual
Biography
Development
Networks Design
Participation
-‐>
Personal
Interests
Organizing
Contributions
-‐>
Participation
Scientific
Knowledge
Tuesday, May 15, 12
20. Theoretical
Framework
Environment
• Iteratively
developed
-‐ Initial
version
based
on
Inputs States Products
literature,
used
to
guide
study Project Inputs
Sustainability
Outcomes
? Commitment
?
-‐ 16
versions
over
3
years Community Satisfaction
Broader
Resources
Impacts
Institutions
• Inputs-‐Moderators-‐Outputs-‐ Mission
Technologies
Inputs
structure
• Example
of
a
relevant
flow: Individual
Inputs
Outputs
Processes Contributions
Skills
-‐ Design
&
Organizing
-‐>
Biography
Networks
?
Design
Individual
Development
Participation
-‐>
Personal
Interests
Organizing
Contributions
-‐>
Participation
Scientific
Knowledge
Tuesday, May 15, 12
21. Emergent
Themes
1. Project
design
approaches
that
favor
science
versus
hobbies
for
participation
design
2. Design
and
organizing
implications
of
engaging
communities
of
practice
3. Relationships
between
physical
environment,
technologies,
participant
experiences,
and
data
quality
4. Information
technology
tradeoffs:
helpful
for
scale
and
communication,
challenging
for
usability
and
resources
5. Resources
and
sustainability
relate
to
institutions
and
scale
of
participation
Tuesday, May 15, 12
22. How
do
virtuality
and
technologies
alter
organizing
in
citizen
science?
• Virtuality
is
inherent
and
a
key
benefit,
but
leads
to
questions
about
quality
-‐ “People
would
gravitate
towards
the
really
charismatic
species,
which
in
the
White
Mountains
is
diapensia.
So
people
would
go
out
with
these
diapensia-‐
tinted
glasses,
and
they’d
see
it
everywhere
and
pass
over
the
least
well-‐known
species.”
• Enables
large-‐scale
research
that
is
more
like
crowdsourcing
than
other
forms
of
scientific
collaboration
-‐ “If
technology
makes
new
things
available,
you
change
your
focus
to
exploit
it.”
• Reduces
coordination
costs
and
improves
quality,
but
ICT
often
unsuited
for
use
in
the
field
-‐ “Someone
entered
in
data
that
said
that
they
saw
a
bee
after
130
minutes,
and
I
think
what
they
were
putting
in
is
that
it
was
at
1:30
in
the
afternoon.”
Tuesday, May 15, 12
23. How
do
virtuality
and
technologies
shape
participation
in
citizen
science?
• Opens
participation
opportunities
to
larger,
more
diverse
population
-‐ “The
skill
base
varies
from
Master
gardeners
and
beekeepers
to
amateur
first-‐time
gardeners.
...
Our
audience
skews
a
little
older.
There
are
far
fewer
schoolchildren
who
participate
than
I
thought
there
might
be.”
• Importance
of
place:
geographic
biases
and
autonomy,
functional
constraints
of
and
emotional
relationships
to
place
-‐ “Folks
do
have
a
real
connection
to
these
mountains.
So
to
feel
like
they
can
do
something
to
help
out,
and
to
protect,
and
get
a
handle
on
what
is
actually
happening
up
here
in
the
mountains,
it’s
valuable.”
• Leads
to
usability
issues
for
some,
but
can
also
be
rewarding
and
more
scalable
-‐ “Some
people
have
difficulty
printing
out
the
data
form,
and
writing
all
this
stuff
in
while
they’re
observing,
and
taking
it
back,
and
then
entering
it
in.”
-‐ “Let’s
give
them
tools
to
do
what
they
want,
and
they’ll
give
us
all
of
their
data.”
Tuesday, May 15, 12
24. How
do
organizing
and
participation
influence
scientific
outcomes
in
citizen
science?
• Diverse
types
of
scientific
outcomes
suggest
more
holistic
criteria
for
evaluating
project
success
-‐ “[eBird]
is
just
getting
to
the
point
where
we
are
going
to
see
more
and
more
information
come
out
that
will
help
drive
policy
and
decision-‐making.”
• Keep
participants
happy:
greater
quality
and/or
quantity
of
contributions
improve
outcomes
-‐ “The
more
people
enjoy
the
project
and
get
some
reward
then
the
better
off
you’ll
be
for
sustaining
it.
We’ve
seen
significant
growth
that
hasn’t
slowed
down
since
we
turned
the
switch
on
and
changed
the
way
we
think
about
it.”
Tuesday, May 15, 12
25. Limitations
&
Future
Work
• Limitations
-‐ Depth
rather
than
breadth
-‐ Focused
primarily
on
organizers
• Future
work
-‐ Integrate
findings
and
framework
with
participant-‐oriented
studies
-‐ Compare
to
entirely
online
citizen
science
projects
-‐ Work
with
organizer
community
to
translate
findings
into
recommendations
for
practice
Tuesday, May 15, 12
26. Contributions
• Theoretical
framework
-‐ Complements
and
extends
prior
models
-‐ Foundation
for
future
research
with
room
for
expansion
&
refinement
• Case
studies
-‐ In-‐depth
description
and
comparison
• New
prospective
best
practices
-‐ Sustainability
planning
in
context
of
organizations
and
resources
-‐ Aligning
scientific
and
personal
interests
as
much
as
possible
-‐ Making
explicit
links
between
individuals,
communities,
and
organizing
-‐ Engaging
non-‐scientist
community
members
as
organizers
Tuesday, May 15, 12
27. Thanks
• Committee
• Case
study
projects
-‐ Mountain
Watch
-‐ Great
Sunflower
Project
-‐ eBird
• Writing
group
-‐ Mohammad
Jarrahi
&
Jaime
Snyder
• Everett
Wiggins
• U.S.
National
Science
Foundation
Grants
09-‐43049
&
11-‐11107
Tuesday, May 15, 12