1. USING SMARTPHONES AS PERSONAL
MONITORING TOOLS FOR THE ACOUSTIC
ENVIRONMENT!
João Cordeiro!
Álvaro Barbosa!
School of Arts - Portuguese Catholic University – C. R. Porto!
CITAR - Research Centre for Science And Technology of The Arts (Porto)!
Faculty of Creative Industries - University of Saint Joseph – Macau, China!

2. Outline!
• Main goals!
• Problem statement!
• System design!
• Experimental procedure!
• Results!
• Conclusions!
• Future work!

3. Main goals!
Better
quality
of life!
Achieve better
soundscapes in
urban areas!
Increase awareness for the
acoustic phenomenon!

4. Problem statement!
How to promote
sound awareness? !
Education!
Music > Sound !
Sound walks!
Sound design!
Public Politics!
Campaigns !
Laws!
Training!
Technology!
Entertaining / Useful / Social!

5. Problem statement!
Entertaining / Useful / Social MOBILE Technology!
!!
Advantages!
!- People are keen to new technologies!
!- Mostly mobile-based!
!- Powerful computing processing!
!- Feature several sensors (microphones, camera, accelerometer…)!
!- Open to developers!
!- …!
!
Disadvantages!
!- Not rigorously calibrated!
!- Not multitasking friendly!
!- Power autonomy!
!- Privacy issues !!
!- Latency communication!
!- …!
!

6. Problem statement!
!
!
>> SOUND BASED <<!
!
!
Audio Games!
(ex: Murky Shooting)!
!
!
Entertaining / Useful / Social MOBILE Technology!

7. Problem statement!
!
!
>> SOUND BASED <<!
!
!
Audio Games!
!
!
Sound Maps!
(ex. Freesound uploads) !
!
!
Entertaining / Useful / Social MOBILE Technology!

8. Problem statement!
!
!
>> SOUND BASED <<!
!
!
Audio Games!
!
!
Sound Maps!
!
!
Sound-Based Social Networks!
(ex. Hurly-Burly)!
Entertaining / Useful / Social MOBILE Technology!

9. Problem statement!
Can we use soundscapes to improve
Online Social Networking?!
(thus creating an engaging service based on sound)!

10. System design!
Hurly-Burly!
(means: boisterous activity) !
!
• senses the sound environment surrounding each user!
• classifies it according to the sound sources!
• measures the sound pressure level!
• implements a online social network!
• distributes the sound metadata through the social network!
• keep a record of user’s soundscapes automatic metadata!

11. System design!

12. System design!
• iOS platform!
• Senses the soundscapes using built
in microphone!
• Classification: Music, Speech,
Environmental Sounds, Silence!
• Measures SPL – dB(A)Leq, 3s!
• Continuous Monitoring (background)!
• Online Social Network!
• Movement Sensing!

13. System design!

14. System design!
• MySQL database!
• PHP web interface!
• Users’ soundscape logs
!- timestamp
!- soundscape type
!- SPL!
• Relational table (who is friends with who)!

15. System design!
• Interactive timeline graph with users’
soundscapes and movement data!
• Affords horizontal zoom in/out and
timeline “traveling”!

16. System design!

17. Experimental procedure!
• Test the system with 3 test groups!
• Total 23 users + 2 control devices!
• 3 days each test!
• Surveys and data logs were collected!
Each group is a
Social Network!
with Cluster
Coefficient = 1!

18. Results!
1
10
2
1
5
1
1
1
1
USJ
UCP
HB
Portuguese
Philippine
Chinese
Israeli
Spanish
Brazilian
Czech
Sample characterization | nationality !

19. Results!
Sample characterization | age distribution!
0
1
2
3
4
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Number
of
subjects
Age
range
USJ
UCP
HB

20. Results!
Sample characterization | frequency of Social Media usage !
18
2
2
1
Everyday
Twice
a
week
Once
a
month
I'm
no
sure

21. Results!
Sample characterization | favourite way to access Social Media!
35%
48%
13%
4%
Mobile
devices
(smartphones,
iPods...)
Laptops
Tablets
Desktops

22. 3
3
4
7
9
13
14
17
Who
is
with
me
My
Loca[on
What
music
Am
I
Videos
Audio
Things
I
Like
Textual
messages
Photos
Results!
Sample characterization | what users prefer to share on Social Media!
non-contextual cues !

23. Results!
Experiment characterization | users proximity with the device during exp.!
48%
52%
Yes,
I
carried
the
device
always
with
me
Most
of
the
[me,
but
not
always
No,
We
were
separate

24. Results!
Experiment characterization | devices used to run the application!
65%
18%
17%
iPhone
iPod
iPad

25. Results!
Experiment characterization | application performance per device!
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
iphone
ipod
ipad
Major
Porblems
Minor
Porblems
Excelent

26. Results!
User experience | what was the first reason to switch of the application!
1
10
3
0
0
0
All
the
above
Low
ba_ery
Privacy
issues
Keeps
my
device
slow
I
don't
likethe
red
bar
I'm
not
sure

27. Results!
User experience | audio clips v.s. soundscapes automatic metadata!
3
7
9
4
I
would
use
it,
I
think
it
increases
the
sense
of
realism
of
the
social
experience
I
might
use
it
with
cau[on
I
would
use
it
only
in
very
controlled
situa[ons.
I
would
never
use
an
applica[on
like
that,
it
violates
my
privacy

28. Results!
User experience | users’ knowledge of the term soundscape !
70%
4%
26%
Yes
No
More
or
less

29. Results!
User experience | hurly-burly systems as environmental gauges!
2
7
14
I
doubt
I
guess
so
Yes,
definitely

30. Results!
User experience | choices according the knowledge of soundscapes !
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
I
know
More
or
Less
I
Doubt
Maybe
Define_ly

31. Results!
Soundscape history | Example 1 – two days with the same pattern!

32. Results!
Soundscape history | Example 2 – office at CITAR – 3 weeks log!

33. Results!
Soundscape history | Example 2a – detail – 10h to 2h (late meeting)!

34. Results!
Soundscape history | Example 3 – 1,5 days - Macau!

35. Conclusions!
• Mobile devices are a great opportunity for sensing the environment,
however, multitasking and power constraints may hinder this use!
• Soundscapes provide an interesting description of people’s activity,
which can be applied to many areas where this information is relevant,
for example Social Networking!
• Recording and sharing audio clips may hinder the use of sound in
Social Networking, due to privacy issues. Classifying the sound with
machine audition techniques represents a good option!
• There’s a relation between knowing the concept of soundscape and
acknowledging it as an important resource. Thus an increase of
education on this field will lead to more conscientious citizens about
their sound environment!

36. Future work!
• Port the application for other platforms (Android)!
• Enrich the study with more users!
• Interleave the application with existent Online Social Networks (e.g.
Facebook, Google +, Twitter…)!
• Developing other technological applications that may lead to an
increase of the sound awareness among non-experts!
• …!

37. Questions?!

38. Thank you!!
João Cordeiro!
Álvaro Barbosa!
School of Arts - Portuguese Catholic University – C. R. Porto!
CITAR - Research Centre for Science And Technology of The Arts (Porto)!
Faculty of Creative Industries - University of Saint Joseph – Macau, China!