A Journey Into the Emotions of Software Developers
ReMashed - An Usability Study of a Recommender System for Mash-Ups for Learning at ICL Conference 2009, Voillach, Austria
1. ReMashed –An Usability Study of a
Recommender System
for Mash-Ups for Learning
Hendrik Drachsler, Dries Pecceu, Tanja Arts, Edwin Hutten, Peter van Rosmalen,
Hans Hummel & Rob Koper
2. My background…
Learning Networks
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 2 | 23 September, 2009
3. Learning Networks
Explicitly address informal
learning
Learners can publish, share,
rate, tag and adjust their
own Learning Activities in a
Learning Network
Open Corpus that emerges
form the bottom upwards
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 3 | 23 September, 2009
4. Personal Environments
Nowadays …
More Information
Blog Reader Providers
Social
Bookmarking
Various
Communities
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 4 | 23 September, 2009
6. Selection problem because …
…of the amount
of data that is
emerging.
…learners can be
overwhelmed by
the plethora of
information.
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 6 | 23 September, 2009
7. Can we create a Recommender System
Today, Recommender Systems
for Mash-Ups for Learning?
supporting our decisions
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 7 | 23 September, 2009
8. What is ReMashed?
A Mash-up environment that allows you to
personalize emerging information of online
communities with a recommender system.
You tell what kind of Web 2.0 services you
use and then you are able to define which
contributions of other members you like
and do not like.
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 8 | 23 September, 2009
9. Goals for ReMashed
1. End-User level
Providing a recommender system for Web 2.0
sources of learners in informal Learning
Networks.
2. Researcher level
1. Offering researchers a system for the evaluation of
recommendation algorithms for learners in informal
Learning Networks.
2. Creating user-generated-content data sets for
recommender systems in informal Learning Networks.
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 9 | 23 September, 2009
10. How does it work?
ReMashed uses collaborative filtering to
generate recommendations.
It works by matching together users with
similar tastes (neighbours) on different
Web 2.0 resources (delicious, Flickr, blog
feeds, Slideshare, Twitter, and YouTube).
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 10 | 23 September, 2009
11. How does it work?
Cold-Start = Tag-based recommendation
Collaborative Filtering with ratings
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 11 | 23 September, 2009
13. The 1st Release
Recommendation
Algorithms
Database User Interface
of Items
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 13 | 23 September, 2009
14. System Evaluation
• At the TENCompetence Winterschool 2009
• Also external users sign up for the evaluation phase
• In total 49 people from 8 different countries
After this slide you
• created some
The evaluation phase ran for one month and was concluded
with an online recall questionnaire. really ugly ones do
you really want to
show them to the
• We received answers from 19 participants in total
audience?
(response rate of 38%).
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 14 | 23 September, 2009
15. Usage of Web 2.0 services
Strongly Agree Neutral Disagree Strongly
agree disagree
16% 5% 31% 11% 37%
1. Use a blog 21%
(n=3) (n=1) (n=6) (n=2) 48% (n=7)
2. Use social 42% 42% 6% 5% 5%
Bookmarking 84%
(n=8) (n=8) (n=1) (n=1) 10% (n=1)
5% 32% 21% 26% 16%
3. Share pictures 37%
(n=1) (n=6) (n=4) (n=5) 42% (n=3)
They just
4. Share 11% 26% 21% 21% twittered
17%
presentations 37%
(n=2) (n=5) (n=4) (n=4) 38% (n=5)
that the
5. Use micro- table is
11% 21% 10% 21% 37%
blogging
32%
(n=2) (n=4) (n=2) (n=4) 58% terrible.
(n=7)
0% 15% 38% 26% 21%
6. Upload videos 15%
(n=0) (n=3) (n=7) (n=5) 47% (n=4)
7. Use YouTube 5% 32% 15% 16% 32%
to collect videos 37%
(n=1) (n=6) (n=3) (n=3) 48% (n=6)
16. Satisfaction with ReMashed
Very Satisfied Unsatisfied Very Total
satisfied unsatisfied
1. Overall 5% 58% 26% 11% 100%
satisfaction (n=1) 63% (n=11) (n=5) 37% (n=2) (n=15)
2. Satisfaction tag-
20% 40% 27% 13% 100%
based algorithm
(n=3)
60% (n=6) (n=4)
40% (n=2) (n=16)
in the beginning
3. Satisfaction tag-
0% 69% 19% 13% 100%
based algorithm
(n=0) 69% (n=11) (n=3) 31% (n=2) (n=16)
at the end
4. Satisfaction
rating-based 8% 53% 31% 8% 100%
algorithm in the (n=1) 61% (n=7) (n=4) 39% (n=1) (n=14)
beginning
5. Satisfaction
rating-based 8% 54% 31% 8% 100%
algorithm at the (n=1) 62% (n=7) (n=4) 38% (n=1) (n=13)
end
17. Open Question
We received 9 answers:
• Integrate social networks like Linkedin, Facebook,
and MySpace.
• Integrate mind mapping tools like Mindmeister.
• Create clusters of Web 2.0 services of the same type,
e.g. combine Picassa and flickr in a subcategory
‘Pictures’.
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 17 | 23 September, 2009
19. The 2nd Release
DUINE Prediction
Engine
Database User Interface
of Items
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 19 | 23 September, 2009
20. You can use it as well!
Register at ReMashed
remashed.ou.nl. starts mashing.
Enter your favorite Taste your
Web 2.0 potatoes. personal
flavor of
Web 2.0.
Join the
community.
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 20 | 23 September, 2009
21. Future R&D
End-user perspective:
1. More Web 2.0 services and social networks
2. Widget / Portlet interface for other Personal Environments
3. Administration interface to create instances of the
ReMashed
Researcher perspective:
1. Creating data sets for Technology-Enhanced Learning
2. Exploring new recommendation technologies
3. Creating web services to offer recommendations to other
Mash Up Personal Learning Environments
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 21 | 23 September, 2009
22. Many thanks for your interest!
This slide is available here:
http://www.slideshare.com/Drachsler
Email: hendrik.drachsler@ou.nl
Skype: celstec-hendrik.drachsler
Blogging at: http://elgg.ou.nl/hdr/weblog
Twittering at: http://twitter.com/HDrachsler
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 22 | 23 September, 2009
23. Please sign up at:
http://remashed.ou.nl
hendrik.drachsler@ou.nl
International Conference on Interactive Computer Aided Learning, ICL 2009,Villach, AT
Page 23 | 23 September, 2009
Hinweis der Redaktion
A solution towards this problem might be recommender system technology. The main purpose of recommender systems on the Internet is to pre-select information a user might be interested in. For instance, the well-known company amazon.com [8] is using a recommender system to direct the attention of their users to other products in their collection. The motivation for a recommender system for Mash-Up Personal Learning Environments is to improve the ‘educational provision’; to offer a better goal attainment and to spend less time to find suitable learning material. Therefore, we developed a recommender system that offers advice to learners based on their Web 2.0 resources regarding the most suitable learning materials to meet their individual competence development.