Emotions in Argumentation: an Empirical Evaluation @ IJCAI 2015
•Download as PPTX, PDF•
4 likes•1,939 views
IJCAI 2015 of the first result of the SEEMPAD project on emotions in argumetation.
Recommended
Recommended
More Related Content
Similar to Emotions in Argumentation: an Empirical Evaluation @ IJCAI 2015
Similar to Emotions in Argumentation: an Empirical Evaluation @ IJCAI 2015 (20)
More from Fabien Gandon
More from Fabien Gandon (20)
Recently uploaded
Recently uploaded (20)
Emotions in Argumentation: an Empirical Evaluation @ IJCAI 2015
- 1. Emotions in Argumentation an Empirical Evaluation Sahbi Benlamine, Maher Chaouachi, Claude Frasson Serena Villata, Elena Cabrio, Fabien Gandon
- 2. Web Agora
- 3. web
- 5. social web
- 6. argument web
- 7. debate web
- 8. emotion web
- 9. semantic web
- 10. debate traces
- 11. question: Connection between the arguments proposed by the participants of a debate and their emotional status? • correlation of polarity of arguments and polarity of detected emotions? • relation between kinds and amount of arguments, and the engagement of participants?
- 13. argumentation support decision-making and persuasion e-democracy and online debates abstract bipolar argumentation [Dung, 1995; Cayrol and Lagasquie-Schiex, 2013] support and attack relations
- 14. e.g. A. Elena: This information is important, we must publish it B. Serena: It is a private information about a person who does not want to publish it C. Elena: This person is the Prime Minister so the information is not private D. Fabien: Yes, being a governmental officer makes the information about him public attack attack support
- 17. emotion computing beyond purely rational behavior detect emotional state to adapt reactions e.g.(serious) games
- 18. emotion detection webcams for facial expressions analysis [FACEREADER 6.0] physiological sensors (EEG) for cognitive states [Chaouachi et al., 2010]
- 19. real–time engagement engagement index [Pope et al.,1995] 𝑒𝑛𝑔 = EEG frequency bands (4-8Hz) α (8-13Hz) β (13-22Hz)
- 20. real-time facial analysis classifying 500 key points in facial muscles neural network trained on 10 000 examples happy, sad, angry, surprised, scared, disgusted. valence, arousal neutral probability.
- 22. Seempad Joint Research Lab Emotions play an important role in decision making [Quartz, 2009] Assess connection between argumentation and emotions Final goal = Detect on the Web… • a debate turning into a flame war, • a content reaching an agreement, • a good or bad emotion spreading in a community… +
- 23. 1st Experiment & Public dataset focus on associating participants’ arguments and the relations among them mental engagement detected by EEG facial emotions detected via Face Emotion Recognition
- 24. protocol of the experiment topics from popular discussions in iDebate & DebateGraph 12 debates - 4 participants and 2 moderators each participants equipped with emotions detection tools messages in plain English through IRC participants are anonymous debate for 20 minutes debrief questionnaire
- 25. participants 6 sessions of 4 participants (-1 session) 20 participants (7 women, 13 men) age range from 22 to 35 years not all of them were native English speakers students in a North American university signed an ethical agreement good computer skills
- 26. data collection during the experiment minimum, average and maximum engagement of every participant in a debate most dominant emotion (having maximum value) pleased/unpleased valence active/inactive arousal
- 27. data annotation after the experiment synchronize arguments, relations and emotional indexes bipolar argumentation labelled with : sources, arguments, emotional states two independent annotators with agreement of 91% Cohen’s kappa 0.82 >> 0.6 = Pr 𝑎 − Pr(𝑒) 1 − Pr(e)
- 29. dataset content xml structure of debate flow 598 arguments in 12 different debates 263 argument pairs 127 supports 136 attacks gender, age and personality type dominant emotion, valence and arousal mental engagement levels
- 30. dataset extract (flow) <argument id="2" debate_id="4" participant="4" time- from="20:30" time-to="20:30">The religion is an independent factor, it should not be a dissociative factor separating people. </argument> <argument id="3" debate_id="4" participant="1" time- from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problem of the life so it's not all bad. </argument> <argument id="4" debate_id="4" participant="4" time- from="20:32" time-to="20:32">Here in Canada it is appreciable to find the liberty of religion a practice in a peaceful way. </argument>
- 31. dataset extract (relations) <debate id="4" title="Religion" task="relation"> <pair id="1" relation="support"> <argument id="2" debate_id="4" participant="4" time- from="20:30" time-to="20:30">The religion is an independent factor, it should not be a dissociative factor separating people. </argument> <argument id="3" debate_id="4" participant="1" time- from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problems of the life so it's not all bad. </argument> </pair> <pair id="2" relation="attack"> <argument id="3" debate_id="4" participant="1" time- from="20:32" time-to="20:32">The religion gives to his followers hope and help them to overcome some problems of the life so it's not all bad. </argument> <argument id="5" debate_id="4" participant="3" time- from="20:32" time-to="20:32">During all the existence of human being, religion makes a lot of issue. It make more hurts than curs. </argument> </pair>
- 32. dataset extract (emotions) <argument id="30" debate_id="4" participant="4" time-from="20:43" time-to="20:43" emotion_p1="neutral" emotion_p2="neutral" emotion_p3="neutral" emotion_p4="neutral"> Indeed but there exist some advocates of the devil like Bernard Levi who is decomposing arabic countries. </argument> <argument id="31" debate_id="4" participant="1" time-from="20:43" time-to="20:43" emotion_p1="angry" emotion_p2="neutral" emotion_p3="angry" emotion_p4="disgusted"> I don’t totally agree with you Participant2: science and religion don’t explain each other, they tend to explain the world but in two different ways. </argument> <argument id="32" debate_id="4" participant="3" time-from="20:44" time-to="20:44" emotion_p1="angry" emotion_p2="happy" emotion_p3="surprised" emotion_p4="angry"> Participant4: for recent wars ok but what about wars happened 3 or 4 centuries ago? </argument>
- 33. initial analysis H1 : some emotional and behavioral trends can be extracted from a set of debates. H2 : the number and the strength of arguments, attacks and supports exchanged between the debaters are correlated with particular emotions. H3 : the number of expressed arguments is connected to the degree of mental engagement and social interactions.
- 34. initial analysis mean percentage of appearance of each basic emotion (with 95% confidence interval) that the most frequent : anger 8:15% to 15:6% second most frequent : disgust 7:52% to 14:8% negativity effect [Rozin and Royzman, 2001] high level engagement (70:2% to 87:7% of time) correlated with anger (r=0.306)
- 35. emotional evolution participant 1 debate 1 before and after argument rejection (hypothesis 2)
- 36. correlation table for Session 3 “Distribute condoms at schools” “Encourage fewer people to go to the university” number of attacks increased linearly with disgust (H2)
- 37. aggregated correlation table supports increase linearly with engagement (r=0.31) more pronounced when no conflict (r=0.80) high engagement for most participative participants (H3)
- 38. to wrap-up connection between the arguments proposed by the participants of a debate and their emotional status? H1 : emotional trends can be extracted from debates H2 : arguments, attacks, supports correlated with emotions H3 : the number of arguments is connected to the engagement
- 39. current work granularity: from sessions to debates existence or not of a priori positive/negative opinion actual changes of opinion impact of personality (big 5 test) dynamics of the debate and emotion changes
- 40. perspectives deeper analysis, different time scales and granularity new experiment with improved protocol NLP analysis and correlations http://fabien.infohttp://project.inria.fr/seempad/ 40/40/…40 :-)
Editor's Notes
- Good morning My name is Fabien Gandon, I am a research director at Inria Sophia Antipolis France and presenting here on behalf of Serena and Elena who recently became mothers.
- The general context of this work is one of the many evolutions of the Web namely the fact it is becoming a world-wide agora
- Initially the Web was perceived as a documentary space for human consumption
- With the advent of Wikis in 94 we reopen the Web in writing mode
- This also starts the social Web evolution with forums, blogs, social networks and so on
- Among the contributions made we now have a number of techniques to detect and analyze an argument made in a contribution
- And link these arguments to reconstruct debate graphs
- We also have more and more means to analyze the emotions of the users from direct sensors to indirect sentiment analysis
- In parallel the semantic Web allows us to formally represent knowledge and data and also annotate all the resources identified on the Web.
- And we can leverage these formalisms to represent and merge these debate traces with other linked data on the Web, to exchange, query, reason on the debate traces.
- So the research question we are considering in this paper is : is there a connection between the arguments proposed by the participants of a debate and their emotional status? And in particular is there correlation of polarity of arguments and polarity of detected emotions? And is there a relation between the kinds and amount of arguments, and the engagement of participants?
- For this we first leverage results from Argumentation theory
- Argumentation theory is used for instance to support decision-making and persuasion for example in scenarios of e-democracy and online debates We rely on abstract bipolar argumentation to capture support and attack relations
- For instance here is a small debate … Behind this dialog is an argumentation graph with support and attack relations that we can capture and represent with KR formalism
- Then it can be analyzed to extract for instance the claims that still stand in a debate and give an overview of the current state of the debate.
- The second domain we rely on is emotion analysis.
- Here the idea is to go beyond purely rational behavior by detecting emotional states of users to adapt the reactions of the system This is used for instance in designing and adapting games
- In this work we rely on two detection means: facial expressions analysis with FACEREADER and physiological sensors EEG
- physiological sensors EEG are used here to compute real–time engagement using Pope’s metric based on theta, alpha and Beta EEG frequency bands This gives us a clue of the engagement of a user in a task.
- The real-time facial analysis classifies 500 key points in facial muscles and uses a neural network trained on 10 000 examples to indicate if the user is happy, sad, angry, surprised, scared, disgusted. The valence and arousal of his emotion And a probability of neutral emotion
- So in the context of a joint lab between Inria and University Montreal called Seempad we combine Argumentation theory and Emotion detections
- We believe emotions play an important role in decision making and we want to assess connection between argumentation and emotions Our final goal would be to be able to detect on the Web… a debate turning into a flame war, a content reaching an agreement, a good or bad emotion spreading in a community
- The article presents the first result of this project which is an experiment to produce a publicly available dataset of recorded debates associating participants’ arguments and the relations among them mental engagement detected by EEG facial emotions detected via Face Emotion Recognition
- Topics obtained from
- during data collection we recorded
- After the experiment we synchronize arguments, relations and emotional indexes The bipolar argumentation is labelled with sources, arguments, emotional states by two independent annotators with a verified good level of agreement Cohen's kappa coefficient is a statistic which measures inter-rater agreement for qualitative (categorical) items. It is generally thought to be a more robust measure than simple percent agreement calculation, since κ takes into account the agreement occurring by chance Pr(a) is the relative observed agreement among raters, and Pr(e) is the hypothetical probability
- Here is the table of content of the publicly available dataset with topics and stats.
- In a nutshell this public dataset contains: An XML structure of debate flow 598 arguments in 12 different debates 263 argument pairs 127 supports 136 attacks gender, age and personality type dominant emotion, valence and arousal mental engagement levels
- Here is what it looks like if you open this dataset. You have here the markup for the argument flow
- the markup for the relations between argument (support/attack)
- the markup for the synchronized emotions detected
- This first dataset allowed us perform so preliminary analysis The general hypothesis was some emotional and behavioral trends can be extracted from a set of debates. With two sub hypothesis: the number and the strength of arguments, attacks and supports exchanged between the debaters are correlated with particular emotions. the number of expressed arguments is connected to the degree of mental engagement and social interactions
- The details are in the paper and I will only give some extracts here Looking at the mean percentage of emotions, most frequent ones are anger and disgusts and you can verify the negativity effect namely that negative emotions last longer. There is also a correlation between anger and engagement
- Here is a extract of the emotion evolution for surprise and disgust for participant 1 in debate 1. And what you see here is what happen when an argument is rejected.
- Here is the correlation table for session 3 with two debates on “Distributing condoms at schools” and “Encouraging fewer people to go to the university” An example of interesting things happening here is to see that the number of attacks increases linearly with disgust
- Now here is the aggregated correlation table for all sessions And you can see for instance that supports increase linearly with engagement and this is even more pronounced when there is no conflict And there is high engagement for most participative participants
- To wrap-up our question was is there connection between the arguments proposed by the participants of a debate and their emotional status? and we have evidences for three initial hypothesis that are
- We are now performing deeper analysis First of all at smaller granularity at the level of debates We are considering the existence or not of a priori positive/negative opinion We are checking the actual changes of opinion We are considering the impact of the personality using the big 5 test And looking at the dynamics of the debate and emotion changes
- On a longer term we target deeper analysis, different time scales and granularity We are preparing a new experiment with an improved protocol We intend to perform NLP analysis on the debate corpus and consider the results in the correlations