SlideShare a Scribd company logo
1 of 30
Download to read offline
R by example: mining Twitter for consumer
attitudes towards airlines

presented at the

Boston Predictive Analytics
MeetUp
by


Jeffrey Breen
President
Cambridge Aviation Research

jbreen@cambridge.aero

June 2011




  Cambridge Aviation Research   • 245 First Street • Suite 1800 • Cambridge, MA 02142 • cambridge.aero




                                                                                                         © Copyright 2010 by Cambridge Aviation Research. All rights reserved.
Airlines top customer satisfaction... alphabetically




http://www.theacsi.org/                                     3
Actually, they rank below the Post
    Office and health insurers




                                     4
which gives us plenty to listen to
                               Completely unimpressed with @continental or @united.
RT @dave_mcgregor:       Poor communication, goofy reservations systems and
Publicly pledging to                       all to turn my trip into a mess.
never fly @delta again.
The worst airline ever.
U have lost my patronage     @united #fail on wifi in red carpet clubs (too
forever due to ur            slow), delayed flight, customer service in red
incompetence                 carpet club (too slow), hmmm do u see a trend?



@United Weather delays may not be your fault,
but you are in the customer service business.
It's atrocious how people are getting treated!
We were just told we are delayed 1.5        @SouthwestAir I know you don't make the
hrs & next announcement on @JetBlue -      weather. But at least pretend I am not a
“We're selling headsets.” Way to           bother when I ask if the delay will make
capitalize on our misfortune.                                    miss my connection
         @SouthwestAir
     I hate you with every            Hey @delta - you suck! Your prices
    single bone in my body          are over the moon & to move a flight
   for delaying my flight by         a cpl of days is $150.00. Insane. I
   3 hours, 30mins before I              hate you! U ruined my vacation!
    was supposed to board.
             #hate
Game Plan
Search Twitter for
airline mentions &
collect tweet text            Score sentiment for    Summarize for each
                                  each tweet              airline
 Load sentiment
   word lists


                                                           Compare Twitter
                                                         sentiment with ACSI
                                                           satisfaction score
                       Scrape ACSI web site for
                     airline customer satisfaction
                                 scores




                                                                                14
Game Plan
Search Twitter for
airline mentions &
collect tweet text            Score sentiment for    Summarize for each
                                  each tweet              airline
 Load sentiment
   word lists


                                                           Compare Twitter
                                                         sentiment with ACSI
                                                           satisfaction score
                       Scrape ACSI web site for
                     airline customer satisfaction
                                 scores




                                                                                15
Searching Twitter in one line
R’s XML and RCurl packages make it easy to grab web data, but Jeff
Gentry’s twitteR package makes searching Twitter almost too easy:

> # load the package
> library(twitteR)
> # get the 1,500 most recent tweets mentioning ‘@delta’:
> delta.tweets = searchTwitter('@delta', n=1500)




See what we got in return:              A “list” in R is a collection of
                                        objects and its elements may be
> length(delta.tweets)                  named or just numbered.
[1] 1500
> class(delta.tweets)
[1] "list"
                                        “[[ ]]” is used to access elements.
Examine the output
Let’s take a look at the first tweet in the output list:

    > tweet = delta.tweets[[1]]
                                       tweet is an object of type “status”
                                       from the “twitteR” package.
    > class(tweet)
    [1] "status"
    attr(,"package")                   It holds all the information about
    [1] "twitteR"                      the tweet returned from Twitter.



The help page (“?status”) describes some accessor methods like
getScreenName() and getText() which do what you would expect:

    > tweet$getScreenName()
    [1] "Alaqawari"
    > tweet$getText()
    [1] "I am ready to head home. Inshallah will try to get on the earlier
    flight to Fresno. @Delta @DeltaAssist"
Extract the tweet text
R has several (read: too many) ways to apply functions iteratively.
•The plyr package unifies them all with a consistent naming convention.
•The function name is determined by the input and output data types. We
have a list and would like a simple array output, so we use “laply”:

> delta.text = laply(delta.tweets, function(t) t$getText() )


> length(delta.text)[1] 1500
> head(delta.text, 5)
[1] "I am ready to head home. Inshallah will try to get on the earlier
flight to Fresno. @Delta @DeltaAssist"
[2] "@Delta Releases 2010 Corporate Responsibility Report - @PRNewswire
(press release) : http://tinyurl.com/64mz3oh"
[3] "Another week, another upgrade! Thanks @Delta!"
[4] "I'm not able to check in or select a seat for flight DL223/KL6023 to
Seattle tomorrow. Help? @KLM @delta"
[5] "In my boredom of waiting realized @deltaairlines is now @delta
seriously..... Stil waiting and your not even unloading status yet"
Game Plan
Search Twitter for
airline mentions &
collect tweet text            Score sentiment for    Summarize for each
                                  each tweet              airline
 Load sentiment
   word lists


                                                           Compare Twitter
                                                         sentiment with ACSI
                                                           satisfaction score
                       Scrape ACSI web site for
                     airline customer satisfaction
                                 scores




                                                                                19
Estimating Sentiment

There are many good papers and resources describing methods to
estimate sentiment. These are very complex algorithms.



For this tutorial, we use a very simple algorithm which assigns a score by
simply counting the number of occurrences of “positive” and “negative”
words in a tweet. The code for our score.sentiment() function can be
found at the end of this deck.


Hu & Liu have published an “opinion lexicon” which categorizes
approximately 6,800 words as positive or negative and which can be
downloaded.


            Positive: love, best, cool, great, good, amazing
            Negative: hate, worst, sucks, awful, nightmare
                                                                        20
Load sentiment word lists
1. Download Hu & Liu’s opinion lexicon:


   http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html


2. Loading data is one of R’s strengths. These are simple text files,
though they use “;” as a comment character at the beginning:

   > hu.liu.pos = scan('../data/opinion-lexicon-English/positive-
   words.txt', what='character', comment.char=';')

   > hu.liu.neg = scan('../data/opinion-lexicon-English/negative-
   words.txt', what='character', comment.char=';')



3. Add a few industry-specific and/or especially emphatic terms:

   > pos.words = c(hu.liu.pos, 'upgrade')         The c() function
   > neg.words = c(hu.liu.neg, 'wtf', 'wait',     combines objects
     'waiting', 'epicfail', 'mechanical')         into vectors or lists
Game Plan
Search Twitter for
airline mentions &
collect tweet text            Score sentiment for    Summarize for each
                                  each tweet              airline
 Load sentiment
   word lists


                                                           Compare Twitter
                                                         sentiment with ACSI
                                                           satisfaction score
                       Scrape ACSI web site for
                     airline customer satisfaction
                                 scores




                                                                                22
Algorithm sanity check
    > sample = c("You're awesome and I love you",
          "I hate and hate and hate. So angry. Die!",
          "Impressed and amazed: you are peerless in your achievement of
          unparalleled mediocrity.")
    > result = score.sentiment(sample, pos.words, neg.words)
    > class(result)
                                   data.frames hold tabular data so they
    [1] "data.frame"
                                   consist of columns & rows which can
    > result$score
                                   be accessed by name or number.
    [1]   2 -5   4
                                   Here, “score” is the name of a column.


So, not so good with sarcasm. Here are a couple of real tweets:

    > score.sentiment(c("@Delta I'm going to need you to get it together.
    Delay on tarmac, delayed connection, crazy gate changes... #annoyed",
    "Surprised and happy that @Delta helped me avoid the 3.5 hr layover I
    was scheduled for. Patient and helpful agents. #remarkable"),
    pos.words, neg.words)$score
    [1] -4   5
Accessing data.frames
Here’s the data.frame just returned from score.sentiment():
   > result
       score                                                                                     text

   1           2                                                         You're awesome and I love you

   2      -5                                                   I hate and hate and hate. So angry. Die!

   3           4 Impressed and amazed: you are peerless in your achievement of unparalleled mediocrity.



Elements can be accessed by name or position, and positions can be
ranges:
   > result[1,1]
   [1] 2
   > result[1,'score']
   [1] 2
   > result[1:2, 'score']
   [1]         2 -5
   > result[c(1,3), 'score']
   [1] 2 4
   > result[,'score']
   [1]         2 -5     4
Score the tweets
To score all of the Delta tweets, just feed their text into
score.sentiment():

    > delta.scores = score.sentiment(delta.text, pos.words,     Progress bar
    neg.words, .progress='text')                                provided by
    |==================================================| 100%   plyr

Let’s add two new columns to identify the airline for when we
combine all the scores later:
    > delta.scores$airline = 'Delta'
    > delta.scores$code = 'DL’
Plot Delta’s score distribution
R’s built-in hist() function will create and plot histograms of your data:
    > hist(delta.scores$score)
The ggplot2 alternative
ggplot2 is an alternative graphics package which generates more refined
graphics:
   > qplot(delta.scores$score)
Lather. Rinse. Repeat
To see how the other airlines fare, collect & score tweets for other
airlines.


Then combine all the results into a single “all.scores” data.frame:

    > all.scores = rbind( american.scores, continental.scores, delta.scores,
    jetblue.scores, southwest.scores, united.scores, us.scores )



                                                rbind() combines
                                                rows from
                                                data.frames, arrays,
                                                and matrices
Compare score distributions
   ggplot2 implements “grammar of graphics”, building plots in layers:
       > ggplot(data=all.scores) + # ggplot works on data.frames, always
            geom_bar(mapping=aes(x=score, fill=airline), binwidth=1) +
            facet_grid(airline~.) + # make a separate plot for each airline
            theme_bw() + scale_fill_brewer() # plain display, nicer colors




ggplot2’s faceting
capability makes it
easy to generate the
same graph for
different values of a
variable, in this case
“airline”.
Game Plan
Search Twitter for
airline mentions &
collect tweet text            Score sentiment for    Summarize for each
                                  each tweet              airline
 Load sentiment
   word lists


                                                           Compare Twitter
                                                         sentiment with ACSI
                                                           satisfaction score
                       Scrape ACSI web site for
                     airline customer satisfaction
                                 scores




                                                                                30
Ignore the middle
Let’s focus on very negative (<-2) and positive (>2) tweets:
    > all.scores$very.pos = as.numeric( all.scores$score >= 2 )
    > all.scores$very.neg = as.numeric( all.scores$score <= -2 )


For each airline ( airline + code ), let’s use the ratio of very positive to
very negative tweets as the overall sentiment score for each airline:
    > twitter.df = ddply(all.scores, c('airline', 'code'), summarise,
    pos.count = sum( very.pos ), neg.count = sum( very.neg ) )
    > twitter.df$all.count = twitter.df$pos.count + twitter.df$neg.count
    > twitter.df$score = round( 100 * twitter.df$pos.count /
                 twitter.df$all.count )

Sort with orderBy() from the doBy package:
        > orderBy(~-score, twitter.df)
Any relation to ACSI’s airline scores?




http://www.theacsi.org/index.php?option=com_content&view=article&id=147&catid=&Itemid=212&i=Airlines

                                                                                                       18
Game Plan
Search Twitter for
airline mentions &
collect tweet text            Score sentiment for    Summarize for each
                                  each tweet              airline
 Load sentiment
   word lists


                                                           Compare Twitter
                                                         sentiment with ACSI
                                                           satisfaction score
                       Scrape ACSI web site for
                     airline customer satisfaction
                                 scores




                                                                                33
Scrape, don’t type
XML package provides amazing readHTMLtable() function:
    > library(XML)
    > acsi.url = 'http://www.theacsi.org/index.php?
    option=com_content&view=article&id=147&catid=&Itemid=212&i=Airlines'
    > acsi.df = readHTMLTable(acsi.url, header=T, which=1,
    stringsAsFactors=F)
    > # only keep column #1 (name) and #18 (2010 score)
    > acsi.df = acsi.df[,c(1,18)]
    > head(acsi.df,1)
                         10
    1 Southwest Airlines 79



Well, typing metadata is OK, I guess... clean up column names, etc:

    > colnames(acsi.df) = c('airline', 'score')              NA (as in “n/a”) is
    > acsi.df$code = c('WN', NA, 'CO', NA, 'AA', 'DL',       supported as a
                       'US', 'NW', 'UA')                     valid value
    > acsi.df$score = as.numeric(acsi.df$score)              everywhere in R.
Game Plan
Search Twitter for
airline mentions &
collect tweet text            Score sentiment for    Summarize for each
                                  each tweet              airline
 Load sentiment
   word lists


                                                           Compare Twitter
                                                         sentiment with ACSI
                                                           satisfaction score
                       Scrape ACSI web site for
                     airline customer satisfaction
                                 scores




                                                                                35
Join and compare
merge() joins two data.frames by the specified “by=” fields. You can
specify ‘suffixes’ to rename conflicting column names:

    > compare.df = merge(twitter.df, acsi.df, by='code',
        suffixes=c('.twitter', '.acsi'))




Unless you specify “all=T”, non-matching rows are dropped (like a SQL
INNER JOIN), and that’s what happened to top scoring JetBlue.


With a very low score, and low traffic to boot, soon-to-disappear
Continental looks like an outlier. Let’s exclude:
    > compare.df = subset(compare.df, all.count > 100)
an actual result!
ggplot will even run lm() linear
(and other) regressions for you
 with its geom_smooth() layer:

> ggplot( compare.df ) +
geom_point(aes(x=score.twitter,
y=score.acsi,
color=airline.twitter), size=5) +
geom_smooth(aes(x=score.twitter,
y=score.acsi, group=1), se=F,
method="lm") +
theme_bw() +
opts(legend.position=c(0.2,
0.85))




                                         37
                                         21
http://www.despair.com/cudi.html
R code for example scoring function
    score.sentiment = function(sentences, pos.words, neg.words, .progress='none')
{
	   require(plyr)
	   require(stringr)
	
	   # we got a vector of sentences. plyr will handle a list or a vector as an "l" for us
	   # we want a simple array of scores back, so we use "l" + "a" + "ply" = laply:
	   scores = laply(sentences, function(sentence, pos.words, neg.words) {
	   	
	   	      # clean up sentences with R's regex-driven global substitute, gsub():
	   	      sentence = gsub('[[:punct:]]', '', sentence)
	   	      sentence = gsub('[[:cntrl:]]', '', sentence)
	   	      sentence = gsub('d+', '', sentence)
	   	      # and convert to lower case:
	   	      sentence = tolower(sentence)

	   	     # split into words. str_split is in the stringr package
	   	     word.list = str_split(sentence, 's+')
	   	     # sometimes a list() is one level of hierarchy too much
	   	     words = unlist(word.list)

	   	     # compare our words to the dictionaries of positive & negative terms
	   	     pos.matches = match(words, pos.words)
	   	     neg.matches = match(words, neg.words)
	
	   	     # match() returns the position of the matched term or NA
	   	     # we just want a TRUE/FALSE:
	   	     pos.matches = !is.na(pos.matches)
	   	     neg.matches = !is.na(neg.matches)

	   	     # and conveniently enough, TRUE/FALSE will be treated as 1/0 by sum():
	   	     score = sum(pos.matches) - sum(neg.matches)

	   	      return(score)
	   }, pos.words, neg.words, .progress=.progress )

	   scores.df = data.frame(score=scores, text=sentences)
	   return(scores.df)
}                                                                                          39

More Related Content

What's hot

Web application attacks
Web application attacksWeb application attacks
Web application attackshruth
 
Software project planning
Software project planningSoftware project planning
Software project planningrajvir_kaur
 
source code metrics and other maintenance tools and techniques
source code metrics and other maintenance tools and techniquessource code metrics and other maintenance tools and techniques
source code metrics and other maintenance tools and techniquesSiva Priya
 
Programming team structure
Programming team structureProgramming team structure
Programming team structureNancyBeaulah_R
 
System programmin practical file
System programmin practical fileSystem programmin practical file
System programmin practical fileAnkit Dixit
 
Real time and distributed design
Real time and distributed designReal time and distributed design
Real time and distributed designpriyapavi96
 
Chapter 1 1
Chapter 1 1Chapter 1 1
Chapter 1 1bolovv
 
software cost factor
software cost factorsoftware cost factor
software cost factorAbinaya B
 
Http request and http response
Http request and http responseHttp request and http response
Http request and http responseNuha Noor
 
Software Cost Estimation Techniques
Software Cost Estimation TechniquesSoftware Cost Estimation Techniques
Software Cost Estimation TechniquesMohamedIbrahim1408
 
MiVoice Business software versions through time
MiVoice Business software versions through timeMiVoice Business software versions through time
MiVoice Business software versions through timedenwacomms
 
Web Technology Lab files with practical
Web Technology Lab  files with practicalWeb Technology Lab  files with practical
Web Technology Lab files with practicalNitesh Dubey
 
Study on Zeus Banking Malware
Study on Zeus Banking MalwareStudy on Zeus Banking Malware
Study on Zeus Banking MalwareShaik Anisa
 
Software Configuration Management (SCM)
Software Configuration Management (SCM)Software Configuration Management (SCM)
Software Configuration Management (SCM)Er. Shiva K. Shrestha
 
Sockets in unix
Sockets in unixSockets in unix
Sockets in unixswtjerin4u
 
CompTIA Security+ Guide
CompTIA Security+ GuideCompTIA Security+ Guide
CompTIA Security+ GuideSmithjulia33
 

What's hot (20)

Web application attacks
Web application attacksWeb application attacks
Web application attacks
 
Software project planning
Software project planningSoftware project planning
Software project planning
 
Software Engineering
Software EngineeringSoftware Engineering
Software Engineering
 
source code metrics and other maintenance tools and techniques
source code metrics and other maintenance tools and techniquessource code metrics and other maintenance tools and techniques
source code metrics and other maintenance tools and techniques
 
Programming team structure
Programming team structureProgramming team structure
Programming team structure
 
System programmin practical file
System programmin practical fileSystem programmin practical file
System programmin practical file
 
Real time and distributed design
Real time and distributed designReal time and distributed design
Real time and distributed design
 
Chapter 1 1
Chapter 1 1Chapter 1 1
Chapter 1 1
 
software cost factor
software cost factorsoftware cost factor
software cost factor
 
COCOMO (Software Engineering)
COCOMO (Software Engineering)COCOMO (Software Engineering)
COCOMO (Software Engineering)
 
Network Sockets
Network SocketsNetwork Sockets
Network Sockets
 
Http request and http response
Http request and http responseHttp request and http response
Http request and http response
 
Software Cost Estimation Techniques
Software Cost Estimation TechniquesSoftware Cost Estimation Techniques
Software Cost Estimation Techniques
 
MiVoice Business software versions through time
MiVoice Business software versions through timeMiVoice Business software versions through time
MiVoice Business software versions through time
 
Web Technology Lab files with practical
Web Technology Lab  files with practicalWeb Technology Lab  files with practical
Web Technology Lab files with practical
 
Study on Zeus Banking Malware
Study on Zeus Banking MalwareStudy on Zeus Banking Malware
Study on Zeus Banking Malware
 
Software Configuration Management (SCM)
Software Configuration Management (SCM)Software Configuration Management (SCM)
Software Configuration Management (SCM)
 
Sockets in unix
Sockets in unixSockets in unix
Sockets in unix
 
software design
software designsoftware design
software design
 
CompTIA Security+ Guide
CompTIA Security+ GuideCompTIA Security+ Guide
CompTIA Security+ Guide
 

Viewers also liked

Sentiment Analysis via R Programming
Sentiment Analysis via R ProgrammingSentiment Analysis via R Programming
Sentiment Analysis via R ProgrammingSkillspeed
 
Sentiment Analysis in R
Sentiment Analysis in RSentiment Analysis in R
Sentiment Analysis in REdureka!
 
How to Win Machine Learning Competitions ?
How to Win Machine Learning Competitions ? How to Win Machine Learning Competitions ?
How to Win Machine Learning Competitions ? HackerEarth
 
Tweets Classification using Naive Bayes and SVM
Tweets Classification using Naive Bayes and SVMTweets Classification using Naive Bayes and SVM
Tweets Classification using Naive Bayes and SVMTrilok Sharma
 
Twitter sentiment-analysis Jiit2013-14
Twitter sentiment-analysis Jiit2013-14Twitter sentiment-analysis Jiit2013-14
Twitter sentiment-analysis Jiit2013-14Rachit Goel
 
Sentiment analysis of tweets
Sentiment analysis of tweetsSentiment analysis of tweets
Sentiment analysis of tweetsVasu Jain
 
Introduction to Sentiment Analysis
Introduction to Sentiment AnalysisIntroduction to Sentiment Analysis
Introduction to Sentiment AnalysisJaganadh Gopinadhan
 
How Sentiment Analysis works
How Sentiment Analysis worksHow Sentiment Analysis works
How Sentiment Analysis worksCJ Jenkins
 
Sentiment Analysis in Twitter
Sentiment Analysis in TwitterSentiment Analysis in Twitter
Sentiment Analysis in TwitterAyushi Dalmia
 
Tutorial of Sentiment Analysis
Tutorial of Sentiment AnalysisTutorial of Sentiment Analysis
Tutorial of Sentiment AnalysisFabio Benedetti
 
Sentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSumit Raj
 
The Who What Where When And Why Of Social Media Lead Generation
The Who What Where When And Why Of Social Media Lead GenerationThe Who What Where When And Why Of Social Media Lead Generation
The Who What Where When And Why Of Social Media Lead GenerationAbhishek Shah
 
Social Media Secrets
Social Media SecretsSocial Media Secrets
Social Media SecretsGuy Kawasaki
 
Zenpayroll Pitch Deck Template
Zenpayroll Pitch Deck TemplateZenpayroll Pitch Deck Template
Zenpayroll Pitch Deck TemplateJoseph Hsieh
 
AdPushup Fundraising Deck - First Pitch
AdPushup Fundraising Deck - First PitchAdPushup Fundraising Deck - First Pitch
AdPushup Fundraising Deck - First Pitchadpushup
 
AppVirality.com - Investor Pitch Deck
AppVirality.com - Investor Pitch DeckAppVirality.com - Investor Pitch Deck
AppVirality.com - Investor Pitch DeckLaxman Papineni
 
How Wealthsimple raised $2M in 2 weeks
How Wealthsimple raised $2M in 2 weeksHow Wealthsimple raised $2M in 2 weeks
How Wealthsimple raised $2M in 2 weeksWealthsimple
 
500’s Demo Day Batch 16 >> Podozi
500’s Demo Day Batch 16 >>  Podozi500’s Demo Day Batch 16 >>  Podozi
500’s Demo Day Batch 16 >> Podozi500 Startups
 
Swipes pitch deck for Beta Pitch 2013 Finals in Berlin
Swipes pitch deck for Beta Pitch 2013 Finals in BerlinSwipes pitch deck for Beta Pitch 2013 Finals in Berlin
Swipes pitch deck for Beta Pitch 2013 Finals in BerlinSwipes App
 

Viewers also liked (20)

Sentiment Analysis via R Programming
Sentiment Analysis via R ProgrammingSentiment Analysis via R Programming
Sentiment Analysis via R Programming
 
Sentiment Analysis in R
Sentiment Analysis in RSentiment Analysis in R
Sentiment Analysis in R
 
How to Win Machine Learning Competitions ?
How to Win Machine Learning Competitions ? How to Win Machine Learning Competitions ?
How to Win Machine Learning Competitions ?
 
Tweets Classification using Naive Bayes and SVM
Tweets Classification using Naive Bayes and SVMTweets Classification using Naive Bayes and SVM
Tweets Classification using Naive Bayes and SVM
 
Twitter sentiment-analysis Jiit2013-14
Twitter sentiment-analysis Jiit2013-14Twitter sentiment-analysis Jiit2013-14
Twitter sentiment-analysis Jiit2013-14
 
Sentiment analysis of tweets
Sentiment analysis of tweetsSentiment analysis of tweets
Sentiment analysis of tweets
 
Introduction to Sentiment Analysis
Introduction to Sentiment AnalysisIntroduction to Sentiment Analysis
Introduction to Sentiment Analysis
 
How Sentiment Analysis works
How Sentiment Analysis worksHow Sentiment Analysis works
How Sentiment Analysis works
 
Sentiment Analysis in Twitter
Sentiment Analysis in TwitterSentiment Analysis in Twitter
Sentiment Analysis in Twitter
 
Tutorial of Sentiment Analysis
Tutorial of Sentiment AnalysisTutorial of Sentiment Analysis
Tutorial of Sentiment Analysis
 
Sentiment Analysis of Twitter Data
Sentiment Analysis of Twitter DataSentiment Analysis of Twitter Data
Sentiment Analysis of Twitter Data
 
The Who What Where When And Why Of Social Media Lead Generation
The Who What Where When And Why Of Social Media Lead GenerationThe Who What Where When And Why Of Social Media Lead Generation
The Who What Where When And Why Of Social Media Lead Generation
 
Social Media Secrets
Social Media SecretsSocial Media Secrets
Social Media Secrets
 
SteadyBudget's Seed Funding Pitch Deck
SteadyBudget's Seed Funding Pitch DeckSteadyBudget's Seed Funding Pitch Deck
SteadyBudget's Seed Funding Pitch Deck
 
Zenpayroll Pitch Deck Template
Zenpayroll Pitch Deck TemplateZenpayroll Pitch Deck Template
Zenpayroll Pitch Deck Template
 
AdPushup Fundraising Deck - First Pitch
AdPushup Fundraising Deck - First PitchAdPushup Fundraising Deck - First Pitch
AdPushup Fundraising Deck - First Pitch
 
AppVirality.com - Investor Pitch Deck
AppVirality.com - Investor Pitch DeckAppVirality.com - Investor Pitch Deck
AppVirality.com - Investor Pitch Deck
 
How Wealthsimple raised $2M in 2 weeks
How Wealthsimple raised $2M in 2 weeksHow Wealthsimple raised $2M in 2 weeks
How Wealthsimple raised $2M in 2 weeks
 
500’s Demo Day Batch 16 >> Podozi
500’s Demo Day Batch 16 >>  Podozi500’s Demo Day Batch 16 >>  Podozi
500’s Demo Day Batch 16 >> Podozi
 
Swipes pitch deck for Beta Pitch 2013 Finals in Berlin
Swipes pitch deck for Beta Pitch 2013 Finals in BerlinSwipes pitch deck for Beta Pitch 2013 Finals in Berlin
Swipes pitch deck for Beta Pitch 2013 Finals in Berlin
 

Similar to Here are a few ways to access elements of the data.frame returned by score.sentiment():1. Select a column by name:result$score2. Select a row by index: result1, 3. Select a specific element:result1,14. Select multiple rows/columns:resultc(1,3), c("score","text")5. Use logical conditions to select rows:resultresult$score < 0,So in summary, data.frames allow flexible access to elements by name, index, or condition for both rows and columns. The $ operator selects columns, while

Sentiment analysis on airlines
Sentiment analysis on airlinesSentiment analysis on airlines
Sentiment analysis on airlinesPiyush Srivastava
 
Final Presentation
Final PresentationFinal Presentation
Final PresentationLove Tyagi
 
AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...
AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...
AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...Amazon Web Services
 
Building a Production-ready Predictive App for Customer Service - Alex Ingerm...
Building a Production-ready Predictive App for Customer Service - Alex Ingerm...Building a Production-ready Predictive App for Customer Service - Alex Ingerm...
Building a Production-ready Predictive App for Customer Service - Alex Ingerm...PAPIs.io
 
DashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docx
DashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docxDashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docx
DashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docxsimonithomas47935
 
Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...
Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...
Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...AWS Germany
 
Exploring Equilibrium Login to the computer and open a web browser.docx
Exploring Equilibrium Login to the computer and open a web browser.docxExploring Equilibrium Login to the computer and open a web browser.docx
Exploring Equilibrium Login to the computer and open a web browser.docxssuser454af01
 
(BDT302) Real-World Smart Applications With Amazon Machine Learning
(BDT302) Real-World Smart Applications With Amazon Machine Learning(BDT302) Real-World Smart Applications With Amazon Machine Learning
(BDT302) Real-World Smart Applications With Amazon Machine LearningAmazon Web Services
 
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter AnalysisIBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter AnalysisTorsten Steinbach
 
Building an Autonomous Data Layer
Building an Autonomous Data LayerBuilding an Autonomous Data Layer
Building an Autonomous Data LayerMartyPitt1
 
Flutter Festivals IIT Goa Session 2
Flutter Festivals IIT Goa Session 2Flutter Festivals IIT Goa Session 2
Flutter Festivals IIT Goa Session 2SEJALGUPTA44
 
IST365 - Project Deliverable #3Create the corresponding relation.docx
IST365 - Project Deliverable #3Create the corresponding relation.docxIST365 - Project Deliverable #3Create the corresponding relation.docx
IST365 - Project Deliverable #3Create the corresponding relation.docxpriestmanmable
 
"R & Text Analytics" (15 January 2013)
"R & Text Analytics" (15 January 2013)"R & Text Analytics" (15 January 2013)
"R & Text Analytics" (15 January 2013)Portland R User Group
 
AWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI services
AWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI servicesAWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI services
AWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI servicesAmazon Web Services
 
Command Query Responsibility Segregation at Enterprise Scale
Command Query Responsibility Segregation at Enterprise ScaleCommand Query Responsibility Segregation at Enterprise Scale
Command Query Responsibility Segregation at Enterprise ScaleTechWell
 
Databaseconcepts
DatabaseconceptsDatabaseconcepts
Databaseconceptsdilipkkr
 

Similar to Here are a few ways to access elements of the data.frame returned by score.sentiment():1. Select a column by name:result$score2. Select a row by index: result1, 3. Select a specific element:result1,14. Select multiple rows/columns:resultc(1,3), c("score","text")5. Use logical conditions to select rows:resultresult$score < 0,So in summary, data.frames allow flexible access to elements by name, index, or condition for both rows and columns. The $ operator selects columns, while (20)

Sentiment analysis on airlines
Sentiment analysis on airlinesSentiment analysis on airlines
Sentiment analysis on airlines
 
Final Presentation
Final PresentationFinal Presentation
Final Presentation
 
The Value of Twitter
The Value of TwitterThe Value of Twitter
The Value of Twitter
 
AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...
AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...
AWS January 2016 Webinar Series - Building Smart Applications with Amazon Mac...
 
Building a Production-ready Predictive App for Customer Service - Alex Ingerm...
Building a Production-ready Predictive App for Customer Service - Alex Ingerm...Building a Production-ready Predictive App for Customer Service - Alex Ingerm...
Building a Production-ready Predictive App for Customer Service - Alex Ingerm...
 
DashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docx
DashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docxDashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docx
DashboardUMUC DATA 620Assignment 3.1Your NameProfessorDateProbl.docx
 
Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...
Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...
Real-World Smart Applications with Amazon Machine Learning - AWS Machine Lear...
 
Exploring Equilibrium Login to the computer and open a web browser.docx
Exploring Equilibrium Login to the computer and open a web browser.docxExploring Equilibrium Login to the computer and open a web browser.docx
Exploring Equilibrium Login to the computer and open a web browser.docx
 
(BDT302) Real-World Smart Applications With Amazon Machine Learning
(BDT302) Real-World Smart Applications With Amazon Machine Learning(BDT302) Real-World Smart Applications With Amazon Machine Learning
(BDT302) Real-World Smart Applications With Amazon Machine Learning
 
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter AnalysisIBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
IBM Insight 2015 - 1824 - Using Bluemix and dashDB for Twitter Analysis
 
Building an Autonomous Data Layer
Building an Autonomous Data LayerBuilding an Autonomous Data Layer
Building an Autonomous Data Layer
 
Flutter Festivals IIT Goa Session 2
Flutter Festivals IIT Goa Session 2Flutter Festivals IIT Goa Session 2
Flutter Festivals IIT Goa Session 2
 
REST APIs, Girls Who Code
REST APIs, Girls Who CodeREST APIs, Girls Who Code
REST APIs, Girls Who Code
 
IST365 - Project Deliverable #3Create the corresponding relation.docx
IST365 - Project Deliverable #3Create the corresponding relation.docxIST365 - Project Deliverable #3Create the corresponding relation.docx
IST365 - Project Deliverable #3Create the corresponding relation.docx
 
"R & Text Analytics" (15 January 2013)
"R & Text Analytics" (15 January 2013)"R & Text Analytics" (15 January 2013)
"R & Text Analytics" (15 January 2013)
 
AWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI services
AWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI servicesAWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI services
AWS Summit Singapore 2019 | Accelerating ML Adoption with Our New AI services
 
Command Query Responsibility Segregation at Enterprise Scale
Command Query Responsibility Segregation at Enterprise ScaleCommand Query Responsibility Segregation at Enterprise Scale
Command Query Responsibility Segregation at Enterprise Scale
 
Databaseconcepts
DatabaseconceptsDatabaseconcepts
Databaseconcepts
 
Blind sql injection
Blind sql injectionBlind sql injection
Blind sql injection
 
Blind sql injection
Blind sql injectionBlind sql injection
Blind sql injection
 

More from Jeffrey Breen

Tapping the Data Deluge with R
Tapping the Data Deluge with RTapping the Data Deluge with R
Tapping the Data Deluge with RJeffrey Breen
 
Getting started with R & Hadoop
Getting started with R & HadoopGetting started with R & Hadoop
Getting started with R & HadoopJeffrey Breen
 
Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)
Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)
Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)Jeffrey Breen
 
Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....
Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....
Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....Jeffrey Breen
 
Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2
Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2
Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2Jeffrey Breen
 
Big Data Step-by-Step: Infrastructure 1/3: Local VM
Big Data Step-by-Step: Infrastructure 1/3: Local VMBig Data Step-by-Step: Infrastructure 1/3: Local VM
Big Data Step-by-Step: Infrastructure 1/3: Local VMJeffrey Breen
 
Move your data (Hans Rosling style) with googleVis + 1 line of R code
Move your data (Hans Rosling style) with googleVis + 1 line of R codeMove your data (Hans Rosling style) with googleVis + 1 line of R code
Move your data (Hans Rosling style) with googleVis + 1 line of R codeJeffrey Breen
 
Accessing Databases from R
Accessing Databases from RAccessing Databases from R
Accessing Databases from RJeffrey Breen
 
Grouping & Summarizing Data in R
Grouping & Summarizing Data in RGrouping & Summarizing Data in R
Grouping & Summarizing Data in RJeffrey Breen
 
R + 15 minutes = Hadoop cluster
R + 15 minutes = Hadoop clusterR + 15 minutes = Hadoop cluster
R + 15 minutes = Hadoop clusterJeffrey Breen
 
FAA Aviation Forecasts 2011-2031 overview
FAA Aviation Forecasts 2011-2031 overviewFAA Aviation Forecasts 2011-2031 overview
FAA Aviation Forecasts 2011-2031 overviewJeffrey Breen
 

More from Jeffrey Breen (12)

Tapping the Data Deluge with R
Tapping the Data Deluge with RTapping the Data Deluge with R
Tapping the Data Deluge with R
 
Getting started with R & Hadoop
Getting started with R & HadoopGetting started with R & Hadoop
Getting started with R & Hadoop
 
Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)
Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)
Big Data Step-by-Step: Using R & Hadoop (with RHadoop's rmr package)
 
Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....
Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....
Big Data Step-by-Step: Infrastructure 3/3: Taking it to the cloud... easily.....
 
Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2
Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2
Big Data Step-by-Step: Infrastructure 2/3: Running R and RStudio on EC2
 
Big Data Step-by-Step: Infrastructure 1/3: Local VM
Big Data Step-by-Step: Infrastructure 1/3: Local VMBig Data Step-by-Step: Infrastructure 1/3: Local VM
Big Data Step-by-Step: Infrastructure 1/3: Local VM
 
Move your data (Hans Rosling style) with googleVis + 1 line of R code
Move your data (Hans Rosling style) with googleVis + 1 line of R codeMove your data (Hans Rosling style) with googleVis + 1 line of R code
Move your data (Hans Rosling style) with googleVis + 1 line of R code
 
Accessing Databases from R
Accessing Databases from RAccessing Databases from R
Accessing Databases from R
 
Reshaping Data in R
Reshaping Data in RReshaping Data in R
Reshaping Data in R
 
Grouping & Summarizing Data in R
Grouping & Summarizing Data in RGrouping & Summarizing Data in R
Grouping & Summarizing Data in R
 
R + 15 minutes = Hadoop cluster
R + 15 minutes = Hadoop clusterR + 15 minutes = Hadoop cluster
R + 15 minutes = Hadoop cluster
 
FAA Aviation Forecasts 2011-2031 overview
FAA Aviation Forecasts 2011-2031 overviewFAA Aviation Forecasts 2011-2031 overview
FAA Aviation Forecasts 2011-2031 overview
 

Recently uploaded

A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI AgeCprime
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxLoriGlavin3
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFMichael Gough
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Strongerpanagenda
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch TuesdayIvanti
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Hiroshi SHIBATA
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterMydbops
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024TopCSSGallery
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Mark Simos
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxAna-Maria Mihalceanu
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxfnnc6jmgwh
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 

Recently uploaded (20)

A Framework for Development in the AI Age
A Framework for Development in the AI AgeA Framework for Development in the AI Age
A Framework for Development in the AI Age
 
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptxThe Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
The Role of FIDO in a Cyber Secure Netherlands: FIDO Paris Seminar.pptx
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
All These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDFAll These Sophisticated Attacks, Can We Really Detect Them - PDF
All These Sophisticated Attacks, Can We Really Detect Them - PDF
 
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better StrongerModern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
Modern Roaming for Notes and Nomad – Cheaper Faster Better Stronger
 
2024 April Patch Tuesday
2024 April Patch Tuesday2024 April Patch Tuesday
2024 April Patch Tuesday
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024Long journey of Ruby standard library at RubyConf AU 2024
Long journey of Ruby standard library at RubyConf AU 2024
 
Scale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL RouterScale your database traffic with Read & Write split using MySQL Router
Scale your database traffic with Read & Write split using MySQL Router
 
Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024Top 10 Hubspot Development Companies in 2024
Top 10 Hubspot Development Companies in 2024
 
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
Tampa BSides - The No BS SOC (slides from April 6, 2024 talk)
 
A Glance At The Java Performance Toolbox
A Glance At The Java Performance ToolboxA Glance At The Java Performance Toolbox
A Glance At The Java Performance Toolbox
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptxGenerative AI - Gitex v1Generative AI - Gitex v1.pptx
Generative AI - Gitex v1Generative AI - Gitex v1.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 

Here are a few ways to access elements of the data.frame returned by score.sentiment():1. Select a column by name:result$score2. Select a row by index: result1, 3. Select a specific element:result1,14. Select multiple rows/columns:resultc(1,3), c("score","text")5. Use logical conditions to select rows:resultresult$score < 0,So in summary, data.frames allow flexible access to elements by name, index, or condition for both rows and columns. The $ operator selects columns, while

  • 1. R by example: mining Twitter for consumer attitudes towards airlines presented at the Boston Predictive Analytics MeetUp by Jeffrey Breen President Cambridge Aviation Research jbreen@cambridge.aero June 2011 Cambridge Aviation Research • 245 First Street • Suite 1800 • Cambridge, MA 02142 • cambridge.aero © Copyright 2010 by Cambridge Aviation Research. All rights reserved.
  • 2. Airlines top customer satisfaction... alphabetically http://www.theacsi.org/ 3
  • 3. Actually, they rank below the Post Office and health insurers 4
  • 4. which gives us plenty to listen to Completely unimpressed with @continental or @united. RT @dave_mcgregor: Poor communication, goofy reservations systems and Publicly pledging to all to turn my trip into a mess. never fly @delta again. The worst airline ever. U have lost my patronage @united #fail on wifi in red carpet clubs (too forever due to ur slow), delayed flight, customer service in red incompetence carpet club (too slow), hmmm do u see a trend? @United Weather delays may not be your fault, but you are in the customer service business. It's atrocious how people are getting treated! We were just told we are delayed 1.5 @SouthwestAir I know you don't make the hrs & next announcement on @JetBlue - weather. But at least pretend I am not a “We're selling headsets.” Way to bother when I ask if the delay will make capitalize on our misfortune. miss my connection @SouthwestAir I hate you with every Hey @delta - you suck! Your prices single bone in my body are over the moon & to move a flight for delaying my flight by a cpl of days is $150.00. Insane. I 3 hours, 30mins before I hate you! U ruined my vacation! was supposed to board. #hate
  • 5. Game Plan Search Twitter for airline mentions & collect tweet text Score sentiment for Summarize for each each tweet airline Load sentiment word lists Compare Twitter sentiment with ACSI satisfaction score Scrape ACSI web site for airline customer satisfaction scores 14
  • 6. Game Plan Search Twitter for airline mentions & collect tweet text Score sentiment for Summarize for each each tweet airline Load sentiment word lists Compare Twitter sentiment with ACSI satisfaction score Scrape ACSI web site for airline customer satisfaction scores 15
  • 7. Searching Twitter in one line R’s XML and RCurl packages make it easy to grab web data, but Jeff Gentry’s twitteR package makes searching Twitter almost too easy: > # load the package > library(twitteR) > # get the 1,500 most recent tweets mentioning ‘@delta’: > delta.tweets = searchTwitter('@delta', n=1500) See what we got in return: A “list” in R is a collection of objects and its elements may be > length(delta.tweets) named or just numbered. [1] 1500 > class(delta.tweets) [1] "list" “[[ ]]” is used to access elements.
  • 8. Examine the output Let’s take a look at the first tweet in the output list: > tweet = delta.tweets[[1]] tweet is an object of type “status” from the “twitteR” package. > class(tweet) [1] "status" attr(,"package") It holds all the information about [1] "twitteR" the tweet returned from Twitter. The help page (“?status”) describes some accessor methods like getScreenName() and getText() which do what you would expect: > tweet$getScreenName() [1] "Alaqawari" > tweet$getText() [1] "I am ready to head home. Inshallah will try to get on the earlier flight to Fresno. @Delta @DeltaAssist"
  • 9. Extract the tweet text R has several (read: too many) ways to apply functions iteratively. •The plyr package unifies them all with a consistent naming convention. •The function name is determined by the input and output data types. We have a list and would like a simple array output, so we use “laply”: > delta.text = laply(delta.tweets, function(t) t$getText() ) > length(delta.text)[1] 1500 > head(delta.text, 5) [1] "I am ready to head home. Inshallah will try to get on the earlier flight to Fresno. @Delta @DeltaAssist" [2] "@Delta Releases 2010 Corporate Responsibility Report - @PRNewswire (press release) : http://tinyurl.com/64mz3oh" [3] "Another week, another upgrade! Thanks @Delta!" [4] "I'm not able to check in or select a seat for flight DL223/KL6023 to Seattle tomorrow. Help? @KLM @delta" [5] "In my boredom of waiting realized @deltaairlines is now @delta seriously..... Stil waiting and your not even unloading status yet"
  • 10. Game Plan Search Twitter for airline mentions & collect tweet text Score sentiment for Summarize for each each tweet airline Load sentiment word lists Compare Twitter sentiment with ACSI satisfaction score Scrape ACSI web site for airline customer satisfaction scores 19
  • 11. Estimating Sentiment There are many good papers and resources describing methods to estimate sentiment. These are very complex algorithms. For this tutorial, we use a very simple algorithm which assigns a score by simply counting the number of occurrences of “positive” and “negative” words in a tweet. The code for our score.sentiment() function can be found at the end of this deck. Hu & Liu have published an “opinion lexicon” which categorizes approximately 6,800 words as positive or negative and which can be downloaded. Positive: love, best, cool, great, good, amazing Negative: hate, worst, sucks, awful, nightmare 20
  • 12. Load sentiment word lists 1. Download Hu & Liu’s opinion lexicon: http://www.cs.uic.edu/~liub/FBS/sentiment-analysis.html 2. Loading data is one of R’s strengths. These are simple text files, though they use “;” as a comment character at the beginning: > hu.liu.pos = scan('../data/opinion-lexicon-English/positive- words.txt', what='character', comment.char=';') > hu.liu.neg = scan('../data/opinion-lexicon-English/negative- words.txt', what='character', comment.char=';') 3. Add a few industry-specific and/or especially emphatic terms: > pos.words = c(hu.liu.pos, 'upgrade') The c() function > neg.words = c(hu.liu.neg, 'wtf', 'wait', combines objects 'waiting', 'epicfail', 'mechanical') into vectors or lists
  • 13. Game Plan Search Twitter for airline mentions & collect tweet text Score sentiment for Summarize for each each tweet airline Load sentiment word lists Compare Twitter sentiment with ACSI satisfaction score Scrape ACSI web site for airline customer satisfaction scores 22
  • 14. Algorithm sanity check > sample = c("You're awesome and I love you", "I hate and hate and hate. So angry. Die!", "Impressed and amazed: you are peerless in your achievement of unparalleled mediocrity.") > result = score.sentiment(sample, pos.words, neg.words) > class(result) data.frames hold tabular data so they [1] "data.frame" consist of columns & rows which can > result$score be accessed by name or number. [1] 2 -5 4 Here, “score” is the name of a column. So, not so good with sarcasm. Here are a couple of real tweets: > score.sentiment(c("@Delta I'm going to need you to get it together. Delay on tarmac, delayed connection, crazy gate changes... #annoyed", "Surprised and happy that @Delta helped me avoid the 3.5 hr layover I was scheduled for. Patient and helpful agents. #remarkable"), pos.words, neg.words)$score [1] -4 5
  • 15. Accessing data.frames Here’s the data.frame just returned from score.sentiment(): > result score text 1 2 You're awesome and I love you 2 -5 I hate and hate and hate. So angry. Die! 3 4 Impressed and amazed: you are peerless in your achievement of unparalleled mediocrity. Elements can be accessed by name or position, and positions can be ranges: > result[1,1] [1] 2 > result[1,'score'] [1] 2 > result[1:2, 'score'] [1] 2 -5 > result[c(1,3), 'score'] [1] 2 4 > result[,'score'] [1] 2 -5 4
  • 16. Score the tweets To score all of the Delta tweets, just feed their text into score.sentiment(): > delta.scores = score.sentiment(delta.text, pos.words, Progress bar neg.words, .progress='text') provided by |==================================================| 100% plyr Let’s add two new columns to identify the airline for when we combine all the scores later: > delta.scores$airline = 'Delta' > delta.scores$code = 'DL’
  • 17. Plot Delta’s score distribution R’s built-in hist() function will create and plot histograms of your data: > hist(delta.scores$score)
  • 18. The ggplot2 alternative ggplot2 is an alternative graphics package which generates more refined graphics: > qplot(delta.scores$score)
  • 19. Lather. Rinse. Repeat To see how the other airlines fare, collect & score tweets for other airlines. Then combine all the results into a single “all.scores” data.frame: > all.scores = rbind( american.scores, continental.scores, delta.scores, jetblue.scores, southwest.scores, united.scores, us.scores ) rbind() combines rows from data.frames, arrays, and matrices
  • 20. Compare score distributions ggplot2 implements “grammar of graphics”, building plots in layers: > ggplot(data=all.scores) + # ggplot works on data.frames, always geom_bar(mapping=aes(x=score, fill=airline), binwidth=1) + facet_grid(airline~.) + # make a separate plot for each airline theme_bw() + scale_fill_brewer() # plain display, nicer colors ggplot2’s faceting capability makes it easy to generate the same graph for different values of a variable, in this case “airline”.
  • 21. Game Plan Search Twitter for airline mentions & collect tweet text Score sentiment for Summarize for each each tweet airline Load sentiment word lists Compare Twitter sentiment with ACSI satisfaction score Scrape ACSI web site for airline customer satisfaction scores 30
  • 22. Ignore the middle Let’s focus on very negative (<-2) and positive (>2) tweets: > all.scores$very.pos = as.numeric( all.scores$score >= 2 ) > all.scores$very.neg = as.numeric( all.scores$score <= -2 ) For each airline ( airline + code ), let’s use the ratio of very positive to very negative tweets as the overall sentiment score for each airline: > twitter.df = ddply(all.scores, c('airline', 'code'), summarise, pos.count = sum( very.pos ), neg.count = sum( very.neg ) ) > twitter.df$all.count = twitter.df$pos.count + twitter.df$neg.count > twitter.df$score = round( 100 * twitter.df$pos.count / twitter.df$all.count ) Sort with orderBy() from the doBy package: > orderBy(~-score, twitter.df)
  • 23. Any relation to ACSI’s airline scores? http://www.theacsi.org/index.php?option=com_content&view=article&id=147&catid=&Itemid=212&i=Airlines 18
  • 24. Game Plan Search Twitter for airline mentions & collect tweet text Score sentiment for Summarize for each each tweet airline Load sentiment word lists Compare Twitter sentiment with ACSI satisfaction score Scrape ACSI web site for airline customer satisfaction scores 33
  • 25. Scrape, don’t type XML package provides amazing readHTMLtable() function: > library(XML) > acsi.url = 'http://www.theacsi.org/index.php? option=com_content&view=article&id=147&catid=&Itemid=212&i=Airlines' > acsi.df = readHTMLTable(acsi.url, header=T, which=1, stringsAsFactors=F) > # only keep column #1 (name) and #18 (2010 score) > acsi.df = acsi.df[,c(1,18)] > head(acsi.df,1) 10 1 Southwest Airlines 79 Well, typing metadata is OK, I guess... clean up column names, etc: > colnames(acsi.df) = c('airline', 'score') NA (as in “n/a”) is > acsi.df$code = c('WN', NA, 'CO', NA, 'AA', 'DL', supported as a 'US', 'NW', 'UA') valid value > acsi.df$score = as.numeric(acsi.df$score) everywhere in R.
  • 26. Game Plan Search Twitter for airline mentions & collect tweet text Score sentiment for Summarize for each each tweet airline Load sentiment word lists Compare Twitter sentiment with ACSI satisfaction score Scrape ACSI web site for airline customer satisfaction scores 35
  • 27. Join and compare merge() joins two data.frames by the specified “by=” fields. You can specify ‘suffixes’ to rename conflicting column names: > compare.df = merge(twitter.df, acsi.df, by='code', suffixes=c('.twitter', '.acsi')) Unless you specify “all=T”, non-matching rows are dropped (like a SQL INNER JOIN), and that’s what happened to top scoring JetBlue. With a very low score, and low traffic to boot, soon-to-disappear Continental looks like an outlier. Let’s exclude: > compare.df = subset(compare.df, all.count > 100)
  • 28. an actual result! ggplot will even run lm() linear (and other) regressions for you with its geom_smooth() layer: > ggplot( compare.df ) + geom_point(aes(x=score.twitter, y=score.acsi, color=airline.twitter), size=5) + geom_smooth(aes(x=score.twitter, y=score.acsi, group=1), se=F, method="lm") + theme_bw() + opts(legend.position=c(0.2, 0.85)) 37 21
  • 30. R code for example scoring function score.sentiment = function(sentences, pos.words, neg.words, .progress='none') { require(plyr) require(stringr) # we got a vector of sentences. plyr will handle a list or a vector as an "l" for us # we want a simple array of scores back, so we use "l" + "a" + "ply" = laply: scores = laply(sentences, function(sentence, pos.words, neg.words) { # clean up sentences with R's regex-driven global substitute, gsub(): sentence = gsub('[[:punct:]]', '', sentence) sentence = gsub('[[:cntrl:]]', '', sentence) sentence = gsub('d+', '', sentence) # and convert to lower case: sentence = tolower(sentence) # split into words. str_split is in the stringr package word.list = str_split(sentence, 's+') # sometimes a list() is one level of hierarchy too much words = unlist(word.list) # compare our words to the dictionaries of positive & negative terms pos.matches = match(words, pos.words) neg.matches = match(words, neg.words) # match() returns the position of the matched term or NA # we just want a TRUE/FALSE: pos.matches = !is.na(pos.matches) neg.matches = !is.na(neg.matches) # and conveniently enough, TRUE/FALSE will be treated as 1/0 by sum(): score = sum(pos.matches) - sum(neg.matches) return(score) }, pos.words, neg.words, .progress=.progress ) scores.df = data.frame(score=scores, text=sentences) return(scores.df) } 39

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. delta.tweets[1] \n\n&quot;0533370677: RT @g: To the @Delta attendant at gate D39 in ATL: your attitude isn&apos;t the best, but you did get me on the earlier flight. Thank you. #ncslatl&quot;\n\n&amp;#x2018;original tweet has two parts: screenname and the text; we need to separate out the text&amp;#x201D;\n
  8. &amp;#x2018;This is one tweet, next will batch process the set&amp;#x2019;\n
  9. &amp;#x2018;can see here the text has been extracted&amp;#x2019;\n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n