Common Ground Initiative

Using Twitter to Measure Political Sentiment

February 6, 2011

I stumbled across this very interesting article on how Twitter can be used to track public sentiments in real-time on the New Media Strategies blog. The following post by NMS Senior Online Analyst Mark LeMunyon analyzes Twitter discussion during last week’s State of the Union Address to gauge sentiment and reactions to the speech. Read more about his findings from his post below.

Twitter Sentiment Analysis: 2011 State of the Union Address

By: Mark LeMunyon of New Media Strategies
Published:
February 2, 20011 on the NMS Blog

This is small project I whipped together by pulling information off Twitter’s API during the 2011 State of the Union Address, and then running the data through a senitment analysis to gauge audience reaction throughout the speech. The result:

This graph represents the ratio of positive words to negative words over time during the SOTU.

For reference, Pres. Obama’s speech began and ended at approximately 9:11PM and 10:13PM. To compare, here’s a timeline of selected quotes and topics discussed during the speech:

  • 21:18 – Economy, international competition
  • 21:20 – Need for innovation.
  • 21:24 – “This is our generation’s Sputnik moment.”
  • 21:24 – Government investment in biotechnology, energy
  • 21:26 – “I’m asking Congress to eliminate the billions in taxpayer dollars we currently give to oil companies.”
  • 21:28 – Education.
  • 21:33 – “We’ve ended the unwarranted taxpayer subsidies that went to banks, and used the savings to make college affordable for millions of students.”
  • 21:36 – Immigration reform in regards to education.
  • 21:36 – Tax reform.
  • 21:45 – “If a bill comes to my desk with earmarks inside, I will veto it.”
  • 21:54 – Salmon joke.
  • 21:59 – al-Qaeda.
  • 22:01 – Foreign affairs, Obama’s planned trips.
  • 22:03 – Independence of South Sudan.
  • 22:04 – Uprising in Tunisia.
  • 22:06 – DADT, military recuiting on college campuses.

If you’re interested in further comparing the data to the speech, I recommend the White House video of the SOTU on Youtube, which has a full transciption timeline. Simply add 9 hours and 11 minutes to the video time to estimate the time in EST.

The data includes 240,000 tweets which use the term “sotu”, “#sotu”, “stateoftheunion” or “state of the union”, sent between 9:00PM and 10:30PM EST on 1/25/2011.

Every tweet was compared to a list of words with predetermined sentiment scores (from the University of Pittsburgh’s OpinionFinder subjectivity lexicon). Each tweet was given two scores, one for the number of matched positive words and another for number of matched negative words. The scores were then summed up by 10 second time intervals, and a sentiment ratio (total positive words / total negative words) was generated for each 10s interval. The ratio is represented on the above graph as the gray line. The magenta line is a 60 second moving average, and the navy line is a 5 minute moving average.

This methodology is nearly identical to that of a study by Carnegie Mellon (O’Connor et al. 2010), which was featured on Mashable and other blogs. As noted in that study, there are lexicons that may work much better with Twitter, as this one was developed for proper English, which is something of a rarity on Twitter. (“#zomg4reels.”)

Again, this isn’t an extremely rigorous peer-reviewed scientific analysis, but it does provide some interesting fodder for discussion around how people reacted to the State of the Union Address.

Focus on More Engagement Before More Fans

January 14, 2011

Interaction Quality Over Quantity

What’s the most important metric to measure the success of your Facebook page? Total Fans? Wrong, although having thousands or millions of users as fans or “likes” on your page represents vast potential (just ask President Obama), it doesn’t automatically spell success. Bottom line: Your fans are only as valuable as how engaged they are. Don’t get too caught up in the raw numbers of total fans, if your fans are not being activated to become involved with the content of your organization or campaign, then they’re simply just a number, not yet a valuable asset.

Create an Environment for Noisy Fans!

Sports fans know that the loudest football stadiums are not always the biggest. I can personally attest this. I’ve been to football games at both small and large stadiums such as the Rose Bowl where 100,000 fans are in attendance and our very own, Autzen Stadium, where the capacity is about 58,000.

Which is louder? It’s not even close, Autzen is way louder! Despite only having a capacity of 58,000, substantially less than most BCS college football stadiums, Autzen is continually ranked among the loudest stadiums in the nation because the fans are extremely engaged in the game and their beloved Ducks. Facebook pages are not unlike Football stadiums, where it’s not all about the number of total fans, but rather how noisy and engaged they are.

Measuring Engagement

Before you move forward with trying to get more users to like your page, think about how you improve the overall engagement of your existing fans. The is place to start for this is Facebook Insights, the free analytical software that comes with your Facebook page. Insights enable you to continually measure the interaction levels and success of your engagement strategies by month, week and even down to an individual post. The ultimate goal is to get as many of your lifetime Likes or fans to convert to active users in any given month.

Insights Overview

In the Insights example below, 1,279 out of 2,153 (60%) total fans are listed as “Active” because they’ve either made a comment, “liked” a post or viewed a post in the last month, in other words they’re actively interacting with the content. The green arrow under ‘Monthly Active Users’ with the 4.9% represents the increase in Monthly active fans from the previous month. If this number is going down, then you’ll want to see why and evaluate your page content. You can also drill down into interaction levels for individuals post when go into ‘details’ section within Insights.

For a comprehensive guide to using Facebook Insights, check out this Mashable blog post.

Testing Engagement

To achieve better user engagement with your posts try testing various strategies to what works best. Based the results of the performance metrics from Insights, you can then quantify and continually refine what works better through A/B testing of your strategies. Here are some messaging strategies and tactics that you can test to increase overall engagement:

  • Pose a Question and Encourage Discussion: Based on the content of your post, ask users what they think. For example; if you’re posting a news article, insert a question about it and ask for feedback from your fans.
  • Create Two Way Dialogue: Respond and thank fans who interact with posts and provide feedback. This will send a message to other page members that interaction is valued and encouraged.
  • Ask For Tips and Advice: Ask your fans for advice and tips on a current issue, policy position or problem your campaign or organization may be experiencing. Facebook users like to help and offer insight when they’re asked, plus it makes them feel valued.
  • Be a Resource: When you come across helpful or insightful content related your campaign or industry, go ahead and post it to your Facebook page because it’s likely your fans would also find it interesting. Don’t assume everybody already saw the content or article, pass it along.
  • Timing: Play around with what time of day you post and test what results in better interaction. Is it best to post during the day? Early morning? Late night? Depends on the demographics of your audience; teenagers will be up late on Facebook, while older adults may be on at 5:00am.
  • Let the Audience Choose: After you post a photo album, ask fans which is their favorite photo or post an opened question, like what’s the best part about being a Republican…
  • Mix It Up: Don’t be afraid to post content that’s not related to your industry or message. Remember, not everyone wants all politics all the time, use relevant current events to put a human face on your Facebook page. If the Civil War game is this weekend, ask your fans who going to win the game.

What did I miss? What are some strategies that you found that work for your organization? Please share and ask.