RASPBERRY PI AND TWITTER API: TEXT MINING FROM TWITTER STREAM WITH TWYTHON
- Layout for this exercise:
1 - Introduction
- The goal of this exercise is to get the Raspberry Pi interacting with Twitter in different ways.
- Raspberry Pi can run several libraries from the Twitter API. In particular, in this case Twython will be used.
- Along this exercise Raspberry Pi will write a Twitter message, also filtering some text from a Twitter stream and launching a message, and finally blinking a LED when an specific word is detected.
2 - Registering a Twitter app
- First all, let's assume that there is a valid Twitter account available.
- The first step to create a Twitter app and enabling the access to the Twitter API is to register it here :
https://apps.twitter.com/
- Signing in with Twitter account credentials:
- Creating a new app:
- Some data (name, description of the app, website, ... ) must be provided and eventually the app whitelist_rb is created:
3 - Authentication keys
- During the process of registering the Twitter app 4 keys are created for authentication, and these keys must be written down or copied carefully, keeping them private to be used later at the Python program.
- The purpose of these 4 keys is to ensure that the transmission of the messages are properly authenticated whenever the app accesses the Twitter API.
- The initial two keys are created automatically, Consumer Key (API Key) and Consumer Secret (API Secret):
- Also, and Access Token is created by the user:
- So 2 additional keys are created, Access Token and Access Token Secret:
- To sum the 4 keys up into a single text, the initkeys.py program is used (let's notice that the keys have been renamed as c_k, c_s, a_t, a_s):
* These values are no longer valid because the application was deleted after publishing this post.
4 - Installing Twython
- Twython is a Python wrapper for the Twitter API, allowing both querying data into the Twitter messages stream as well as uploading images, updating status, changing users settings, etc ...
https://twython.readthedocs.io/en/latest/
- To install Twython on the Raspberry Pi follow these two commands (pip is an installer for Python packages):
5 - Publishing a Tweeter message
- Now, let's publish a tweet from the Raspberry Pi using a Python script with the interactive mode:
- Almost immediately the tweet is published:
- Let's analyze the code. First, the Twython class is imported:
- Then, the 4 keys are invoked using execfile and passing as a parameter the file initkeys.py, what contains or defines the 4 authentication keys:
- Constructing the Twython object (a) and passing the 4 keys as parameters:
- Finally, update_status allows to pass a text or message as a parameter to the Twython object:
6 - Text Mining and Sentiment Analysis
- Text Filtering or Text Mining with Twython Streamer is a very powerful tool offered by the Twitter API that can be applied to the so called "sentiment analysis".
- Sentiment analysis (sometimes known as opinion mining or emotion Artificial Intelligence) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine.
https://en.wikipedia.org/wiki/Sentiment_analysis
- Twitter can be used as a valid online indicator of a very diverse range of sentiments, for instance related to politics, music, sports, brands, art, commerce, trends, religion, etc ...
- Twython Streamer object class allows to examine public Twitter streams.
- In the next Python program a class MyStreamer is written so that once the term "Spain" is detected at least 5 times on the stream of Twitter messages an alert or message is launched:
- Giving permissions to the Python program to be run:
- Executing the program:
- Now, 5 messages are created including the word "Spain":
- As a result, the program launches the message "Spain is popular!"
- Let's analyze the code of the program. First, the TwythonStreamer is imported:
- The 4 authentication keys are assigned with execfile passing the file initkeys.py as a parameter:
- Then, the class MyStreamer (extension of TwythonStreamer) is written, what counts from 0 to 5 the times that an specific text ("Spain") matches the content of the Twitter messages stream. Also, when the counter arrives to 5 a message is printed ("Spain is popular!"):
- The 4 keys are passed as parameters to the class:
- Finally, the method statuses.filter() searches for the text "Spain" in the Twitter stream:
7 - LED blinking after publishing a tweet with an specific text
- The layout for this section of the post connects a red LED to the Raspberry Pi module:
- A program is created so that the Raspberry Pi responds by blinking a LED starts whenever the text "fire" is detected into the Twitter stream:
- Giving permissions to run the program:
- Running the program:
- Sending a tweet with the text "fire":
- As a consequence of publishing a Twitter message with the text "fire" the red LED starts blinking, as it can be checked in this video:
