1 - Raspberry Pi and Twitter API: text mining from Twitter stream with Twython

RASPBERRY PI AND TWITTER API: TEXT MINING FROM TWITTER STREAM WITH TWYTHON

- Layout for this exercise:






1 - Introduction

- The goal of this exercise is to get the Raspberry Pi interacting with Twitter in different ways.

- Raspberry Pi can run several libraries from the Twitter API. In particular, in this case Twython will be used.

- Along this exercise Raspberry Pi will write a Twitter message, also filtering some text from a Twitter stream and launching a message, and finally blinking a LED when an specific word is detected.


2 - Registering a Twitter app

- First all, let's assume that there is a valid Twitter account available.

- The first step to create a Twitter app and enabling the access to the Twitter API is to register it here :

https://apps.twitter.com/




- Signing in with Twitter account credentials:




- Creating a new app:




- Some data (name, description of the app, website, ... ) must be provided and eventually the app whitelist_rb is created:




3 - Authentication keys

- During the process of registering the Twitter app 4 keys are created for authentication, and these keys must be written down or copied carefully, keeping them private to be used later at the Python program.

- The purpose of these 4 keys is to ensure that the transmission of the messages are properly authenticated whenever the app accesses the Twitter API.

- The initial two keys are created automatically, Consumer Key (API Key) and Consumer Secret (API Secret):




- Also, and Access Token is created by the user:




- So 2 additional keys are created, Access Token and Access Token Secret:




- To sum the 4 keys up into a single text, the initkeys.py program is used (let's notice that the keys have been renamed as c_k, c_s, a_t, a_s):



* These values are no longer valid because the application was deleted after publishing this post.


4 - Installing Twython

- Twython is a Python wrapper for the Twitter API, allowing both querying data into the Twitter messages stream as well as uploading images, updating status, changing users settings, etc ...

https://twython.readthedocs.io/en/latest/





- To install Twython on the Raspberry Pi follow these two commands (pip is an installer for Python packages):






5 - Publishing a Tweeter message

- Now, let's publish a tweet from the Raspberry Pi using a Python script with the interactive mode:




- Almost immediately the tweet is published:




- Let's analyze the code. First, the Twython class is imported:




- Then, the 4 keys are invoked using execfile and passing as a parameter the file initkeys.py, what contains or defines the 4 authentication keys:




- Constructing the Twython object (a) and passing the 4 keys as parameters:




- Finally, update_status allows to pass a text or message as a parameter to the Twython object:





6 - Text Mining and Sentiment Analysis

- Text Filtering or Text Mining with Twython Streamer is a very powerful tool offered by the Twitter API that can be applied to the so called "sentiment analysis".

- Sentiment analysis (sometimes known as opinion mining or emotion Artificial Intelligence) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information. Sentiment analysis is widely applied to voice of the customer materials such as reviews and survey responses, online and social media, and healthcare materials for applications that range from marketing to customer service to clinical medicine.

https://en.wikipedia.org/wiki/Sentiment_analysis

- Twitter can be used as a valid online indicator of a very diverse range of sentiments, for instance related to politics, music, sports, brands, art, commerce, trends, religion, etc ...

- Twython Streamer object class allows to examine public Twitter streams.

- In the next Python program a class MyStreamer is written so that once the term "Spain" is detected at least 5 times on the stream of Twitter messages an alert or message is launched:




- Giving permissions to the Python program to be run:




- Executing the program:




- Now, 5 messages are created including the word "Spain":




- As a result, the program launches the message "Spain is popular!"




- Let's analyze the code of the program. First, the TwythonStreamer is imported:




- The 4 authentication keys are assigned with execfile passing the file initkeys.py as a parameter:






- Then, the class MyStreamer (extension of TwythonStreamer) is written, what counts from 0 to 5 the times that an specific text ("Spain") matches the content of the Twitter messages stream. Also, when the counter arrives to 5 a message is printed ("Spain is popular!"):




- The 4 keys are passed as parameters to the class:




- Finally, the method statuses.filter() searches for the text "Spain" in the Twitter stream:





7 - LED blinking after publishing a tweet with an specific text

- The layout for this section of the post connects a red LED to the Raspberry Pi module:







- A program is created so that the Raspberry Pi responds by blinking a LED starts whenever the text "fire" is detected into the Twitter stream:




- Giving permissions to run the program:




- Running the program:




- Sending a tweet with the text "fire":




- As a consequence of publishing a Twitter message with the text "fire" the red LED starts blinking, as it can be checked in this video:

10 - Arduino: sensing light with a Photoresistor

ARDUINO: SENSING LIGHT WITH A PHOTORESISTOR

- Layout for this exercise:





1 - Introduction

- The purpose of this exercise is to demonstrate how an Arduino board can be used to implement a safety measure consisting of establishing a line between a source of light (LASER or flashlight) and a photoresistor, so that when the light line breaks the alarms are triggered:

https://www.youtube.com/watch?v=mr834Cs9ncs

https://www.youtube.com/watch?v=KX2_LCUkhDs


- A photoresistor is a light controlled variable resistor, occurring that the resistance decreases with increasing incident light intensity, exhibiting photoconductivity.

- A photoresistor can be applied in light sensitive detector circuits, and light and dark activated switching circuits.

https://en.wikipedia.org/wiki/Photoresistor

- The schematics for this exercise is pretty simple, the only remarkable issue is to connect one of the legs of the photoresistor to the analog pin A0, where the light level readings are received.


2 - Code

- Code for this exercise:









- Let's analyze the code.

- First of all, the pins for the photoresistor (A0), buzzer (11) and RGB LED (3,5,6) are set:




- The photoresistorPin is directly connected to the analog A0, so the input reading measures the light received from the enviroment:

 


- If the reading is below the threshold 500 nothing happens, for instance when a LASER  or a great source of light is pointing to the photoresistor.

- However, if the reading goes above the threshold 500, for instance when the light line of sight is broken between the source of light and the photo cell, both the buzzer alarm and the Colors() function are triggered, and also an ALARM message is printed at the serial monitor:



- Finally, the functions Colors() and noColors() determine the combination of the RGB LED lights when the alarm is triggered and when the system runs normally.


3 - Running the program

- The serial monitor displays readings from the analog pin A0, showing an ALARM message only when the reading goes above the threshold 500:




- Testing the circuit, in this video it can be checked how the system is quiet when a flashlight points to the photoresistor. However, as expected all the alarms go off when the flashlight is retired:

9 - Arduino: Shift Register

ARDUINO: SHIFT REGISTER

- Layout for this exercise:





1 - Introduction

- The 74HC595 Shift Register (SR) used in this exercise is an Integrated Circuit (IC) with 8 digital outputs that can be controlled with just 3 digital pins of the Arduino. To increase the number of digital outputs multiple SR could be daisy-chained, allowing unlimited number of outputs with the same 3 digital pins of the Arduino.

- Pinning of the 74HC595 Shift Register:

https://www.arduino.cc/en/uploads/Tutorial/595datasheet.pdf



























- In this exercise two examples are explained using a Shift Register chip connected with an Arduino board.



2- Stepping up and down 8 red LEDs 

- The goal of this fist exercise is to design a lighting sequence of 8 red LEDs from first to last and in reverse order.

- Arduino sketch:




- Let's analyze the code.

- The SR uses the SPI (Serial Peripheral Interface) serial connection with 3 pins:

- A global variable is used to send data to the SR:




- The 3 pins are set in output mode:




- The loop includes the function UpDown():




- The Serial Peripheral Interface (SPI) has a clock line that controls the speed of the data transfer. shiftIn() and shiftOut() are commands used to access parts that use the SPI interface.

- shiftOut() shifts out a byte of data one bit at a time, starting from either the most (in this case, MSBFIRST) or least significant bit. Each bit is written in turn to a data pin, after which a clock pin is pulsed (HIGH and LOW) to indicate that the bit is available.

- The shiftWrite() function that allows to put the SR outputs into HIGH or LOW state is composed of this subfunctions:

• bitWrite() makes individual bits of "data" (a byte) on/off,  where "pin" is the SR output from 0 to 7 and "state" is either HIGH or LOW, 
•  shiftOut() shifts out a byte of data one bit at a time, starting from either the most (in this case MSBFIRST) or least significant bit. Each bit is written in turn to a data pin, after which a clock pin is pulsed (HIGH and LOW) to indicate that the bit is available.
• Toggling the pinLatch the shift register makes appear the data to the outputs, activating the high to low transition.

- The function UpDown() defines how the 8 red LEDs are lighted one after the other in two consecutive sequences (up and down), using the shiftWrite function for each LED (from 0 to 7) and a delay of 750 milliseconds:





- Testing the circuit and the code, the sequence of 8 LEDs light consecutively stepping up and down:

3 - Binary counter

- The goal of this exercise is to display the 0 to 255 binary bit patterns using the 8 red LEDs.

0 =     00000000

1 =     00000001

2 =     00000010

3 =     00000011

4 =     00000100

5 =     00000101

.........................

.......................... 

253 = 11111101

254 = 11111110

255 = 11111111

- The Arduino sketch: 


















































- Let's analyze the code.

- The first part is identical to the previous exercise, so we will go directly to the section related with the function BinaryNumbers().

- BinaryNumbers() function is called into the loop():




- The function BynaryNumbers() increases every 1 second the value of the data, and writes out (shifts out, better said) that data to the 8 red LEDs:





- Testing the circuit and the code, the 8 red LEDs show the numbers from 0 to 255 in a binary representation :

8 - Arduino: decoding TV remotes infrared signals with an IR receiver

ARDUINO: DECODING TV REMOTES INFRARED SIGNALS WITH AN IR RECEIVER

- Layout for this exercise:




- The TV remotes used in this exercise are of very common brands, like Samsung and Sony:





1 - Introduction

- Infrared (IR) radiation has slightly longer wavelength and shorter frequency than visible light, just next to the red color, but it is undetectable to the human eye. 

https://en.wikipedia.org/wiki/Infrared

- IR radiation can be used as a wireless communication technology. 

- IR is used in so many environments, for instance for turning on/off TV with a remote, and for selecting different options like channels, brightness, volume, etc ...

- A TV remote uses an IR transmitter that sends information to an IR receiver on the TV device, with usually a modulated frequency of 38 KHz. 

- The goal of this exercise is double: on the one hand to decode the IR signals from two different TV remotes, on the other hand to use the TV remotes to make colored LEDs to blink, taking advantage of the hexadecimal values of the decoded IR signals:

https://en.wikipedia.org/wiki/Remote_control#Technique

- About the schematics used in this exercise, the IR sensor has got three legs connected in this way:

• VCC = 5 V
• GND = GND
• Data = pin 11

2 - Decoding TV remotes with an IR receiver

- Before using the Arduino with an IR sensor the IRremote library must be installed:




- The Arduino sketch for decoding the TV remote signals:



- Once the TV remotes are pointed to the IR sensor and pressed some buttons the serial monitor outputs the hexadecimal values of the decoded signals:





- Actually, in this exercise just the two first values will be used, corresponding to the "power" buttons of the Samsung and Sony remotes:

• 0xE0E040BF
• 0xA90 (three times)

- Testing the decoding process:

- Let's analyze the code of the Arduino sketch.

- First of all, the IRremote library is imported:




- The pin 11 is assigned to the receiving and the decoding process, because it is connected to the data leg of the IR sensor, and a receiver object is created:




- The serial monitor is set to 9600 bauds, and the receiving process is started:



- If a code is received True is returned, the value is stored in the variable "results", and the serial monitor prints that value. Finally, irrecv.resume() resets the receiver and prepares it to detect another code:





3 - Blinking LEDs with remotes

- Now, two LEDs are handled with the TV remotes, using the Sony remote to turn on the yellow LED and the Samsung remote to turn on the red LED.

 - Code for the Arduino sketch:




- Let's analyze the code.

- First of all, the values achieved at the previous exercise are declared:




- The pins are assigned and a receiver object is created:








- The serial monitor is set to 9600 bauds, the receiving process is started, and also the LED pins are set to output mode:




- As explained before, if a code is received True is returned, the value is stored in the variable "results", and the serial monitor prints that value. Finally, irrecv.resume() resets the receiver and prepares it to detect another code:




- Now, depending on the value of the received code, either matching 0xE0E040BF or 0xA90, the yellow or red LEDs start blinking every half second:





- Testing the circuit, when the Samsung remote's "power" button is pressed the yellow LED blinks, and when the Sony remote's "power" button is pressed the red LED blinks:

7 - Arduino: strobe light effect

ARDUINO: STROBE LIGHT EFFECT

- Layout for this exercise:






1 - Introduction

- The objective of this exercise is to achieve the light effect of strobe based on emitting flashing light by 8 red and yellow LEDs, simulating the flag of Spain.

- The strobe light effect is a visual phenomenon caused by aliasing that occurs when continuous motion is represented by a series of short or instantaneous samples. 

- It occurs when the view of a moving object or blinking light is represented by a series of short samples as distinct from a continuous view at a rate close to the sampling rate.

https://en.wikipedia.org/wiki/Stroboscopic_effect

- The schematics of this exercise is an Arduino board connected to 8 LEDs and a potentiometer connected to the analog pin A0. 



2 - Code

- The code for this exercise:




- Let's analyze the code.

- Setting the pins for the couple of LEDs (digital pins 12,11,10 and 9) and the potentiometer (analog pin A0):




- The readings on the A0 are calibrated from 0 to 1023 giving an interval of 10 to 500 ms, what is a time enough short so that the human eye perceives the light as an overlay of flashes leading to the strobe effect.




- As seen in the previous section the function repeater is applied to the digital pins 12, 11, 10 and  9:




3 - Testing the circuit

- The video shows how the blinking light speed is controlled by the potentiometer in the interval of 10 to 500 ms, simulating the flashing colors of the Spanish flag:

6 - Arduino: Relay

ARDUINO: RELAY

- Layout for this exercise:




1 - Introduction

- A relay is an electromechanical switch that can control much more voltage and current than an Arduino pin.

- For instance, a relay could be useful to connect an Arduino to a device like a lamp or any other electrical machine of 120 V.

- A relay consists of a coil of wire and switch contacts. When power is applied to the coil it becomes magnetized and pulls the switch contacts closed. 

-  A transistor is used to handle the current because in this case much more power is needed than an Arduino can provide.

-  There are 3 contact pins used to connect the device handled by the relay:

• COM = common            <- connected to power
• NC = Normally Closed  <- connected to the yellow LED
• NO = Normally Open    <- connected to the red LED

- Also, there are 2 more pins for the coil used to control the relay. One side is connected to the collector of the transistor and the other one to 5 V.

- The relay can be in 2 different modes:

• OFF = COM is connected to NC
• ON = COM is connected to NO

- Also, a diode is used to protect the transistor from the voltage spike generated when power is disconnected from the coil, what could damage the transistor. One leg of the diode is connected to the transistor collector and the other to the coil.


2 - Code

- Arduino sketch:




- Let's analyze the code.

- First of all, the transistor base is assigned to the Arduino pin 4 in output mode:



- Now, the transistor base drives current (pin 4 is set to HIGH) energizing the coil and the switch contacts are closed. As a consequence COM connects to NO and the relay is ON for 2 seconds (the red LED lights). 






- Then, when the transistor base is LOW the COM connects to NC and the relay puts into OFF mode for another 2 seconds (the yellow LED lights):



3 - Testing the circuit and the code

- In this video the circuit and the code are tested. The 2 LEDs light alternatively every two seconds and the relay contacts "click" is heard very clearly: