-
Notifications
You must be signed in to change notification settings - Fork 0
Tutorial: Twitter product comparison
This tutorial will guide you through creation of the pipeline that will compare sentiments from Twitter users about several products. It assumes that you have either registered at cloud version of Exynize or deployed your own copy of the platform, and finished the "Hello world" tutorial.
First, we'll create a source component that will connect to Twitter API and will continuously deliver us messages of interest.
To simplify the creation, we'll rely on twit npm package.
Here's how the code will look:
import Twit from 'twit';
export default (
consumer_key,
consumer_secret,
access_token,
access_token_secret,
filter_lang,
keyword,
obs
) => {
// create Twit client with user keys
const T = new Twit({
consumer_key,
consumer_secret,
access_token,
access_token_secret,
});
// get status feed filtered by keyword(s)
const stream = T.stream('statuses/filter', {track: keyword});
// listen for incoming tweets
stream.on('tweet', (tweet) => {
// construct the data we're interested in
const data = {
id: tweet.id,
created_at: tweet.created_at,
text: tweet.text,
username: tweet.user.name,
lang: tweet.lang,
url: `https://twitter.com/${tweet.user.screen_name}/status/${tweet.id_str}`,
};
// filter by language
if (filter_lang === data.lang) {
obs.onNext(data);
}
});
// don't forget to listen to errors
stream.on('error', error => obs.onError(error));
};This component will keep dispatching tweets as they come in without ever completing (so, it has to be stopped manually).
Next, we'll create a simple sentiment component that will calculate sentiments using the input text.
Once again, we'll use existing npm package called sentiment.
Here's how the code will look:
import sentiment from 'sentiment';
export default (data) => {
// calculate sentiments
const res = sentiment(data.text);
// append new field to incoming data
data.sentiment = {
score: res.score,
comparative: res.comparative,
};
// dispatch new data
return Rx.Observable.return(data);
};This processor will first calculate sentiments for the text field of incoming data object, then it will append sentiments to the data object as new sentiment field and return new data object.
You can test this by entering {"text": "all is good"} into data field in Exynize editor and hitting "Test" button.
After test succeeds, hit the "Save" button to save your new processor component.
Finally, we need to create a render component that will display results. We'll create a simple one that will split the incoming data into two preset columns with hard-coded product names. Here's how the code will look:
// renders one tweet
const renderTweet = (tweet) => (
<div className="list-group" key={tweet.id}>
<div className="list-group-item">
<div className="row-content">
<div className="least-content">
<span className={'label label-' + (
tweet.sentiment.score === 0 ? 'default' :
tweet.sentiment.score > 0 ? 'success' : 'danger'
)}>
{tweet.sentiment.score}
</span>
</div>
<h4 className="list-group-item-heading">
{tweet.username}
</h4>
<div className="list-group-item-text">
<p className="text-muted">{tweet.text}</p>
</div>
</div>
</div>
</div>
);
// renders total sentiment for given collection
const overallSentiment = (collection) => (
<h3>
Positive:
<span className="label label-success" style={{marginLeft: 10, marginRight: 10}}>
{collection.reduce((sum, it) => sum += it.sentiment.score > 0 ? it.sentiment.score : 0, 0)}
</span>
Negative:
<span className="label label-danger" style={{marginLeft: 10, marginRight: 10}}>
{collection.reduce((sum, it) => sum += it.sentiment.score < 0 ? it.sentiment.score : 0, 0)}
</span>
Total:
<span className="label label-info" style={{marginLeft: 10, marginRight: 10}}>
{collection.reduce((sum, it) => sum += it.sentiment.score, 0)}
</span>
</h3>
);
// main render generator function
export default () => React.createClass({
render() {
const iphone = this.props.data.filter(tweet => tweet.text.toLowerCase().indexOf('iphone') !== -1);
const galaxy = this.props.data.filter(tweet => tweet.text.toLowerCase().indexOf('galaxy') !== -1);
return (
<div className="container-fluid" style={{marginTop: 20}}>
<div className="row">
{/* iphone col */}
<div className="col-xs-6">
<h1>iPhone 6s</h1>
{overallSentiment(iphone)}
{iphone.slice(0, 10).map(renderTweet)}
</div>
{/* galaxy col */}
<div className="col-xs-6">
<h1>Galaxy S6</h1>
{overallSentiment(galaxy)}
{galaxy.slice(0, 10).map(renderTweet)}
</div>
</div>
</div>
);
}
});This component will render two columns - one for tweets containing word "iphone" and another one for tweets with word "galaxy".
Both columns will display 10 most recent tweets with corresponding sentiments, as well as overall sentiment of the product calculated from all fetched tweets.
Now that all the components have been created, we need to assemble them into a pipeline.
The procedure is very much the same as in "Hello world" tutorial, except we'll need to pass some configuration options.
When adding Twitter source, you'll need to provide your Twitter API access keys.
You can find them in your applications console on Twitter website.
Additionally, make sure to provide the following keywords: iphone 6s,galaxy s6 and set filter language to en.
Processor does not require any configuration - just adding it is sufficient.
Finally, no configuration is required when adding render component since we'd hardcoded everything.
Make sure to test the pipeline by pressing "Test" button before saving it using the "Save" button.
Now that you've assembled, tested and saved your new pipeline, you can start it and view the rendered result by clicking "Web" button next to pipeline name.