How to build and deploy a Twitterbot on your free AWS EC2 instance and my three-month retrospective on the subject

A few months ago, I got a bee in my bonnet to build my own Twitterbot. This was quite the turn of events as I wasn’t even on Twitter a year ago. I didn’t quite understand the curious 140-character platform and was only spurred to join after taking over marketing at 3D Robotics. It was only fair that I liked and retweeted updates and release announcements while asking the same of the team.

Since then, I have embraced Twitter as my definitive source for tech news and trends. However, combing through the disorganized cacophony is challenging, especially after an industry change. Once I got my feet underneath me at Orbital Insight, I was faced with the daunting challenge of finding the best geospatial analytics/new space accounts to follow while purging the collection of drone accounts that I had so carefully curated. As I was building up a new list of accounts to follow, I noticed that many of them had similar characteristics–they were often mentioning Orbital Insight, Planet, Tellus Labs, Descartes Labs, or another new space company by name. Why manually comb through Twitter to find these interesting accounts when I could just spin up a bot to do the same thing while I fished for Mahi Mahi in the middle of the South Pacific?

Fortunately for me and my collection of side projects, this idea is not original. There are dozens of tutorials online explaining how to use Tweepy, the most popular Python library for accessing the Twitter API, or one of the equivalent node.js equivalents. And my goals were simple. I just wanted to follow and like accounts that were tweeting about new space. Rather than rewrite some excellent guides already out there, I’ll just leave the code here and make a few comments.

First, the block below accomplishes everything I had imagined. It grabs the top 50 tweets mentioning UrtheCast, a Planet competitor that sells cubesat and small SAR-sat imagery, favorites all of the tweets, follows all of the tweeters, and then takes a break.

for tweet in tweepy.Cursor(api.search, q=’@UrtheCast’).items(50):
print tweet.text

# Print User Name who tweeted
print ‘\nTweet by: @’ + tweet.user.screen_name
time.sleep(5)

# Favorite the tweet

try:

tweet.favorite()
print ‘Favorited the tweet’
time.sleep(5)
except tweepy.TweepError as e:
print(e.reason)

# Follow the user who tweeted

try:
tweet.user.follow()
print ‘Followed the user’
time.sleep(5)
except tweepy.TweepError as e:
print(e.reason)

time.sleep(60)

Second, Tweepy supposedly takes care of rate limiting and retries. I did not find this to be the case. Twitter favorites are throttled an unknown amount and the Twitter API returns an error once that limit has been reached. Make sure to wrap your calls with try and except, otherwise your bot will fail when it bangs against that limit. Finally, make sure to include some kind of logging in your bot’s code. Mine is obviously very simple, but there are some weird idiosyncrasies that you will encounter when deploying your code and a simple log on the server will help enormously in diagnosing problems. The Twitter API is extremely reliable, but you will get errors every now and then.

Now that the code is done, it’s time to deploy. Most of the tutorials out there use Heroku, which is a) not free and b) totally overkill for this application. Heroku is designed for hosting serious applications without having to worry about devops or infrastructure, not running simple scripts like Twitterbots on a remote server. Furthermore, their free, intro tier is only good for a couple months.

A better solution is to use the free EC2 instance to which every Amazon customer is entitled. If you are a human on planet earth, you probably fall into this category. To access this hidden benefit, point your browser at the AWS Console and log in with your plain vanilla shopper credentials.

Click on launch a virtual machine and select a free tier EC2 instance. Unlike Heroku, we can run this one around the clock without paying a dime, which is just perfect for our Twitterbot.

Once your machine is launched, you need to connect and upload your code. AWS will walk you through the process of using the private key, .pem file, that you downloaded while outfitting your server to connect over ssh. You will type something like

ssh -i “twitter.pem” ec2-user@ec2-34-214-28-117.us-west-2.compute.amazonaws.com

into the bash terminal. If you are successful, you will be logged into a remote, virtual machine sitting in an Amazon server farm somewhere in the middle of Virginia. To upload your code to your personal server, use the scp command below.

scp -i ~/.ssh/twitter.pem ~/Documents/twitter_bot.py ec2-user@ec2-34-214-28-117.us-west-2.compute.amazonaws.com:/home/ec2-user

Now that your code is in the cloud, you will need to provision your server just like your local machine. You could do this the fancy way using Docker or a virtual machine, but with a single dependency like your Twitterbot, it is easier to just pip install tweepy from your server’s command line and be done with it. Once Tweepy is installed, give your code a whirl. Type python twitter_bot, just like you would on your local machine and watch your script go to work. Done, right? No. We have one more step. When you run your code like this, it will interrupt as soon as your end the session.

The easiest way to circumvent this problem is using screen, a Linux utility that lets processes run in the background. Type screen twitter_bot and you will enter a fresh shell window. From there, execute your code as before, hit CMD+a+d (on a Mac) to exit that screen, and disconnect from the server. Your Twitterbot will run until the end of time on Amazon for free. If you want to check your log or kill your bot, type screen -R twitter_bot. If you forget what you’ve called your Twitterbot screen, type screen -ls to get a list of all the running processes. Using this technique, it is possible to run dozens of Twitterbots and other scripts on a single free server.

So what was the point of all of this? Was it worth the 30 minutes of effort? Absolutely. I would recommend a Twitterbot to all of my friends. Since launching my own Twitterbot a few months ago, I have roughly doubled my Twitter followers, quadrupled the monthly impressions of my Tweets, initiated a handful of interesting phone conversations with colleagues working in new space, generated a few inbound leads for Orbital Insight, and kicked off quite a few interesting Twitter conversations. Your Twitterbot will toil away day and night to fill the top of your personal curiosities funnel and probably throw off some professional dividends as well. The only downside is that occasionally I will follow a questionable account or like a strange Tweet, but this can be corrected by periodic manual intervention.

Some 40% of Twitter accounts are bots, and I totally see why. Twitterbots are fun, free, and come with tons of benefits. Get your own online today (and be sure and follow danielmckinn0n :-)).