Guest Post: How to hack Burning Man with APIs and text messages

Post contributed by Manuel – follow him on twitter @maebert and check out his personal blog here

Different people have different ideas about their perfect vacation. Some day-dream of relaxing on white beaches and palm trees. Others seek the thrill of snowboarding down the rockies. And then there are those who really just want to run through the desert half-naked, entangled in EL wire and exploding things to toss anvils 100ft into the air.

That’s me. And the probably only place on earth you can do this is the Burning Man festival. Problem is, I completely failed to get a ticket in time this year, and in between a lot of travelling I woke up with the sudden realisation that there were only five days left to find a ticket. The ePlaya forum had plenty of people offering tickets at face-value, however the demand this year was so high that the average time until a ticket got sold was less than four minutes. I was still stuck in Edinburgh, UK; and there was no possible way I could monitor that page all day round to call dibs on the next ticket offered.

Kimono Labs and my Inner Nerd to the rescue.

I would personally find it unethical to write a bot to get a ticket for me, but at least the process of sitting on a computer and hitting the refresh button can be automated. The battle plan is to turn the HTML mess of the ePlaya phpBB board which only barely graduated from nested tables into clean data with Kimono and then notify my whenever that data changed.

First, let’s identify what we need in our API. Here’s what a section of the forum looks like:

Kimono API

We obviously need the title of the post since we’re ultimately only interested in tickets offered, not tickets needed. The date of the post might also be interesting, as are the number of replies (since we won’t care unless we’re the first). To select the date and replies, simply highlight the text by clicking and dragging your cursor over it the way you would do it in a text editor:

Opening the raw data view, we can now see the the data, the whole data, and nothing but the data:

{
  "Title": {
    "text": "Oh... my... god...",
    "href": "https://eplaya.burningman.com/viewtopic.php?f=370&t=71750"
  },
  "Date": {
    "text": "Sat Aug 30, 2014 9:16 pm",
    "href": "https://eplaya.burningman.com/viewtopic.php?f=370&t=71750"
  },
  "Replies": "0"
}

Note how KimonoLabs neatly captures the URL to the individual thread, too. We’ll use that later.

Now, we can tell Kimono to send us an email whenever that data changes. However we’re

  1. not interested in getting email every time somebody posts a reply to an old thread and
  2. the lowest interval for automatic polling is 15 minutes.

So, let’s write a few lines of Python to deal with the data. We’ll use the requests module to fetch the API:

[code lang=python]
import requests
URL = "https://www.kimonolabs.com/api/6zzoaezg?apikey="
response = requests.get(URL).json()
data = response['results']['collection1']
[/code]

Now data will be a list of dictionaries like the one above. Next we need to need to find all new posts. Let’s have a closer look at the URLs in the title attribute:

https://eplaya.burningman.com/viewtopic.php?f=370&t=71750

Here, 370 is the id of the forum, and 71750 is the id of the topic. Before we run the script the first time, let’s create a file called latest_topic.txt and put 71750 inside. Back in Python, we load the file find find the newest topic, split every URL to find the topic id of the post, and if it’s newer than the newest topic from the last time we run the script, we remember this posts in a new_posts list. And while we’re on it, we also keep track of the maximum topic id we encounter so we can save it when we’re done finding new topics:

[code lang=python]
latest_topic = int(open('latest_topic.txt').read())
max_topic = latest_topic
new_posts = []

for post in data:
topic = int(post['Title']['href'].split('t=')[-1])
if topic > latest_topic:
new_posts.append(post)
max_topic = max(max_topic, topic)

with open('latest_topic.txt', 'w') as f:
f.write(str(max_topic))
[/code]

Great. Now let’s turn to our friends at Twilio for turning data into a concierge web notification service. Twill allows you to send text messages to any phone and takes less than 5 minutes to set up. Let’s write a short method that will take a post and send a text message to a number:

[code lang=python]
def send_message(post, number):
URL = "https://api.twilio.com/2010-04-01/Accounts//SMS/Messages.json"
text = "{}: '{}' ({} replies) {}".format(
post['Date']['text'],
post['Title']['text'],
post['Replies'],
post['Title']['href']
)
params = {
"From": "",
"To": number,
"Body": text
}
requests.post(URL, data=params, auth=("", ""))
[/code]

The format operation turns the post into a string such as

“Sat Aug 30, 2014 9:16 pm: ‘Oh… my… god…’ (0 replies) https://eplaya.burningman.com/viewtopic.php?f=370&t=71750”

Finally, a simple POST request to the Twilio API is enough to send this string to a number. All that’s left to do is sending a text message for every post in new_posts. But wait… have you heard of the term “geospatial arbitrage”? Instead of notifying only myself, let’s send this message to friends in three different timezones in the US, the UK and Japan to maximise the chance of one of us being able to reply first:

[code lang=python]
for post in new_posts:
for number in ('+1 415 123-4567', '+44 20 1234567' '+81 3 12345678'):
send_message(post, number)
[/code]

Done. If you’re on a free Twilio trial account, make sure all of the recipients’ numbers are verified. Last thing is to make sure this script runs, like, very often. Let’s save it as ticket_search.py and make an entry in our Crontab by typing crontab -e in our Terminal and adding a line

[code lang=sh]
* * * * * python ticket_search.py
[/code]

To make this script run once per minute.

Lean back.

At 7:04 PST in the morning, three phones on three different continents gently buzzed as part of our globally orchestrated concert of APIs and data, scripts and servers. I was the first one to tap the reply link, and sure enough:

I've got a golden ticket!

Incredibly happy and exhausted, I paypal’ed the money to the seller, wrote a hand-letter postcard to say thanks, and started packing my bag for Burning Man.

You can get the full script here.

2 thoughts on “Guest Post: How to hack Burning Man with APIs and text messages

  1. Matty says:

    wow. If I had instructions it would take me sooo long. Thanks for sharing how you did this. Tickets were a genuine hurdle this year. Glad you made it!

Comments are closed.