Foursquare API, and SSL, and OAuth. Oh, my!
Continuing our deeper dive into the nuts and bolts of how we built #mom, let’s walk through how we used the foursquare API. We we wanted the foursquare platform to: (1) handle user authentication, (2) supply general user data, (3) retrieve user checkins, and (4) promote the service by displaying #mom in the friend checkin feed.
Much like Twilio, our first step was to register our application with foursquare. They call this process “register a new consumer.” It made sense, after a while, since as a developer we are “consuming” foursquare data. Foursquare also asks you for a website and callback URL. We hadn’t settled on the hashtagmom.com domain yet (my original idea was theresafe.ly) but luckily it doesn’t matter what you put into this field. The callback URL is more important and a not well documented. We had issues with invalid redirect uri errors when specifying the full callback path (http://www.hashtagmom.com/path/to/callback) for our app. If instead you just give foursquare the hostname (http://www.hashtagmom.com/) they will allow you to specify any path under that host in your OAuth2 request, so that seems like the way to go. The foursquare API was also easier to work with locally than Twilio, as OAuth2 uses a browser redirect that works to localhost, so you don’t need a localtunnel set up. Please read our previous Twilio post if this distinction is unclear.
With the goal of connecting to foursquare and retrieving data from a simple standalone ruby program, I Googled “foursquare gem.” Unfortunately the first result, in my version of Google, is an out of date gem. I futzed with that until I stumbled upon a more recent gem called, wait for it, foursquare2 gem. Foursquare’s libraries page points to the latest gem so it’s always a good tip to just follow their official links rather than my brain-ingrained-let’s-Google-it approach. There is also a foursquare-api gem but I haven’t used it.
Ok, so we were able to query foursquare and retrieve data from endpoints that only required “userless” access permission. These are endpoints like get information about venues, tips, and lists. To access checkin details, user data, or many of the other interesting bits available in the platform, you need to act “as” the user— you need a user to authenticate your application with foursquare and explicitly get permission to query the platform as if you were that user.
The following is a bit of an aside and not specifically related to any particular platform but necessary to understand. You’ve likely connected to Facebook before. You’re on SomeGreatSite.com, they have a connect to Facebook button, you click it, and now you’re redirected to a Facebook page that read, “Hi, would you like SomeGreatSite to have access to x, y, z Facebook permissions?” Yup, hit yes and you’re redirect back to SomeGreatSite.com. This flow is brought to you by OAuth2
: check out this technical description
and foursquare’s overview
. From the developer’s perspective: you a user on your domain and want to query an API on that user’s behalf. Rather than ask for andstore a user name and password, you direct the user to the service and wait to hear back if you have permission or not. If you do get permission, you get a special token that is specific to that user. (Remember going to the arcade as a kid and getting special tokens that only fit into the video game machines?) Now, you can query the other service with this token and act as the user. The developer does not have to see or store the user’s authentication details. If the user wants to revoke your site’s access to the platform, they can just tell the platform to stop accepting that token. Change of password, no problem. Badaboom.
We’ve shipped the user off to foursquare but does foursquare know how to return the user back to us? How do we know which user they’ve sent back to us? The callback URL! You inform foursquare which application you are with your client ID and secret on the redirect call and foursquare calls your application back at your predefined URL. We can test this locally since a localhost callback URL will work. Just set up a route in Rails that handles the URL call: in our case this was /foursquare_callback and we handle this call in our Users controller. Foursquare returns the user’s access token in the callback URL which you can parse from the params hash. The next problem: which user are they referring to? We had a few options: (1) we thought about passing some user ID to foursquare and then parsing it back out of the callback but we’re not sure that would work (not to mention it seemed like a bad idea to pass a 3rd party an app specific user specific ID), (2) matching some foursquare user data against our user information but we already removed email address from the sign up process (and this also felt wrong at a gut level), (3) storing an ID in the browser cookie for that user’s session.
Rails easily allows you to store a session ID in the user’s browser’s cookie and access per-session data in your controllers. We bank on the fact that the callback URL will occur within the same session. Rails queries the user’s session from the cookie and then we look up which user had that session ID. We also store the OAuth2 access token (a string) in the user object and use it anytime we need to query foursquare on the user’s behalf.
It’s a windy road to get started with user authentication. You have to play by the platform’s rules. Once you have an access token, you’re ready to query!
We want the foursquare checkins as they happen. When someone includes “#mom” in his or her shout (foursquare’s field name for the 140 character message per checkin), we want to kick off a process that parses the message, looks up the user’s settings for call or text and tells Twilio to go. Foursquare has two ways to get checkins for a user: pull
and real-time push
. During the development process, we used the pull API to access the latest checkins and manually scheduled when to refresh this data. We’d briefly store the checkins then cycle through them looking for “#mom” and call the next step. This helped us manually test the service and we could even generate fake data within our database saving us an embarrassing number of #mom public checkins (though I generated like 20 of them in production the night we got it all working). We figured we could get push working later and guarantee that our data flow worked without the need for push. Push also requires SSL which I’m not aware you can do in local testing.
Handling both push and pull calls from foursquare was made simple by abstracting checkin processing into a Checkin model. This model knows how to send out calls or texts based on the information contained in a checkin, and doesn’t care how that checkin got created.
Pulling data from foursquare is more straightforward than receiving pushes. One tricky part is that foursquare doesn’t take care of de-duplicating the checkins they send you, so you have to make sure not to process the same checkin twice. Here’s our pull code, which fits nicely in the User model:
With this code, we can dump a User’s latest unprocessed checkins into our database, where we can manually process. We considered using the delayed job framework to pull and process checkins in the background, but eventually decided to take a shot at hooking into the push API.
For push, foursquare posts to your servers whenever an authorized user has checked in to a venue. The complicating issue that kept us from starting with push is similar to the problem we ran into with Twilio— it’s a pain to use these webhook APIs from a local development environment. Foursquare’s API was even more complicated in this regard, as they only allow pushing of data over SSL. This ruled out the localtunnel solution from Twilio, and we never succeeded in pushing data to our local servers. It would be awesome if foursquare could allow non-SSL connections for testing (maybe restricting to only getting your personal checkins pushed). We would also be happy with a solution similar to what Stripe
does with their Webhooks framework, where they send you a non-sensitive event ID that you can then use to make a secure request to their API from your server.
Regardless, we needed to get SSL up and running in production even if it wouldn’t work for development. We’re using heroku for our hosting and they offer free SSL via their *.herokuapp.com certificate. This SSL isn’t useful for user-facing pages,
since we’d have to redirect people away from www.hashtagmom.com to the scary-looking hashtagmom.herokuapp.com, but is perfect for giving foursquare a secure path to post to. Once we had a valid SSL-enabled path for foursquare, processing the push data itself is straightforward:
Getting push working was a huge win for us. We’ll process a checkin within seconds of it being made.
Like many things, the first time you get this all set up is much harder than the second attempt. This workflow touches many different technology stacks (HTTP, 3rd party APIs, local development, databases, browser sessions, security, user permissions, authentication through OAuth2, and a few others) so it’s not easy to just sit down and do it. I highly recommend working with a friend so you can catch each others mistakes and talk through what each part of your program intends to do before diving into a Google or StackOverflow-athon of article reading and reckless Github code copy and pasting: it won’t work. Writing about your efforts afterwords clarified the process as well; it feels good!
If you have any questions, would like to see more code snippets, or have other topics you’d be interesting in reading about please let us know.