Walk all Manhattan

Background

I moved to New York about 6 months ago. I walked to work every day and one day I wondered if there was a street I subconsciously avoided on my way to and from work. I started a few attempts to track this, writing it down in a note book, plotting it on a Google My Maps. Both proved to be unsatisfying and buggy. At the same time, discussion with friends about my time in New York inspired the question, how much of New York had I actually seen and whether it was possible to walk all the streets of Manhattan.

As a result, I started writing an application to help me to track all the streets I had walked. At first, I looked in to whether I could do it with existing tools. Latitude proved to be too noisy and inaccurate even at a street-level granularity. Other apps that used GPS drained my already insufficient battery to an unacceptable level. So instead, I created an app where I would manually plot the tracks using my iPhone.

The App

The iPhone app I wrote allows me to drop waypoints on to a map, move them around, correct them and then plot the path that I took. At first, I thought I would need to use Open Street Maps (OSM) to get the street level data and write my own routing algorithm. But later, I realized there is a Google Directions API which could plot a walking route between a set of waypoints.

Each route I add has a date and time attached and an optional name. The app stores the data in a local CoreData (iOS/Mac storage format) store. It also has an export feature to send JSON encoded data to my own server for inspection and more permanent storage.

The app has three modes, the first is to enter the routes, which is done on a simple map view where long presses will drop a waypoint as a pin. Each change of the waypoints triggers a Google Directions API call to route a reasonable walking route. That usually is the route I took, and if it isn't, I can just refine the waypoints until it resembles it.

The second mode allows me to review each of the routes and modify them such as modifying the waypoints, adding to them or adjusting the date. The list is in reverse chronological order.

The third mode is the most fun. It takes all the paths and plots them on a single map. That way, I can quickly see near me what streets I hadn't explored. It is also quite satisfying to see which places I've most explored and least visited.

Discoveries

Manual input could be time consuming, but I found the approach of only using way points made it super lightweight. I think this solution is superior to the auto tracking solution because I can backfill data from memory, unlike if this was fully auto.

I didn't know until after a while that Google Directions only supported 10 waypoints. IT means some walks I've had to split into multiple ones.

Google Directions API gives me the street name metadata. I could potentially use this to create a massive list of all the streets I've actually walked. Or make my walks searchable.

MapKit and Google Maps Polyline drawing leaves quite a bit to be desired in terms of clarity. Even though I've specified either a 1 or 2px line thickness, it often is thicker due to zooming.

Looking ahead

I've not thought about generalizing this for others, but I think this method is suitable for other purposes too, like cycling and driving and other cities. I think I will try to opensource the code or publish it to the AppStore for others to use. However, it will need a lot more work.

Having better autocompletions for locations (frequent locations like work and home), and maybe reminders using iPhone's geo-fencing capabilities to remind me to start recording would be useful.

Mixing in some other signals from Latitude and Foursquare would also be useful for recalling a journey after the fact.

Ability to override the paths returned from the Google Directions API is needed. I've discovered some interesting mapping errors using the directions API in this way. For instance, you'll notice that Park Avenue below the 40th street seems to be unwalkable according to directions. I've had to avoid plotting routes down it because of this bug. I hope this will be fixed soon.


I talked about this at the Quantified Self New York Meetup in May 2012. You can see some rather information free slides I presented. The video will be posted soon.


You can reply to me about this on Twitter: