First things first, go HERE to see what this post is about. If you are interested in how I did this … keep reading.
Note: Nobody from Treefort Music Fest knows I did this, I just did it for fun.
Last year I made a heatmap to show the geographical layout of all the artists that would be playing at Treefort Music Festival. You can see that post here and the original heatmap here. For v2.0 I had a couple of ideas that I wanted to implement.
- Search
- Thumbnail pictures of bands
- Better layout
- Auto scrolling to bands location on the map
I started this task pretty much the same way I did last year. I went to treefortmusicfest.com/artists/ and right-click “View Page Source” (⌘ + ⌥ + u). This let me download the raw HTML of that page which I just pasted into Sublime Text. I immediately noticed that the repetitive nature of the <article class="portfolio-post-entry"> nodes. This allowed me to match with regular expressions all 348 artist entries that I could find. Then through a couple macros and some more creative search and replace I generated artists-2014.js which contained artist names, city and state, links to the artist page and the artist thumbnail, and the date they were playing at Treefort.
The only problem now (which I came across last year too) was that I could only scrape the city and state or city and country for the artists on the main artists page. Last year I solved this problem by figuring out all the unique location combinations and googling the latitude and longitude and manually copy and pasting these values, HUGE WASTE OF TIME. This year I decided to make use of the Google Maps Geocoding API which lets you submit basic location information (i.e., Boise, Idaho) and asynchronously sends you back a JSON response containing the latitude and longitude information according to The Google. One issue I came across was that the maps API limited me to 1 request per second, so I couldn’t use this API on every page render (not to mention that would kill my page load). I tried through timeouts not to go faster than every second but you can see from the screenshots that I failed.
To solve this problem I wrote the code on my local box and made http requests via chrome dev tools, then I wrote back (through the js objects) the lat and lng information. I had to run this process a few times since I went over the quota multiple times.
var idx = 0;
geocoder = new google.maps.Geocoder();
_.each(artists, function(artist){
if (!artist.lat)
{
setTimeout(function(){
geocoder.geocode( { 'address': artist.city}, function(results, status){
if (status == google.maps.GeocoderStatus.OK)
{
console.log("Artist: " + artist.name + " was great success");
artist.geometry = {};
artist.geometry.location = results[0].geometry.location;
artist.geometry.location_type = results[0].geometry.location_type;
}
else
{
console.log("Artist: " +
artist.name + " geocode was not successful for the following reason: " + status);
}
});
}, 1050 * idx++); // attempt at staying under my 1 request per second quota
}
});
Once all the artists had lat and lng information, I was able to JSON.stringify() the objects in memory and format them in Sublime Text. Now artists-2014.js was complete with lat and lng information and ready for prime time.
The js code was pretty straight forward, I ditched all the combining same locations and grouping on a chart marker with a crappy looking InfoWindow and went in favor of a list of artists (searchable) on the right hand side of the page. The list controls the map and makes markers and InfoWindows as hovers are detected on the list. The underlying heatmap is the exact same chart layer I was using before.
Truthfully the CSS was the hardest part for me, and there are still a few hacks and probably lots of errors but hey, it looks pretty good for now!
I did get some freebies this year from the treefortmusicfest.com/artists/ parsing:
- Band thumbnails (incorporated into the InfoWindow)
- Date of band playing (incorporated into the InfoWindow)
- Artist page links (click on list items or anywhere in the InfoWindow that pops up for each artist)
Bonus tasks:
- mod_rewrite learning on my wamp server, wanted to rewrite https://fosteri.zone/treefort/heatmap to https://fosteri.zone/treefort/heatmap.html so I didn’t look like such a n00b
Check out my code and give me honest feedback! Make sure you check out v1.0 as well to see the progression from 2013 to 2014