PDA

View Full Version : Freezing Saddles Heat Maps



chris_s
02-04-2016, 02:54 PM
I think this is ready for prime-time now.

Freezing Saddles Heat Maps (http://dcbikemap.com/heat/)

You can view a map for any individual team, or for all participants. You can look at all rides, the last 24 hours of rides, or the last 7 days of rides.

All-time data updates once a day in the middle of the night.
Weekly data updates once a day in the middle of the night.
Last 24 hours data updates hourly on the hour.

BEWARE: viewing the heatmap for all-participants for all-time is kind of a beast and may cause your PC and/or browser to scream for mercy.

More to come once hozn starts returning me time-series data.

consularrider
02-04-2016, 03:01 PM
Cool, my Ukraine rides even show up.

Sunyata
02-04-2016, 03:18 PM
Oooh! Cool! Looks like Team 8-Ballin' has had riders in Korea and South Africa, in addition to Las Vegas! :cool:

S. Arlington Observer
02-04-2016, 03:51 PM
Excellent tool. Thank you!

Terpfan
02-04-2016, 03:55 PM
Can really see the influence of cycling infrastructure like MUPs to it.

DCAKen
02-05-2016, 10:50 AM
Who's been cycling in Antarctica?

Steve O
02-05-2016, 11:06 AM
Who's been cycling in Antarctica?
That's one way to get true badass FS cred

S. Arlington Observer
02-05-2016, 11:08 AM
Who's been cycling in Antarctica?

Actually, different paths show for each team, but every team (I checked) had somebody cycling there. Must be an issue with Strava. Perhaps a default location when all else fails?

DCAKen
02-05-2016, 11:10 AM
That's one way to get true badass FS cred

Nah, it's summer down there.

Raymo853
02-05-2016, 11:12 AM
This is simply a wonderful use of Strava's data openness. Any chance you mind my sharing it to a few folks beyond BAFS?

Also, any chance you would be willing to share your methodology?

americancyclo
02-05-2016, 12:02 PM
The "last 24 hours" view seems like it would be really useful after a big storm to see where folks abandon the MUPs for local roads when the ice or snow is too difficult to ride through.

LeprosyStudyGroup
02-05-2016, 12:15 PM
10877
Team 13 is looking bullish about your heatmaps ;)

chris_s
02-05-2016, 12:27 PM
Actually, different paths show for each team, but every team (I checked) had somebody cycling there. Must be an issue with Strava. Perhaps a default location when all else fails?

Probably manual rides or some such.

chris_s
02-05-2016, 12:35 PM
This is simply a wonderful use of Strava's data openness. Any chance you mind my sharing it to a few folks beyond BAFS?

Also, any chance you would be willing to share your methodology?

1) Share it far and wide. To be clear though, I can only get this data because people have authorized the Freezing Saddles app and hozn has made it available to me.
2) hozn is making everyone's rides available to me as GeoJSON files. My code reads through the appropriate GeoJSON file, pulls the latitude and longitude for each point in each ride and shoves it into a delightful little open source library called leaflet-heat which generates the heat map for me and display it over a MapBox map that is display with mapbox-js (which is based on the open source leaflet library).

The longer-term plan is for hozn to also give me the timestamp associated with each point (in addition to lat/lng) which will hopefully let me do an animated version where you can see how the heatmap grows over the course of the time period you're looking at.

chris_s
02-05-2016, 12:40 PM
The "last 24 hours" view seems like it would be really useful after a big storm to see where folks abandon the MUPs for local roads when the ice or snow is too difficult to ride through.

hozn pointed out that I could potentially intuit road & trail conditions from the ride data based on a) whether somebody traveled on it and b) what speed they did so. Something like:

No rides in last 24 hours = impassable [though you'd have to account for the fact that there are streets that people rarely ride on even in good conditions]
All rides in last 24 hours under 5 mph = difficult conditions (people are walking their bikes or biking very slow)
All rides in last 24 hours under 10 mph = use caution
More than 50% of rides in last 24 hours over 10 mph = clear

It'd probably have to be tweaked to be more sophisticated than that, but I bet you could get a pretty decent idea of road & trail conditions off of strava ride data automatically, without any manual reporting given a sufficient # of people authorizing the app.

lordofthemark
02-05-2016, 12:55 PM
hozn pointed out that I could potentially intuit road & trail conditions from the ride data based on a) whether somebody traveled on it and b) what speed they did so. Something like:

No rides in last 24 hours = impassable [though you'd have to account for the fact that there are streets that people rarely ride on even in good conditions]
All rides in last 24 hours under 5 mph = difficult conditions (people are walking their bikes or biking very slow)
All rides in last 24 hours under 10 mph = use caution
More than 50% of rides in last 24 hours over 10 mph = clear

It'd probably have to be tweaked to be more sophisticated than that, but I bet you could get a pretty decent idea of road & trail conditions off of strava ride data automatically, without any manual reporting given a sufficient # of people authorizing the app.


City of Alexandria dispatcher - See this road (uphill near lord of the mark's home)? The cyclists on it have consistently been riding under 10MPH. Go clear it.

City of Alexandria employee, on arrival - Whiskey Tango Foxtrot - it is completely clear!

chris_s
02-05-2016, 01:04 PM
City of Alexandria dispatcher - See this road (uphill near lord of the mark's home)? The cyclists on it have consistently been riding under 10MPH. Go clear it.

City of Alexandria employee, on arrival - Whiskey Tango Foxtrot - it is completely clear!

That sort of thing would definitely be a problem. It would also have trouble with congestion, stop lights, etc. It'd be quite good at telling where the trails and such are definitely clear, but it'd have some real issues telling the difference between "not clear" and "people aren't biking there or are biking there slowly for some unrelated reason".

You could up the accuracy quite a lot by only comparing to previous ride through that same area by that same person, rather than looking at specific speeds. So things would be based off "this guy went through here 90% slower than he usually does" rather than "this guy went through here very slowly" but then that gets thrown off by people changing their routes due to conditions and requires a lot more people to be submitting rides in order for you to have the critical mass of data necessary.

Mikey
02-05-2016, 01:17 PM
What you could probably do is determine the baseline average speed for that stretch of path, road, etc, then track the last 24 hrs departure from baseline. And then color code kind of like the traffic widget on google maps. i.e. Blue - faster than normal. Green 0-5% slower Yellow 5-10% slower, or something like that.

lordofthemark
02-05-2016, 01:25 PM
That sort of thing would definitely be a problem. It would also have trouble with congestion, stop lights, etc. It'd be quite good at telling where the trails and such are definitely clear, but it'd have some real issues telling the difference between "not clear" and "people aren't biking there or are biking there slowly for some unrelated reason".

You could up the accuracy quite a lot by only comparing to previous ride through that same area by that same person, rather than looking at specific speeds. So things would be based off "this guy went through here 90% slower than he usually does" rather than "this guy went through here very slowly" but then that gets thrown off by people changing their routes due to conditions and requires a lot more people to be submitting rides in order for you to have the critical mass of data necessary.

You would definitely need multiple riders. Otherwise it becomes "lordofthemark normally rides that segment at 11MPH, but this week he rode at 9MPH, something is wrong" - "Sure something is wrong, his air pump broke, and he has been riding with underinflated tires" For other people it could be the carbon fiber road bike is in the shop, and they are now riding on the MTB, or whatever.

S. Arlington Observer
02-05-2016, 02:06 PM
Also, note that the time stamp is not up to date in real time. (Not sure how it updates.) As of 2 p.m. it does not show my morning ride this a.m. but does show yesterday afternoon's evening commute. It is great and I love it. But it isn't actually in real time.

chris_s
02-05-2016, 02:10 PM
Also, note that the time stamp is not up to date in real time. (Not sure how it updates.) As of 2 p.m. it does not show my morning ride this a.m. but does show yesterday afternoon's evening commute. It is great and I love it. But it isn't actually in real time.



All-time data updates once a day in the middle of the night.
Weekly data updates once a day in the middle of the night.
Last 24 hours data updates hourly on the hour.

There may also be additional lag between when you upload to strava and when hozn processes the data for me to access.

chris_s
02-05-2016, 03:03 PM
There may also be additional lag between when you upload to strava and when hozn processes the data for me to access.

I went ahead and double-checked, and my auto-updater has been running as expected - it is updating the 24 hour data on-the-hour as expected. Was it the 24 hour data that was stale or are you looking at one of the longer period that update less frequently?

americancyclo
02-05-2016, 03:13 PM
I went ahead and double-checked, and my auto-updater has been running as expected - it is updating the 24 hour data on-the-hour as expected. Was it the 24 hour data that was stale or are you looking at one of the longer period that update less frequently?

Interesting. Looking at my team's 24 hour map at 3:11pm, It shows my ride yesterday at 5:12PM (phone app) but not the 4:06PM or this morning's ride at 7:03AM. (garmin recordings)

chris_s
02-05-2016, 03:32 PM
Interesting. Looking at my team's 24 hour map at 3:11pm, It shows my ride yesterday at 5:12PM (phone app) but not the 4:06PM or this morning's ride at 7:03AM. (garmin recordings)
I bet we've got a timezone issue. Probably I'm speaking local and it's expecting UTC or something. I'll check with hozn.

Vicegrip
02-05-2016, 04:02 PM
hozn pointed out that I could potentially intuit road & trail conditions from the ride data based on a) whether somebody traveled on it and b) what speed they did so. Something like:

No rides in last 24 hours = impassable [though you'd have to account for the fact that there are streets that people rarely ride on even in good conditions]
All rides in last 24 hours under 5 mph = difficult conditions (people are walking their bikes or biking very slow)
All rides in last 24 hours under 10 mph = use caution
More than 50% of rides in last 24 hours over 10 mph = clear

It'd probably have to be tweaked to be more sophisticated than that, but I bet you could get a pretty decent idea of road & trail conditions off of strava ride data automatically, without any manual reporting given a sufficient # of people authorizing the app.I like it and agree that I think we have that right now with the standard Strava heat map. I (the grumpy old guy) bet the city folks have no idea about it nor would they avail themselves to or act on the info.

Could always hop off the bike where the tarmac ends and the snow remains and walk off the letters "Plow this trail!" with the Garmin in hand. Of course you would have to be fluent in the lost art of cursive. With all this snow I have to say I have been practicing a wee bit.

hozn
02-05-2016, 06:59 PM
I went ahead and double-checked, and my auto-updater has been running as expected - it is updating the 24 hour data on-the-hour as expected. Was it the 24 hour data that was stale or are you looking at one of the longer period that update less frequently?
I will double check, but we should be getting gps tracks every hour or so (more frequently, I think).

Edit: timezone would definitely be an issue here. We are storing localized times, but I think the parser may indeed be translating naive timestamps to UTC. We can change that to the competition timezone (EST/EDT).

jrenaut
02-05-2016, 08:40 PM
Dates and times are a disaster as far as computers are concerned. Everyone knows about the Y2K stuff but the crap that programmers deal with every day is even worse. We really need to switch to a base 10 time system recorded as milliseconds since some significant event a few thousand years ago and be done with it. We can either do it now on our own or have it forced on us when the artificial intelligences take over.

chris_s
02-05-2016, 11:44 PM
So it turns out while working on some other stuff, hozn introduced a bug that was causing all recent rides to come back with swapped latitude and longitude. So your new rides were appearing....but in places like Antarctica and Iran rather than where you expected them. Looks like it is fixed now.

hozn
02-06-2016, 12:04 AM
Re: dates/times, the computers are just modeling the f'd up world we've made for them! :-). But as chris_s notes this wasn't the problem here (apparently I had remembered to make the times naive); apparently few systems can agree on whether it is lat,lon or lon,lat. The db geometries and geojson expect (lon, lat) while Strava returns (lat, lon) for coordinates. Hence, this was getting twisted around in several places.

I am now fetching lower resolution streams (to improve performance) and replacing the ride geo tracks. That will continue to happen over the evening and into tomorrow. So there may be more quirks, though I did do some basic inspection of the new data to check that it is not flipped again :)

At some point (later this weekend), I will add the time component to the API which will enable the animations chris mentioned earlier.

P.S. you may notice there is a "photos" page now on the site where you can see all the competition photos.

S. Arlington Observer
02-06-2016, 01:11 AM
So it turns out while working on some other stuff, hozn introduced a bug that was causing all recent rides to come back with swapped latitude and longitude. So your new rides were appearing....but in places like Antarctica and Iran rather than where you expected them. Looks like it is fixed now.

I've always wanted to go to Antarctica. I guess virtual is better than nothing.

Still in awe of you tech guys.

hozn
02-06-2016, 09:46 AM
The geo tracks are all low-resolution now (max 100 points per track, instead of 1000), so heat map loading times should be much faster.

consularrider
02-06-2016, 11:35 AM
So it turns out while working on some other stuff, hozn introduced a bug that was causing all recent rides to come back with swapped latitude and longitude. So your new rides were appearing....but in places like Antarctica and Iran rather than where you expected them. Looks like it is fixed now.

So that's how I ended up in Ukraine, blame it on hozn! :p

Rod Smith
02-07-2016, 09:55 PM
When I view my team last 7 days, I see clearly the route I rode with Teammate David Last Sunday but not the ride I did with him and Will yesterday which should show up more since there were three of us vs. two. Love the heat map, just wanted to point out a possible glitch.

cvcalhoun
02-07-2016, 11:35 PM
Okay, I'm amused by this. The "last seven days" for my team clearly shows my own routes. Since they start and end exactly where I do, I'm assuming that no one else is actually using those roads.

chris_s
03-08-2016, 03:01 PM
Now with animation! Pick a team (or all-participants), pick a time-span and hit "animate"!

All-Participants for last 24 hours looks the coolest IMHO. :)

http://dcbikemap.com/heat/

KWL
03-08-2016, 07:14 PM
So that's how I ended up in Ukraine, blame it on hozn! :pI thought it was love. :)

FFX_Hinterlands
01-12-2018, 02:56 PM
Actually, different paths show for each team, but every team (I checked) had somebody cycling there. Must be an issue with Strava. Perhaps a default location when all else fails?

Shhhh... that's where the All-Powerful Bike Lobby has its secret meetings. We need someplace secure to discuss world domination.