PDA

View Full Version : Data pull schedule/algorithm?



peterw_diy
12-12-2013, 09:03 AM
How often does bafs pull new data from Strava? What are the requirements for it to see my activities? Privacy flags, how many days after the ride can I manually enter/upload, etc?

I'm trying to help test the plumbing and it'd be helpful to have an idea what the batch jobs are and when they run.

guga31bb
12-12-2013, 09:33 AM
See here (http://bikearlingtonforum.com/showthread.php?6326-Freezing-Saddles-Scoreboard-backend-quot-soft-launch-quot&p=71734#post71734).

hozn
12-12-2013, 09:58 AM
Yes, for clarity this is the plan (subject to change based on how many requests each of these use vs. the rate limits Strava imposes):

- Every 15 minutes pull all new/unseen rides for every athlete since competition start date. We fetch rides in batches of 200.
-- For all new rides, we pull all Strava segments and store those separately. (This is to enable leaderboards for pointless prizes like "most times around Hains point.)
- Daily (every night) we completely resync rides. This handles the case where an athlete deletes a ride or edits a ride. Currently we also re-fetch all segments for every ride, but that needs to change; it's too request-expensive currently.
- Daily (every night) we sync weather data for new rides (maybe for every ride? I need to double check on that). We use geographic data (city/state or lat/lon) from the rides to pull in the weather for that ride. We extract a bunch of stuff we care about (temperatures, etc.) and store off the original JSON response in case we want more weather data later.

So that's the plan right now. Strava did not used to impose limits (since they had no developer keys), but they do now, so we'll have to be careful about that.

My understanding is that we cannot see private activities -- i.e. they will not count toward the competition.

You can submit rides as late as you want (before the end of competition) and they will be counted. Of course if people decide they want to give away weekly pointless prizes for something, then those that wait to upload might miss out. Similarly, if rides are entered manually only contestants may miss out on other pointless prizes such as elevation or average temperatures during ride, etc.