No longer using Overpass, instead relying on local OSM data that’s updated daily
Changed LifeMap from being built on the fly witih each request to being fully rebuilt after each activity is saved
Significantly decreases load on database
Supporters are near instant (I’ve seen some delays up to 5 minutes), while the delay for non-Supporters usually starts off at ~30 minutes in the beginning of the day and grows to 5-6 hours by the evening
Block LifeMap updates for non-Supporters who have not logged in for 1 month
Rework how data is processed after a city sync
This is the most subtle/understated entry ever - I think I’ve spent the last four months on this
Created a new City Sync Report, available to Supporters
Moved the city sync results out of notifications, into a Sync Report
These reports are accessible from the menu on the city page
The email notifications are still sent, however they only include counts now
Replaced the user list page with the Leaderboard page
Fixed issue with Garmin’s unannounced “Historical Data” requirement during login
Fixed an issue with city syncing that was not removing certain nodes from streets (when an OSM Way was renamed to a street name that exists in the same city)
Changed Hard Mode to allow manually completing streets
That is a bummer, IMO. To me, part of hard mode was knowing manual completions weren’t allowed.
Regarding syncs, it appears the option to initiate a sync (for supporters) has disappeared, or has it moved somewhere else? For example with Austin I don’t see the button or menu option (whichever it was) to initiate a sync anymore.
Oh, no. No no no. They’re not - it’s the only way that this gets to a 100% success rate.
Seriously, though, I won’t know the cadence for a while. A whole lot of cities are going to need to sync before I can get a sense of that.
I have user-requested syncs turned off at the moment. I’m reviewing performance after the work I’ve done (specifically around the LifeMap) to determine if I can devote more resources to the post-sync processing.
My two cents: someone wanting to go into hard-mode should be expected to fix OSM issues (as I’ve done all over Austin, and in other cities). Part of the reward for doing that is the increased confidence that their completions are legit (and also that they’ve probably contributed back to make OSM better for the benefit of everyone).
Thank you James. The only real reason for asking was I visited a new town recently and got 68% done in a week but also edited OSM to remove logging / mountain trails so was curious now what the correct number would be. This is already my most walked ‘city’ as living in a urban sprawl and then visited a small rural Welsh village. #NeedyUser
I assume you were inspired by the same post I saw pass by about some company doing such hashing first. The comments I’ve seen to that article from GIS people seemed to be along the lines of “spatial index already does this for you” though. Makes total sense to give it a try though, that was my immediate thought too when reading it
Oh, yeah, How we made geo joins 400× faster with H3 indexes (open that in a new tab, their page destroys browser history ). I saw it in HN & I reached out to the commenter who shared a bunch of helpful info - haven’t heard back though
My geohash (and later h3) attempts were in October. The performance was way faster, but there are a lot of inaccuracy issues e.g. an activity not passing into the geohash/tile but still being within 25 meters of the street. I think the fix might be to associate activities/nodes with multiple geohashes/tiles (all its neighbors), but I was overwhelmed at the time and decided to put it down … might be able to try again, though…
I understand that non-supporters are low on your list of priorities but still would like to get a sense if you are planning any additional improvements to LifeMap build time. Up until a couple weeks ago LifeMap was building pretty much instantaneously after my run but after your latest changes it takes hours (haven’t tried to time it but certainly not 30 min and in any case significantly longer than in the past). I know there is a separate thread where you discussed some of the issues and I was under impression that you were working to improve it. But based on your release notes it sounds like long delay is the end state. Is it really as good as it gets for non supporters? Just trying to wrap my head around the fact that downgraded performance for us is the best you can do, when it wasn’t the case just a couple weeks ago
The prior LifeMap setup was serving it live for every request. This placed incredibly heavy load on the database, because it was loading data every zoom/pan of the map. This caused a lot of contention with activity processing and post-city-sync processing.
I’ve experienced a number of inflection points as CityStrides has grown since 2013. I was able to do things when there were under 100 people that I couldn’t do with 1,000 … similarly, at about 10,000 and 50,000 there needed to be adjustments (I’ve gone through two versions of automated infrastructure scaling while working through these inflection points) … and I’m seeing that as it nears 100,000 there are new and interesting adjustments that need to be made.
These release notes are more of a current-state-of-things update. Right now the LifeMap takes a while to build, but if I can speed that up without destroying my budget then I definitely will. I think that’s true of anything in CityStrides - if I can improve it, I will (sometimes things take longer than I’d like, though).
It’s not a contest, but what is it then? It’s a nice way to get out and run all the roads. But you don’t need hard mode for that. Hard mode adds the extra challenge of having to complete 100% of the roads. If you take that away what’s left?
I know people cheat, so you shouldn’t worry about them. They’re only cheating themselves.
BTW @JamesChevalier , you might want to update this description:
Not tough enough for you? Check the box below, and the rules change - you have to complete 100% of each street’s nodes to mark it as complete. No manual completions allowed.
True, it is not a contest against others, just yourself. Why change hard mode to have a manual option? Just switch back to normal mode.
regarding @JamesChevalier commenting about hard mode and OSM. If you are willing to go into hard mode you should be willing to modify OSM or work with someone that can.
It’s been a while since I’ve posted one of these
)
). I saw it in HN & I reached out to the commenter who shared a bunch of helpful info - haven’t heard back though 