Fully reworked the city update process to address heaps of issues
- Addressed issues include…
- Stats not being updated
- Street counts not updating, resulting in incorrect percentage progress values
- Completely clearing the streets from cities
- Behind-the-scenes infrastructure issues resulting in failed updates
- Many notifications sent for the same update
- Cities getting their timestamp updated but no actions take place
- Changes include…
- Over the past ~month I’ve had the city update process only running between 7pm-6am Eastern, but have since returned the city update process to a 24/7 operation (as of late in the day on April 11)
- The system runs minutely, queueing up 50 cities per batch as long as there aren’t too many existing updates running or too much queue backlog & I’ve reduced the aggressiveness of the process to make it quicker to pause the process (skip enqueuing cities in that minutely run) as that backlog grows
- Adjusted the overall process to hold onto data that needs to be reprocessed (due to created/deleted/moved nodes etc) and do all of that reprocessing in a single step near the end of the update … This reduces overall load, removes duplicate notifications, and incorrect stat displays/reporting during the update process … This may cause some stats to be inaccurate during the update process, but this appears to be on the scale of minutes in most cases
- Fixed an issue where the city update process wasn’t correctly removing nodes outside of the city border e.g. if the border was changed since the last update
- Addressed issue where Overpass can sometimes incorrectly return 0 streets for a city
- Addressed issues include…
- Not user-visible, but the nodes database table was cleaned up a bit to remove some duplicate fields … just one of those fun things that 2013 me didn’t know, and 2024 me does
- Added a limitation to only allow 1 activity deletion per hour in order to address the load that is placed on the system when someone deletes a ton of activities
Added a new entry to the In The News page- Added/awarded the National Walking Day badges
Added a timeout to NodeHunter to address an issue where these could hang, eventually bringing down the whole system- Fixed a bug with the International badge that blocked it from leveling up
- Fixed the badge achievement link in the in-site notification you receive if you happen to be online while receiving the badge
package updates
8 Likes
- Cities getting their timestamp updated but no actions take place
Confirming this is no longer happening on Praha, Praha - CityStrides and it has been updated successfully yesterday. Thanks!
With the update schedule working differently now, what is the new expected update frequency for a city?
@JamesChevalier I’ve never been in the mode of your changes being ridiculous - but I hope the deletion process gets queued. I just went through and deleted about 50 activities. Yes, I’ve been lazy lately with deleting the neighborhood walks.
The problem is, that we cannot differentiate what gets pulled into CS. I do not want my walks in my neighborhood in CS, it makes the map even more horrendous to look at.
1 Like