Are you being slammed?

According to the status page, there don’t appear to be any significant delays.

However, when I look at my run from almost four hours ago, there are no streets affected. I would have thought that I’d have completed a few streets and got partial credit on some others.

Additionally, I was having problems loading your site.

I don’t want to list missing activities if this will work out on it’s own, but it sure looks like something is hitting you. Hard.

4 Likes

I’ve been contemplating making a similar post all day. Glad I’m not the only one with this issue! No big blocks were showing up when I checked sync status, so I was starting to think it might be another “me problem.”

Same here - it shows zero streets completed and that is definitely incorrect.

Yeah, there’s serious load right now in a way that I’ve never seen before. I’m trying to sort things out, but I’m not making much useful progress.
:sweat: I also had a huge amount of yard work to do today from a recently trimmed tree, so part of my approach was waiting and seeing if it would fix itself. :grimacing:

Narrator’s voice: It did not.

Everything will eventually be sorted out - either by waiting for the slow processing, or by me figuring out a way to speed things along.

:nerd_face:
You can stop reading here if you’re like “:man_shrugging: just fix it” :laughing:

My background job system uses Redis and the server that’s running that is at 100% load. I’ve never seen this load on that server before.
Part of my job system schedules work a few seconds in the future because I’ve seen some strange behavior with bulk data imports (saving lots of GPS records attached to activities) and then immediately processing that saved data (I use Sidekiq for background jobs & it’s ‘too fast’). Right now I have 6 million jobs queued up to process these GPS records, and they’re not being added to the queue for some reason - just sitting there displaying “4 hours ago” as their run-time.
I also have some code that tries to prevent duplicate jobs (e.g. someone mashing on the ‘sync now’ button) by checking for itself in the queue. I think this is causing the bulk of the problems, because it’s searching through 6m+ queued jobs. :sweat_smile:

1 Like

My recent activity has uploaded and is showing on City Strides, but the nodes are still red, any idea why?

Yeah, it’s this: Are you being slammed?
I’ll merge this into that thread now

1 Like

This is probably not related to this particular issue, but judging by my last activity showing up in triplicate some idiots mashing the ‘sync now’ button still manage to get past your code :laughing:

Yeah, when load gets really bad (now) my de-duplication efforts fall apart.
Though I’m a little surprised that the activity dupe-checking I have didn’t kick in. :man_shrugging: I probably won’t spend much time looking into that if it doesn’t happen more frequently…

3 Likes