@8f7162110d9eeaf907ab, I’m with you, I would think pausing for those that havent logged in for more than a month or two would definitely help. If you have free calls to use at the very end of the day then sync them but otherwise don’t waste resources.
My two cent/pence from what I’ve seen here…
-
for new users, import their latest 500 runs. If they subscribe they can do a full import (this helps with sustainability as well!).
-
for non-subscribers, stop syncn’ing after x number of weeks/months (3 months), this will again encourage subscribers, and also mean not wasting API calls for those who tried it for a short while and then lost interest.
-
I notice smashrun.com imports directly from Garmin, looks like when you looked in to this there was a $5k fee, but wondered if there was anyway round this.
1 Like
I notice smashrun.com imports directly from Garmin, looks like when you looked in to this there was a $5k fee, but wondered if there was anyway round this.
Or at what point does it become worth paying it? 1000 subscribers at $5 and you have it paid off. There are 880 subscribers listed on the page.
Honestly I don’t think you will have a shortage of new users either. Seems like your hitting a tipping point with exposure. Strava basically promotes the site for you because people see their friends doing these weird heatmapping runs in their feed and they get interested as well.
Come to think of it, is there even a way to leave without becoming a resource drain? Unlinking an account deletes all activities, so there is a strong incentive no to do that, even when a project is completed.
A “Pause sync” toggle would allow a clean exit, combined with a "sync from " option it would allow seamless service hopping.
@dallas.devries There are two whole categories of “syncing” to keep in mind: per-activity and account. You’re right that a bunch of new signups will each trigger an account sync, which can be anywhere from dozens of activities to thousands (there are several people with tens of thousands).
The easiest limit I could place is a 100 activity/day limit, where the backlog slowly syncs in at 100 activities per day until they’re all in. I so very much do not want to do this. My opinion is that signing up for CityStrides and having the first experience be “welcome, now come back in a week” is 
(which means that I currently think the situation right now is ).
@8f7162110d9eeaf907ab The logic behind accurately figuring out what an “inactive member” is worries me. My auth system keeps a current/last sign in date, but I can’t tell when it updates those values. Then there’s the problem of what happens when an “inactive member” becomes “active” - their experience is then “oh hi welcome back, come back later after your activities have synced in” … similarly to the delayed initial sync 
@chriskeene Yeah, the limited sync for non-supporters is one approach. I kind of dislike having an “incomplete” experience of CityStrides. So far, I’ve tried to keep the core CityStrides experience free & give the option to pay for more features (particularly geared towards those that are way into it).
Yeah, Garmin has since dropped the buy-in requirement. I’ve got access to the documentation, but it came about right around the time of the global update - so I focused on that instead (I’d make the same decision if I had to do it all over again). It’s probably time for that project to be my priority.
@8f7162110d9eeaf907ab No, the only way to leave is to revoke access from within your tracking service, which deletes all that data in CityStrides. I really like that simplicity and how it offloads all of that effort onto an existing process. Adding a feature to allow people to pause/stop syncing for a connected service seems useful, though - that should go into #ideas (if it isn’t already in there) 
1 Like
Economics is the study of scare resources and this sounds like a textbook economics problem. There is no solution that will do everything you want so at some point you’re going to need to select a group of users that will get a experience. Unless you can use an API without a limit like garmin.
1 Like
@JamesChevalier I dont think you have to limit it to 100 per day but I think limiting it while at capacity makes sense. I think taking stock of the remaining calls in the last 2-4 hours of the day and auto-allocating them to make sure you max out your 30k limit while leaving a small amount of head room for individual activities to sync. So you sync 100 straight off then say if at 5pm you still have a bunch of requests left sync some more. I think this logic works for inactive users as well, continue to sync them but only with calls that are leftover at the end of the day with lower priority to new user activities. That way if they do log back in their priority is immediately jumped and they shouldn’t be far behind in the syncing anyways. I’m sure its easy to calculate how many calls you use a day on inactive users that could go towards your new users & active users which I would think are much more valuable to you. Of course that is all just a bandaid because I do believe you’ll continue to get a lot of new users and a good amount of supporters. On a side note you might want to consider displaying ads for the non-supporters. You would be surprised by how much revenue you can bring in with the right network with a few well placed ads (from another developer that took his long time rails hobby to a fulltime gig)
I have no idea what happens in the engine room at this site, but if sync now is digging in to the api calls, then you should just remove it. It’s nice to have but not need to have.
It’s nice to get something extra as a supporter, but my main reason for being a supporter is mostly to help you cover the server costs.
After reading a little here https://developers.strava.com/docs/webhooks, could @dbafounta have a point about wasting api calls for whenever someone changing title/type/privacy.
Could you save some api calls by ignoring titles or perhaps delaying import of new activties for perhaps 30 min.
I have another idea, not sure how Strava would look at it…
Perhaps you can tell new users only new activities will be synchronized when signing up.
Then you set up a new webpage like https://tapiriik.com, but only synchronizing between strava and citystrides.
New members will then have to use that site to get old data to citystrides. The new site will have it’s own api deal, and all your trouble will be in the past 
Those updates don’t use up any API calls because their webhook contains all the info required.
I recently made some changes that prioritize subscribers and further delay full account sync for non-subscribers. None of this really helps though because there’s effectively a 15k activity/day limit … with ~12k connected Strava accounts, for people who are quite likely to run one per day, that doesn’t leave much left for history syncing. 
Can you accidentally delete all the queued activities again so its fast again? 
Did you figure out what % of those 12k users are active? If you haven’t logged on in over a month or two you probably aren’t that serious about using the site. Even if you do log back in after a long period of time I think letting the user know that they were paused but that it would begin processing their activities is a much less bad experience than new or existing users not being able to get their updates. Its also probably a much smaller percentage of users. This could be a way to at least kick the can down the road a little while you explore options.
I did notice in my status page my same activity is listed twice. I’m not sure if thats because I renamed it or not but something worth mentioning in case its using extra calls.
2 Likes
https://veloviewer.com/ is actually a good example of a site that actively has you sync Strava by clicking a button and it fetches all of your activities then. This would have to reduce your calls significantly.
1 Like
You could make it so that supporters have one daily sync and non supporters 2 weekly syncs ?
Heck a auto sync could be for supporters and a button or manual sync could be for non-supporters. This would automatically weed out those users not active recently as well. Could be another good reason to push people to support the site. If you’re worried about user experience long term you can always change it when you have more options.
Did you figure out what % of those 12k users are active?
There are 7,259 Strava accounts connected to a user who hasn’t signed in within the last 3 months. IF that database field is correct. My account’s last sign in value is Fri, 03 Apr 2020 17:37:53 EDT -04:00 so that seems decently accurate. I still haven’t figured out when devise sets that value (if any fellow coders want to try & hunt that down for me 
).
This could be a way to at least kick the can down the road a little while you explore options.
Yeah, that’s all this does. Once (if) those people come back to the site, they’re going to want their activities and I’m (currently) going to have to run a manual sync for their account which will add to the API usage. I’m not actually saving myself by doing this. I’m just deferring the pain.
https://veloviewer.com/ is actually a good example of a site that actively has you sync Strava by clicking a button and it fetches all of your activities then. This would have to reduce your calls significantly.
This would not. This would defer the pain. This is also explicitly stated in Strava’s rate limiting documentation as well as their support team’s responses as the thing not to do. They want me using webhooks. I don’t know what VeloViewer is doing to work around that.
You could make it so that supporters have one daily sync and non supporters 2 weekly syncs ?
The core issue is nuanced. It’s not, generally, the full account sync that is the issue. It’s the individual activities that are coming in, and it’s the new user full account sync - or the combo of two, paired with the low rate limit. If I can handle just over 1 activity per person per day, and then 50-100 new people show up with anywhere from 50-1,000s of activities, I cannot do both things (handle new activities and sync new people’s history).
Heck a auto sync could be for supporters and a button or manual sync could be for non-supporters.
I can’t do this because Strava wants and expects me to use webhooks. They’ve been very clear about this in their documentation and their email responses.
I’ve been making piles of changes over the last week to help work through this.
- I self-limit to 300 requests per fifteen minute period, which allows the site to communicate with Strava over a full 24hr period (their 600 call limit every fifteen minutes only allows you to be active with their API for half the day
) - I set aside 2/3 of the available API calls for subscriber accounts (active supporters; either on a monthly subscription or a one-time contribution with active months left)
- These changes allow me to decrease the “saved” daily calls from 1000 to 100 (I might be able to go even lower; I’m monitoring that)
- I changed the full history sync to only sync 100 activities per day
- I’m about to release a change that skips people who have not signed in within the past 3 months. I’ve deleted their currently-queued syncs as well (this dropped about 10k from the queue). I’m going to work on an email that tells them about this deactivation & directs them to either use the site again or to revoke access (which deletes all that data in CityStrides; do not revoke access unless you want your data deleted).
- I’m disabling Sync Now for Strava-connected accounts.
Still 
as well … More thoughts/ideas are welcome - I’m happy to correct people with insider knowledge as needed. Thanks everyone!
Update: Changed above numbers from 6 months to 3 months.
5 Likes
Fundamentally you are prioritizing potential future gripes from non-active users over current, real complains, from active, subscribing users. I don’t understand how the API works but I think I see a problem here.
Fundamentally you are prioritizing potential future gripes from non-active users over current, real complains, from active, subscribing users.
Do you see my bulleted list of things I’ve done as not being enough? Do you think any of those should be expanded?
3 Likes
Those changes sound like they will help a lot with the API limits! Especially dropping the 7k non-active users should make a massive difference (assuming they contribute a proportional share of activities).
I’m more optimistic on this point. As long as <100% come back you come out ahead in terms of API calls (unless the manual sync requires more calls ). I think the number of folks coming back is likely significantly lower.
One thing that isn’t mentioned in the bullets and that might be worth considering is a way to ensure you always max out the 30k calls per day. Is there some way for you to have a set of non-priority activities in a reserve queue that you can use to fill the quota when all the priority stuff is taken care of? Keeping the “API call debt” as low as possible would also mitigate the issues around having to catch people up if they come back.
As long as <100% come back you come out ahead in terms of API calls
I’m going to hold off on telling any of these people that this is happening. One problem at a time. 
(unless the manual sync requires more calls
Yeah, if all 7k are like “oh yeah forgot about that let’s do this!” then it’s basically 7k new signups all at once (or however I handle that re-onboarding).
ensure you always max out the 30k calls per day.
Oh, it will max out. I have to actively hold calls at the end of the day in order to not block people from sign up/in (those are also API calls that count towards the limit). Right now I’m holding 100 calls. I’ll review that each night to see if I can decrease that number.
2 Likes
Definitely a good idea to inform them in batches (if at all) so they don’t all show up at once 
. I would just have a message that displays when they log in next time. Would you have to reimport the whole account or only from today?
1 Like