Just an idea: a toggle switch somewhere that allows you to disengage/reengage sidewalks in Route Builder.
Reason: When zoomed out a bit and building routes, I find myself accidentally selecting sidewalks instead of streets. This isn’t always terrible, but it can make GPX files a mess later on.
If no one else has this problem, forget I mentioned it. Maybe I have have fat fingers and a crappy mouse. 
I have this problem all the time in my hometown, where sidewalks aren’t drawn if you zoom out far enough
But are drawn when you zoom in
I’ve worked around the issue by zooming in before dropping each point, then zooming back out to continue planning/building the route.
I’ll take some time to play with sidewalk_factor and see if that helps…
Update: Changing sidewalk_factor and walkway_factor to forcefully avoid those breaks routing quite frequently. I’m playing around with using auto instead of pedestrian along with options that will keep the route off highways etc…
Update: 
Using auto requires (obviously) a lot of customization (ignore one-ways, ignore certain restrictions, don’t use highways, etc). I’m pretty sure that if I release a version that switches from pedestrian to auto that I’ll be causing problems for lots of people as I work through figuring out more and more customizations that are required to keep things accurate/safe/sane.
I’m going to leave things as-is while I monitor feedback on this post.
Yes, I would categorize this feature request under strictly “would be nice” rather than “crucial.”
Yeah, this is super annoying, but it’s an OSM issue. I hate that people are mapping sidewalks more and more (and they are doing a half-assed job of it too). But I’ve just taken to zooming in and being extra-picky with my clicks.
Been annoying forever, but I agree with Mitch. Gotta be careful when building and if you notice an issue just undo. A great way around it, at least for straight sections, is to just switch to straight-line mode 
The only problem with straight-line mode is that it makes audio cues for my turn-by-turn navigation have a stroke. Something the “path” feature’s listing in GPX just weirds it out. I guess that’s a ME problem, though.
I’ve been working in OpenStreetMaps a lot on this. The biggest issue is that people who map sidewalks aren’t including “links” or “unmarked crossings”, which should be added to essentially every single intersection. So the routebuilder ends up following a sidewalk some insane distance before it can find a way back to the streets.
In areas where sidewalks are mapped as separate geometries, it is commonly seen that every node at which a sidewalk crosses a road, an unmarked crossing is added.
Here’s an example of someone not quite finishing a sidewalk correctly:
Obviously if you accidentally clicked on that sidewalk instead of the road the routebuilder would add a weird out and back.
So I try to fix these whenever I see routebuilder go nuts.
I believe the solution is:
