James - is it possible to query for a list of cities that have this problem in order to generate a list of OSM relation IDs that need fixing?
Wow there are a lot!
I’ve exported the list to this Sheet in Google Drive.
This list is weird. Poland has more than >14k entries (including over 1k duplicates), and that’s just cities with 0 streets? Wikipedia indicates that there are less than 3000 municipalities in the whole country
Here’s an example of a “city” that is on the list, that is probably stretching the definition a bit: https://citystrides.com/cities/122875.
I suspect there is an error in the process that generates the city list. It seems unrealistic to fix that many OSM relations (although setting them as maproulette tasks might help), so here’s hoping there is an easier fix…
These countries make up 90% of the list:
The process is that a volunteer imports data in some kind of bulk process. Usually from some kind of government GIS system. This is probably worth sharing with the osm import mailing list. If the error is systematic it could be possible to script a fix.
Check out the Full Country/Region List sheet (I’m slowly moving the Countries sheet data into this one, but not really putting a lot of effort into it because the real source of truth is the CityStrides database itself) - it looks like I took Regions from admin level 4 and Cities from admin level 8.
Stare Polesie (the OSM link for the CityStrides link you shared) is at admin level 8.
So this does sound a little more like what @zelonewolf is suggesting - some bulk import Did Something™.
I looked into that Stare Polesie example. The reason it came up empty for you is because there are no roads inside that boundary. No roads with a name, in any case. So I bet Poland just has a ton of these small villages alongside unnamed rural roads.
Is there some way to separate these cases from ones where there’s an actual problem with the boundary polygon?
Nope. I only know “has streets” and “does not have streets” in my database, for this particular issue. I don’t know anything about what’s going on back in OSM based on my local data.
To complement what @zelonewolf said above, I did some spot checks and found the following categories:
- Regions with tiny hamlets that are crossed/accessed by a road that is either unnamed or only letter+number type (these are tagged as ref in OSM).
- Same as above, but uninhabited
- Villages/cities with named roads irl, but where names are not included in OSM yet
- Cities with named roads in OSM, where there is an import error
My take is that type 1. and 2. can be removed entirely since they are not cities. I guess there is an argument for keeping them if trails are included in the future. Type 3. should eventually get street names in OSM, so should be checked periodically (a lot of the French and Russian ones are in this category). Type 4. and 5. are the ones that could be fixed now.
I found this website (example is from the list) which checks relations for completeness. The web interface unfortunately only takes one relation at a time, but if a way to pass the whole list can be found this is probably the best way to find boundary errors.
I have some code for boundary validity checking. I could probably bounce it against that list to filter for ones that are actual problems.
Okay, after abusing the hell out of one of the public overpass servers for about an hour, here’s the answer:
Only 155 of the relations are actually broken polygons. The rest are simply empty of real streets and I recommend you just delete them out.
Here’s a list of the 155 actual broken polygons:
Just to be sure I understand … there are large numbers and the word “delete” so my stomach is churning
So I can delete those 27,760 cities, assuming I’m ok with the idea that they’ll never/effectively-never be mapped to include any missing streets. (specifically phrased because it looks like Lados should have some streets, but it’s definitely small enough to just and move on to more pressing issues)
Yes, deleting all but the 155 would also remove some actual cities. I think a more nuanced approach would be to move the 155 to the missing cities sheet (and maybe create a maproulette task to fix the boundaries) and then checking manually for the most important countries (by number of striders). For example USA has 71 left on the list, which is doable, especially since none of these are urgent… For the top four countries on the list something more automated or an executive decision to discard would be required.
If there is no cost to keeping all the non-places in, you could also just ignore the issue altogether. The whole point of the list was to identify broken polygons, this is already achieved (nice work @zelonewolf!)
Well, I was just thinking from a resource perspective that anything not in use potentially costs you money. Are there examples of cities not in the 155 that should actually exist?
There are a few, like Houston Texas, or Charlotte, North Carolina (both last edited by you, incidentally ) that should probably be kept. I guess something like 175599, Shrub Oak, New York is borderline.
After trying to find some examples from other countries I agree with you that the vast majority of that list is made of places that shouldn’t be included or that no-one will miss, even if they can tenuously be called cities. It probably makes more sense to have a mechanism for requesting a place rather than to have every village in Siberia ready to go just in case someone runs there.
Oh right. I forgot about the ones I’ve since fixed
My city, Fort Worth, TX, is in your list, it has 0 streets and 0 striders. Any idea of how to fix that?
Also is there a way to create smaller “city” sections, by ZIP code area for example, since the city is huge? It could be split into something similar to this:
I have been away for a while so I do not know when this happened. In my quest to complete towns in NH, I came across Kilkenny, NH which did not have any streets. I completed all the trails in this town and was able to see Kilkenny, NH in my list of completed cities. However, it has now disappeared. I am wondering why cities with 0 streets were removed. I am not sure what issues it creates since a strider would not see it in their list of “cities in progress” or “cities completed” unless they had at least one run within the city boundaries. Even though it did not have any streets, it is a key piece of information needed to keep track of my progress on all cities in NH. Is there any reason we can’t bring these towns back? Many Thanks