It seems that when importing data from OpenStreetMap, citystrides does not filter out properly features that are not runnable. For example, I have a lot of
and so on (forest, administrative boundary, quarriesâŚ)
Would it be possible to filter out all those features ? I guess that filtering on the âhighwayâ key would do the job (tag k=âhighwayâ) Highways - OpenStreetMap Wiki
Some of these should be removed (such as rivers and interstate highways), but I would argue that things like parks and buildings should be left inâŚalthough possibly in a separate category.
I would rely on the community to manually flag the objects that are impossible to run.
Buildings especially throw off things like the percentages in big cities. I am in Seattle, which has thousands of buildings and many of them due to the block sizes are going to be impossible to complete because of their corners being too far away or would require going onto private property. I think buildings are also just outside the spirit of what the site is trying to be.
I like having parks, but parks can suffer from similar issues because you have to get near their borders, which can be impossible or unsafe at times.
+1
Some of the features are errors from the OSM, e.g. http://citystrides.com/streets/4500364 is a point in the middle of tram tracks, far from where the actual Place Dumon is located.
I would keep parks - not necessarily their borders, but if there is a walking path you can run, why not?
However one has to be careful - some highways are normally not runnable, but you can run there during a race, e.g. the bridge between Europe and Asia which is runnable only once every year during the Istanbul Marathon. It is not that easy to distinguish them from really unrunnable highways - I am stuck at 95% in Kraainem, Belgium partly because of that - but maybe it is better to be conservative?
Sorry if I donât understand your point of view. Why do you want to keep buildings or park limits ? Do you run those features ? I run on paths in parks but not on the limit of the park.
Hey Vincent & everyone else in the thread - I just want to quickly poke my head in and let you all know that Iâm aware of the problem & really want to get it fixed. I donât have a ton of time right now to get into all the details and decisions that got CityStrides here, but I can at least start to explainâŚ
Yeah, Iâm in a tough situation on this one. The (somewhat) shortened version of the situation is that OSM doesnât have perfect data, so I opted to take in more than I needed with the intention of trimming out the excess. I added âflag for reviewâ buttons on each street, kind of taking a cue from Foursquare & how their user base helps manage the siteâs data (or at least they used to, by giving certain âsuper usersâ more access to the moderation system).
So Iâve got the start of a moderation system, but I havenât really been able to follow through on that completely - adding different levels of access to certain users, or automatically removing streets that are flagged a certain number of times, or something elseâŚ
If I only import âhighwayâ tags, I lose out on things that are not tagged completely. In some areas, this would make my import useless.
If I import everything (like I am) and sort things out within CityStrides, then I start with a mess and need to clean things up. Thatâs where I am now.
Gotta run for now, but Iâll be back when Iâve got some more time for more explanation. If anyone has any more questions/suggestions, please share them - Iâll reply everywhere I can!
Thanks for talking about this everyone, I really appreciate it!
Hi James. Thank you for the explanation.
I think that if there is a problem in the data, maybe the best solution is to fix it in OSM directly. OSM is a community map. Everybody can contribute and see his/her work appearing in the map within the minute. You can contribute by drawing missing features, editing attributes or even just post a note explaining what is wrong in the area (an OSM contributor will try to fix it). Of course then Citystrides will have to face the challenge of syncing its data with OSM more oftenâŚ
In the area where I run OSM seems to be reliable. In this kind of area, there is a big amount of work for runners to flag non-runnable item, and a big amount of work for you (or a reviewer) to delete them. So Iâm not sure this system will work in big cities.
Could you point me an area where feature are not tagged the way they should be, sothat I can better understand the problem ?
Jamesâitâs awesome that you have started planning a moderation system and it is probably the best way to filter out the data. Even the streets themselves are intricateâŚfor instance there are streets within gated communities that are impossible to runâŚand busy streets without sidewalks that are not safe to run.
The community should probably try to reach some kind of consensus about what data should be retained. Even parks can be complicated. I personally think it is cool to be able to run parksâŚbut maybe a moderator could move the tags onto the paths instead of on the borders of the park (would be cool, but probably tricky to implement). Then there are a few parks (in Seattle) that have named trails, which I think are also important and worth retaining since they are frequented by runners. Iâm ok with buildings being deleted. I bet it would only take a few dedicated users per city to filter things out.
Have you tried the overpass API or overpass Turbo to import data from OSM ? http://overpass-turbo.eu and http://overpass-turbo.eu/
It lets you query OSM through a web service and get the data immediatly as GeoJSON, gpx, kml, etc.
You can specifiy the type of object, a location filter, a time filterâŚ
It is very simple to use and the wizard understands natural language.
For example, If I type âHighway in Walhainâ in the overpass turbo wizard, I retrieve all roads in my village.
Yeah, thatâs how I build my queries when adding more countries. Each countryâs data isnât necessarily at the same level (not all countries adhere to an admin level organization of âRegion = 4â and âCity = 8â), so I test different queries with the Overpass API until I find the best admin level matches.
Sounds like I just havenât figured out how to further filter out non-streets⌠I also donât want to filter too much - like I mentioned that in my previous post about how some things that are âhighwayâ arenât tagged that way.
Iâm at least assuming this is possible, because itâs definitely possible for a city to exist in OSM but not be accessible for my needs. An example of this is the recently requested city IaČi, Romania. You can see it in Google Maps:
But if you search for it via Nominatim, you get two resultsâŚ
I canât use the city version, because itâs a ânodeâ. If you search âOSM:â on that page, youâll see where it notes that.
If you search âOSM:â on the state version, you can see how itâs a ârelationâ. Thatâs the only type of data I can use from OSM, because thatâs the only type that holds the street data - Iâm not aware of a way to search for streets within a node.
So, my logic is that if entire cities arenât completely categorized then itâs probably pretty likely that there are streets that arenât categorized correctly as well.
Thatâs why I collect âtoo muchâ (streets, as well as things that might not be streets) ⌠I figure I might be collecting data that isnât properly categorized as a street - stuff that I wouldnât have received if I had a tighter query that was ignoring things that are definitely not streets.
Thereâs a lot that I donât know about OSM, though. Maybe thereâs a better answer out there that Iâm not aware of yet.
In my local areas lots of stuff is in need of removal. For example, station platforms, airport runways and similar stuff is included in the list of streets.
Two things would help
Ability to flag multiple streets at once.
Ability to hide/mark streets already flagged by me/another user.
More suggestions:
3. Mark on the street that it has been flagged. (I have flagged the same street multiple times because I didnât realize it was already flagged.
4. Use the power of Wiki to allow the community to remove objects.
Has there been any progress on this moderation system front? I have flagged a lot of features near me, quite a few of them several months ago, without any change so far. It would be nice to finally reach 100% in some of the places near me.
I am imagining an ever growing queue of flagged objects that only you can go through and it sounds like a hellish job.
Haha! Yeah, you nailed it. I can go in there once in a while and handle a number of flagged streets but I canât keep up.
I would like to build in some way for a community flagging system, so that any member can flag streets and view the list of flagged streets to approve/deny their flag.
So one person could flag a street and if X number of people approved the flag, it would be deleted & if X number of people denied the flag, it would be un-flagged.
Do you think anyone should be able to have a voice, or do you think there should be levels of membership that allow people to have yay/nay voting rights?
This feels like something that requires different levels. I would never trust the internet in general. It would not take much to create x accounts and start flagging and removing several streets. Of course, deciding on âvaluableâ members can be tricky in and off itself.
First I think we need to decide on a fixed set of rules for flagging rather than relying on each personâs individual opinion. I know personally that I think certain things like parks and interesting landmarks should be kept in, assuming it is possible to ârunâ them. The nodes might have to be manually moved. Others in the community might disagree though.
Once the rules are decided, there should probably be designated âmoderatorsâ that have the ability to modify/remove streets. Ideally these would be experienced users who understand the system and are willing to follow the rules that the community agrees on. The moderator should also only have permission for his/her own cities. In other online forums, a moderator is often selected after demonstrating some sort of level of participation/dedication to the site.
I also agree that it would be helpful to see if a street has been flagged and if a comment has been left on the flag. Then everyone will have more information and can provide additional comments that agree or disagree with the original flagging. After that dialogue occurs, the moderator can make the final decision that is consistent with the rules that we have agreed upon.
A more advanced feature would be to allow the community to see the decisions that the moderator has made and offer critiques. That evidence could be used if there are complaints or if things need to be reverted back to a previous version. Additionally, community members could vote yes/no on the decisions that a moderator makesâŚmaybe a 1-week voting period after the moderator submits a change. If there are no votes, the decision would be automatically approved.
Maybe thatâs way too complicated but it would certainly be fun if it was implemented.
I think what might be most important for you James is something simple and also not a burden time wise at least to start. Some ideas along those lines:
Limit moderation capabilities to contributors.
In the UI, keep showing features that have been flagged, but separate them and donât count them towards a percent complete of a city or leaderboard. This way its only a calculation change and less code changed to build something like a moderation system.
Another thought is to build in some obvious pattern matching to exclude things like âBest Westernâ, âapartmentâ or things with say âparking lotâ in the name and flag those automatically. I think large swathes could be automatically flagged in a systemic manner.
I have been doing a lot of flagging in the cities I have run. There are many commonalities that exist in bad data: âOne wordâ, Starts or ends with the word lake, Ends with Park, contains the word building, âparking lot.â Most of these signals are not 100% but if these patterns are detected, that could fast track them for removal. Company names are common in some cities but not in others â but these are also harder to filter for.
Maybe the easiest system to have a review queue and when 1 (or 2 people) approve removal, itâs gone. This should be restricted to active users who have been on the site for a while.
Oh! Right!
Thanks @brettajohnson & @wetmig ⌠Iâm taking your advice & bulk-removing Street records that match certain values, like youâve noted. I think Iâve already removed 20k+ bad streets. This will help a lot!
