Getting Boundary-Line data into OSM

Support City/Street/Node Issues
1

Sorry if this is discussed anywhere else and I missed it…

I’ve been trying to find some ward (effectively town) level boundaries in OSM to add to the import sheet. Some are present in OSM but some aren’t, and those that are are sourced from OS Boundary-Line data (https://www.ordnancesurvey.co.uk/business-government/products/boundaryline).

I’ve looked into how to get the data across for others, and it looks reasonably complicated - the info I’ve found is all pretty technical problems that sound like you need a reasonable knowledge of mapping data formate. So I thought I’d check if anybody has a simple way, or maybe has previously done this and has any scripts or anything that’d save me learning the detail! :wink:

2

@tkajstura @8f7162110d9eeaf907ab :smiley:

I’m not totally convinced that ward level areas should be in CityStrides, as I understand them, but the three of us have been discussing this recently.

3

The UK boundaries are already well mapped and imported into OSM, to the degree that everyone can agree on. Wards as political voting areas, like census areas in the states, are contentious as they are statistical areas more than political boundaries. For a good description of the UK boundaries, see: User:Csmale/ukboundaries - OpenStreetMap Wiki.

I don’t think wards should be pulled into CityStrides en masse as it will duplicate certain areas, but could be useful in an area where a civil parish and a ward are effectively one in the same (but then just pull in the civil parish).

4

Ah, OK, I over-simplified the issue a little, wasn’t sure if people were familiar with the (stupidly inconsistent) structure of English administrative areas!

You’re quite right that most areas (by geographical size at least!) have two-tier county councils, and in these areas civil parishes are the best administrative area for a “town”. But in the 6 Metropolitan Counties there are no Civil Parishes, and the only administrative area for a town sized area is the ward.

For example (and the reason I’m looking into this) - my friend lives in the town of Marple, which is in the metropolitan borough of Stockport - Marple, Greater Manchester - Wikipedia. She’s been running all over the town for some time hoping to try and complete it, and now has it largely covered on her lifemap (she actually got me into CityStrides in the first place!), but the lowest level area for her is Stockport, which has around 4,000 streets over 50 sq miles, meaning it’s not really a very achievable target.
Meanwhile in the metropolitan borough of Manchester, which is contiguous to Stockport, and is where I work, and so run quite a lot, the wards are present. So rather than the unachievable task of running all 6,300 streets in Manchester, I get to try and work on completing Ancoats and Picadilly (my work is roughly on the border).

You are right that wards are political voting areas, and as such don’t always perfectly match towns - Marple, for example, is split into Marple North and Marple South. I agree that this isn’t ideal, but they are generally based around what local people see as their “town”, and they are the only defined & fixed boundary for those towns which I’m aware of.
There may be some in other areas I’m not familiar with which are not “town-based”, but I’m only suggesting bringing them in manually as per the current system, not a mass import. So the ones which are requested will be, by definition, those where somebody wants to try and complete their home town, by a definition which makes sense to them.

As I say, the addition of the boundaries to CS is pretty simple (for us at least) thanks to Jim’s spreadsheet. It’s the addition to OSM which looks more taxing. This is something I think needs doing in OSM anyway for consistency (see Manchester vs Stockport above), though I appreciate that’s a discussion for general map enthusiasts, not CS!

5

Btw, on the point of duplication of areas. Obviously I’ve not seen the conversation on this, but are you aiming to ensure that any given point is only in one area, or do you just mean avoiding over-lap of “equivalent” areas?

If the former, this seems very limiting - while some very enthusiastic members my have a hope of completing a 10,000 street city, it surely adds more to the experience, for more people, if they can get started by trying to complete their local area of a few hundred streets?
This appears to be the case in most countries - I was just looking at somebody who had completed their own suburb of Boston MA, but had a long way to go to get the whole City.

I’d agree that “overlap” would not be desirable, but as long as it’s maintained sensibly, as a hierarchy without too many levels (2 most likely, maybe 3 in some cases), this shouldn’t be a problem. While the UK’s local government system comes in many local flavours, any given point will fall into one specific hierarchy or another.

6

Hi Aiden, this doesn’t directly answer your questions, but I wanted to share a few thoughts as background on how the current CS cities were imported.

The big difficulty with bringing cities into CS is finding something that a) covers all the map, b) doesn’t have overlap with existing areas (this currently breaks street counts and challenges, see the discussion here), c) the areas being available as boundaries in OSM and the hierarchy being at least somewhat consistent so that it can be done in bulk.

For most European countries, including UK, a) means that rural areas will often be very large since there isn’t a consistent layer below (that also has a boundary) in OSM. The issue of overlaps should eventually get sorted out (I hope!), possibly in a way that would also allow for more manageable challenges than 10k streets. c) is probably the least problematic, since the process of importing entire countries is (mostly) completed. OSM data is messy and inconsistent, every country has its own historical quirks and oddities, finding a way to be consistent on a global level is hard.

7

Thanks Joms, makes sense.

Obviously unless there is no hierarchy, and every street exists in one and only one area, then the duplication issue is something that will need sorting separately anyway.

I’d have thought that at this point the best solution is to allow local users to, within reason, pick what they feel is an appropriate representation of where they live / are able to run. The very fact that we have to reply on OSM data means this shouldn’t be abused too much - ie people can’t just make up some nonsense and add it (or not without doing so very deliberately and with a fair amount of effort, by which point any system that is agreed could be “cheated” if people really want).

I don’t pretend to have any insight into the goals of the site as a whole, but assuming it aims to engage as many people as possible (APIs allowing :wink:), I do think different levels of hierarchy are required. As we all know, in the world of running, there is already somebody more extreme than you, and somebody less. Some people will enjoy, and expend a lot of effort, trying to complete their own local suburb, while other nutters will be out there trying to do the whole of London or NYC. It seems sensible to cater for all.

8

I entirely agree with you, everyone should be able to find a challenge to match their ambition. It’s really cool to see the many different takes on the simple premise of “Run your city, street by street”, I don’t see the point in any type of gatekeeping.

From a practical perspective, I feel it would be easier to have a good structure in place before too many overlaps get added, as it means less untangling later…

3 Likes
9

Specifically for the UK, I’ve realised that a lot of this comes down to unparished areas:
Unparished area - Wikipedia.

This applies to metropolitan counties such as those I’ve mentioned, but within those, wards do make a suitable substitution for parishes.

A less straightforward problem are rural towns, such as Buxton, near me. It’s a historic, clearly physically defined town of 22,000 inhabitants, which is an unparished area. As far as I’m aware, this means Buxton has no political / administrative definition at all. Unless somebody is planning to run across all 200 sq miles of the district of High Peak (and believe me, I’ve had a good a go as anybody at doing just that over the years, and I’m on 14% :stuck_out_tongue_winking_eye:), there’s nothing for them to aim for.

But the rest of High Peak does have parishes, so I’m in agreement that the ward idea doesn’t work here, as parishes and wards are not equivalent. This leaves the only “definition” of Buxton as being an area surrounded by a 4 or 5 other parishes :sob:

Anyway, not really got any idea how to solve that one, but for the metropolitan counties using wards seems a pretty sane (/the only) solution.

10

Really great discussion. Unparished areas in the UK seem to mirror the issue of unincorporated towns in the US. Technically they don’t have administrative boundaries as OSM defines them, but in those cases it does seem appropriate to use alternative official boundaries such as statistical boundaries instead. Wards for these does sound like the way to go.

A note on overlap - I am not problem with overlap per se (if my city neighborhoods were in OSM it would be awesome to see that progress alongside the city progress), but the way importing works at the moment any single street would end up imported as a street for a neighborhood and again as a street for the city. If you ran just that one street during a run, you’d have a run that said 2 completed streets, with the same name and nodes. We can also think how to solve that, which would more or less remove any issue of overlap.

11

Buxton is a good example, as there is no boundary in OSM below High Peak district one would have to be created first (and survive subsequent OSM reviews, since it wouldn’t have a foundation in any admin framework).

As for running High Peak, it’s a big area but it doesn’t have that many streets. If you complete the handful of cities you’re in with a shout! Check out this guy for inspiration!

12

There’s inspiration and then there’s insanity :rofl:
Seriously impressive though!

You do touch on a pretty fundamental point… the clue is in the name with City Strides I guess. The whole “street bagging” concept doesn’t necessarily translate to more rural areas, and indeed, one of the joys of living somewhere like High Peak is that most of the time I can leave the houses behind and go and run trails across the hills. In fact the original reason I got into CS was more for the Life Map than the street counting / town completion.
As you can see here, I’ve covered a good part of High Peak from a geographical / distance sense, but only done 14% of streets - High Peak, England - CityStrides.
Wikipedia lists 33 notable settlements in High Peak, and it’s unlikely I’ll be driving half an hour or so to each to meticulously chalk of their streets… not when there are all these beautiful hills to run up! :grinning:

I guess the key take-out is that the Buxton problem probably isn’t solvable, but also isn’t that important beyond a completist mindset.
I do feel that the issue of Marple (which is a suburb and much better suited to CS) is more relevant though, and wards offer a good way to resolve it.

13

@8f7162110d9eeaf907ab @aidangrant
So this is what makes this so hard for me: taking that Buxton example, it is not in city strides and does not appear at first glance to have a boundary if you just search Buxton. But if you dig, it actually does have boundary (type:boundary, boundary:place).

CS (if I’m remembering what Jim said) basically exclusively pulled in administrative boundaries of a certain admin_level tag because that is the most consistent way to get cities/towns.
If someone on the Google Sheet of missing towns said “Hey, I want Buxton, please!” I would dig up this relation, put in the right info for Jim to import and he’d get it in.
But I have no clear solution on what to do - I can’t change the tags on this relation to boundary:administrative because it’s NOT administrative, and Jim can’t pull in every boundary:place because we’d duplicate AND because place boundaries are often not just cities/towns!

14

That makes a lot of sense Tymoteusz, I can certainly see why you need to restrict the types / levels of boundaries to ensure a level of consistency. I guess the problem is that if you allow wards within metropolitan counties, where they are both consistent and useful, you could also potentially allow people to add wards in other districts where they duplicate & overlap existing parishes (which are more useful, and should certainly be retained). The only solution I could see to that is some degree of human moderation and control to enforce rules such as “no wards in areas with parishes”, but I can see that is far from straightforward… I believe I saw some talk of “local admins” somewhere?

Will that Buxton boundary in OSM have just been manually created by somebody using their own judgement? I can’t see any attribution to another source on there. (I’m guessing it could be the boundaries of the old municipal borough of Buxton, which existed until the 70s).
How did you actually find it? I’d searched for Buxton on OSM and only found a single node at the centre.

15

@aidangrant
Re: Buxton
That user in particular is very active on OSM and though it looks like it was just drawn in (and doesn’t have a source) it likely does. There are many examples of place boundaries like this in the UK. Not familiar enough with the area to know the authoritative source they used to pull them in.
I’ll gladly share my boundary location secrets :slight_smile:. In this case, it was quick - search for Buxton, click the node that shows up in the search, recognize it is part of a relation, which often means it is used as a label for a boundary (as in this case). This doesn’t work when the relation doesn’t have a label or is a way. Other tips: if cities/towns are close, search the surrounding towns and see if any of them have borders. If they do and the borders are relations of multiple ways, see is a segment that would be share is in fact in two relations. If coastal, or near obvious other natural boundary features, look at the coast/river/valley/whatever to see if there is a boundary there. You can also follow trails of changesets to find otherwise difficult to locate nodes or ways. And once you’re beyond making just a quick change or two in OSM, jOSM and other editors are much more powerful and tend to make this type of work easier (once you’re past the learning curve).

16

Cheers Tymoteusz. I’d been trying the method of finding a boundary section and seeing what is on either side… this is how I confirmed that the Stockport Wards are not there. The Boundaries I looked at would have Manchester and Stockport on either side, and also a Manchester Ward (eg Gorton), but nothing else.

Weirdly when I search for Buxton, it now takes returns that relation in results, not just the node. Not sure what I’m doing differently!
Can confirm that this boundary does not correspond to the neighbouring parishes though, so definitely not much use to us.

I did manage to unearth this map of parishes, which illustrates the idea that you could define Buxton as that specific unparished area (about 2/3 of the way up, on the left, in case you’re not familiar with the area!): http://www.derbyramblers.org.uk/images/derby/parishes2001.pdf
But it would be a very specific case, and I realise that means it’s not really viable for CS (unless the idea of localised moderators was rolled out maybe).

17

Doesn’t look like that one in OSM is the old Borough of Buxton, either. The old borough matches the unparished area instead, which makes sense:

Municipal Borough of Buxton - Wikipedia,Derbyshire(1970).svg

Thought it could be postcode sector, but it’s not that either!

I’m going to ask the dude who made it…

18

Wild understatement, there, haha. Live in Baltimore, MD, US and have only been to the UK on layovers! Everything I know if from research trying to figure this out for CityStrides.

But yes, you’ve hit the issue - we’re most interested in a way to do things in an automated fashion.

19

:rofl:

Fair enough, it’s hard enough for me to decipher the UK’s arcane and inconsistent system of local government as a Brit!