ID editor: It is time for us to end this abusive relationship(openstreetmap.org)
openstreetmap.org
ID editor: It is time for us to end this abusive relationship
https://www.openstreetmap.org/user/woodpeck/diary/391175
23 comments
Disclaimer: I've edited OSM a bit, but I don't know much about it. I had no idea before this post that iD was unpopular among any group.
The OSM database consists, primarily, of nodes (points) and ways (ordered sets of nodes). On its own, these mean nothing—a way could be a road, a coastline, or the outline of a building. Nodes and ways are described using key=value tags, which give them meaning. Users can input arbitrary tags, but for them to be useful there obviously have to be some sorts of conventions about what tags get used for what. These conventions are captured on the OpenStreetMap wiki.
The iD editor is one of several tools for editing OSM, and is the one recommended to new users. Because manually entering key=value pairs isn't particularly user-friendly, iD has some built-in knowledge about what tags mean what, allowing a user to mark a way as, say, a "Parking Aisle" and have iD automatically tag it as highway=service service=parking_aisle. It seems that one of the main complains is that the authors of iD are adopting conventions inconsistent with those used by the community, leading to inconsistent data.
There also seem to be some other complaints. iD includes some links to the OSM community, and apparently featured "proprietary" (and, presumably, more usable) platforms such as Slack and Reddit over "open" platforms (IRC, mailing lists). iD also uses logos of some brands (if I mark a building as a McDonalds, the McDonalds logo shows up) and apparently loads these from Twitter and Facebook, leading to privacy concerns.
The OSM database consists, primarily, of nodes (points) and ways (ordered sets of nodes). On its own, these mean nothing—a way could be a road, a coastline, or the outline of a building. Nodes and ways are described using key=value tags, which give them meaning. Users can input arbitrary tags, but for them to be useful there obviously have to be some sorts of conventions about what tags get used for what. These conventions are captured on the OpenStreetMap wiki.
The iD editor is one of several tools for editing OSM, and is the one recommended to new users. Because manually entering key=value pairs isn't particularly user-friendly, iD has some built-in knowledge about what tags mean what, allowing a user to mark a way as, say, a "Parking Aisle" and have iD automatically tag it as highway=service service=parking_aisle. It seems that one of the main complains is that the authors of iD are adopting conventions inconsistent with those used by the community, leading to inconsistent data.
There also seem to be some other complaints. iD includes some links to the OSM community, and apparently featured "proprietary" (and, presumably, more usable) platforms such as Slack and Reddit over "open" platforms (IRC, mailing lists). iD also uses logos of some brands (if I mark a building as a McDonalds, the McDonalds logo shows up) and apparently loads these from Twitter and Facebook, leading to privacy concerns.
OSM is more like a community of communities: There are people mapping with very diverse value systems (ranging from underserved people to semantic data aficionados), groups and corporations building software, volunteers running the servers etc. Making a decision with such a large body of people with very different interests can be very troublesome, but this is something you have to deal with.
A central point of the ecosystem is the editing software: This is the gateway through which people will touch the data and can massively steer the data and thus the community into a certain direction. E.g. if you throw a validation error on every object which doesn't have a name, soon all objects will have names.
There are several editors available for OSM, but iD is the most prominent one at the moment, as it will be presented to you if you click the "Edit" button on osm.org. Thus people developing iD have a large burden to carry. And there is the growing disconnect that the author is writing about: A small group (in other words: two people) control in which direction the project is being steered. And with a complaint from within the community, one of the maintainers has disabled the issue tracker for the general public. (Edit: Issue tracker has been reopened now)
Making a decision with everyone involved is hard and takes a lot of effort, but at the moment we have a situation in which very few decide without the mandate of the community.
Hope I could shed some light (though it's late over here).
A central point of the ecosystem is the editing software: This is the gateway through which people will touch the data and can massively steer the data and thus the community into a certain direction. E.g. if you throw a validation error on every object which doesn't have a name, soon all objects will have names.
There are several editors available for OSM, but iD is the most prominent one at the moment, as it will be presented to you if you click the "Edit" button on osm.org. Thus people developing iD have a large burden to carry. And there is the growing disconnect that the author is writing about: A small group (in other words: two people) control in which direction the project is being steered. And with a complaint from within the community, one of the maintainers has disabled the issue tracker for the general public. (Edit: Issue tracker has been reopened now)
Making a decision with everyone involved is hard and takes a lot of effort, but at the moment we have a situation in which very few decide without the mandate of the community.
Hope I could shed some light (though it's late over here).
> Can anyone with comprehension of the project explain what this is about?
The OSM project has a default editor called iD. iD is developed by developers from Mapbox. As it is the default editor, iD via what it puts forward has a lot of leverage on the projects.
Recently, some people have complained about some of iD features. Notably iD pulls images from Facebook and some people feel it is an unnecessary loss of privacy.
Unfortunately, one of the iD maintainer seems slightly burnt out and as a result has questionable communication skills. This lead to poor handling of the Github issue and this blog post.
The OSM project has a default editor called iD. iD is developed by developers from Mapbox. As it is the default editor, iD via what it puts forward has a lot of leverage on the projects.
Recently, some people have complained about some of iD features. Notably iD pulls images from Facebook and some people feel it is an unnecessary loss of privacy.
Unfortunately, one of the iD maintainer seems slightly burnt out and as a result has questionable communication skills. This lead to poor handling of the Github issue and this blog post.
I mean, this blog post is poor communication, so...
Minor correction: one of the developers is working for Mapbox
OK, this is going to be longish.
While OSM is similar to Wikipedia in many aspects, it is very different in others. In particular while both movements have a supporting formal body, in the case of Wikipedia the WMF, for OSM the OSMF, that run (fsvo run) the central infrastructure and formally publish the data from a legal pov, in practice they are very very different.
This starts with the size of the budgets (the OSMF is roughly 1/1000 of the WMF one), to the fact that the OSMF is a membership organisation that determines its board in a democratic fashion (coming up soon), and so on.
But the relevant difference for this discussion is that the OSMF has never been involved with developing software (with one small exception) and that all the bits and pieces that make up what you experience as "OSM" have been written and maintained by the community. This has always included commercial entities and major parts of the editing API and the website have been written by developers on the payrolls of companies with an interest in OSM.
Editors (the programs that allow you to add to and change the data in the OSM database) have kind of always been an exception. The 2 in-browser editors Potlatch 1 & 2 that led to the breakthrough of early OSM were for the major part written by Richard Fairhurst, JOSM the most popular desktop editor is developed by a group of volunteers and similar things are true for most of the less known apps.
Back in 2012 it was clear that the Flash based Potlatch 2 had only a very limited remaining life in front of it and Richard started work on an PoC for a JavaScript based replacement that he named iD.
This gave Mapbox the idea that they could repeat what they had already successfully done once with Tilemill, to apply to the Knight Foundation for a grant to improve the OSM website and create a new JS based editor. Naturally this was not just out of the goodness of their hearts, just as with Tilemill this was also an opportunity to scale up the company. In any case Mapbox was awarded half a million $ for this work which led to the release of iD in May 2013.
Note that the current developers and maintainers of iD were not involved at this time, and I'm not even sure of any of the original devs are still with Mapbox.
In due course iD replaced P2 as the default editor on openstreetmap.org, that is the editor you get when you simply click on the "Edit" tab. This is a rather privileged position as for nearly all new contributors this is the point of first contact for OSM and can be formative for ongoing editing behaviour (while from a number of users iD is by far the most popular tool, JOSM outstrips it in the actual volume of edits).
Obviously the $500k didn't last forever and while there was one larger refactoring of the code that was worked on by a team in Mapbox, the involvement of Mapbox was obviously scaled down after the money ran out.
In 2015 (iirc) Mapbox hired the current lead developer and last year he was joined by a further developer employed by a consultancy. There have never been any formal communications from either of the employers as to scope of their current and future engagement, not even as simple as to where the money to finance this rather expensive undertaking is coming from.
Which brings us to the tensions that led to Frederiks blog post.
We've always had a bit of a dictatorship of the developers when it comes to editors, and all of us (including myself) tend to have "good ideas" (that now and then turn out not to be so good and have to be undone :-)) that are not pre-vetted with the community at large.
What is different with iD is its privileged position, and that because OSM has grown so much the much larger, the impact that changes to the default editors behaviour have, the IMHO in-transparent decision processes (who is -actually- calling the shots?) and the easy to offend devs. Combined this is rather an explosive mix.
The latest episode is the decision to start displaying brand logos in the editor and retrieving these online and mainly from Facebook. There are a lot of reasons why this is problematic, not the least that it violates EU privacy regulation without prior consent of the users (this is true regardless of the source of the logos, but obviously using Facebook makes the whole thing even more controversial). Not unexpected this led to complaints which in turn led to the devs locking down issue creation on the iD repo (this has AFAIK now been undone).
Obviously this has to be defused in one way or the other and running a patched version of iD on openstreetmap.org that is under more community control is one possible solution (though not my favourite).
Simon
While OSM is similar to Wikipedia in many aspects, it is very different in others. In particular while both movements have a supporting formal body, in the case of Wikipedia the WMF, for OSM the OSMF, that run (fsvo run) the central infrastructure and formally publish the data from a legal pov, in practice they are very very different.
This starts with the size of the budgets (the OSMF is roughly 1/1000 of the WMF one), to the fact that the OSMF is a membership organisation that determines its board in a democratic fashion (coming up soon), and so on.
But the relevant difference for this discussion is that the OSMF has never been involved with developing software (with one small exception) and that all the bits and pieces that make up what you experience as "OSM" have been written and maintained by the community. This has always included commercial entities and major parts of the editing API and the website have been written by developers on the payrolls of companies with an interest in OSM.
Editors (the programs that allow you to add to and change the data in the OSM database) have kind of always been an exception. The 2 in-browser editors Potlatch 1 & 2 that led to the breakthrough of early OSM were for the major part written by Richard Fairhurst, JOSM the most popular desktop editor is developed by a group of volunteers and similar things are true for most of the less known apps.
Back in 2012 it was clear that the Flash based Potlatch 2 had only a very limited remaining life in front of it and Richard started work on an PoC for a JavaScript based replacement that he named iD.
This gave Mapbox the idea that they could repeat what they had already successfully done once with Tilemill, to apply to the Knight Foundation for a grant to improve the OSM website and create a new JS based editor. Naturally this was not just out of the goodness of their hearts, just as with Tilemill this was also an opportunity to scale up the company. In any case Mapbox was awarded half a million $ for this work which led to the release of iD in May 2013.
Note that the current developers and maintainers of iD were not involved at this time, and I'm not even sure of any of the original devs are still with Mapbox.
In due course iD replaced P2 as the default editor on openstreetmap.org, that is the editor you get when you simply click on the "Edit" tab. This is a rather privileged position as for nearly all new contributors this is the point of first contact for OSM and can be formative for ongoing editing behaviour (while from a number of users iD is by far the most popular tool, JOSM outstrips it in the actual volume of edits).
Obviously the $500k didn't last forever and while there was one larger refactoring of the code that was worked on by a team in Mapbox, the involvement of Mapbox was obviously scaled down after the money ran out.
In 2015 (iirc) Mapbox hired the current lead developer and last year he was joined by a further developer employed by a consultancy. There have never been any formal communications from either of the employers as to scope of their current and future engagement, not even as simple as to where the money to finance this rather expensive undertaking is coming from.
Which brings us to the tensions that led to Frederiks blog post.
We've always had a bit of a dictatorship of the developers when it comes to editors, and all of us (including myself) tend to have "good ideas" (that now and then turn out not to be so good and have to be undone :-)) that are not pre-vetted with the community at large.
What is different with iD is its privileged position, and that because OSM has grown so much the much larger, the impact that changes to the default editors behaviour have, the IMHO in-transparent decision processes (who is -actually- calling the shots?) and the easy to offend devs. Combined this is rather an explosive mix.
The latest episode is the decision to start displaying brand logos in the editor and retrieving these online and mainly from Facebook. There are a lot of reasons why this is problematic, not the least that it violates EU privacy regulation without prior consent of the users (this is true regardless of the source of the logos, but obviously using Facebook makes the whole thing even more controversial). Not unexpected this led to complaints which in turn led to the devs locking down issue creation on the iD repo (this has AFAIK now been undone).
Obviously this has to be defused in one way or the other and running a patched version of iD on openstreetmap.org that is under more community control is one possible solution (though not my favourite).
Simon
Fairly accurate! A few corrections:
> Note that the current developers and maintainers of iD were not involved at this time, and I'm not even sure of any of the original devs are still with Mapbox.
At the time I contributed my first (‘Mapbox’s first’) PR to iD, the only previous developer was Richard Fairhurst and the codebase was strictly a proof-of-concept. He continued to be involved for a little while and we had very open lines of communication (and still get along to this day)!
Out of the original developers, about 2/4 are still at Mapbox, but the other two don't contribute to it anymore, for the obvious reason: it's utterly thankless, a guaranteed pit of abuse regardless of how you do it. It's hard to find the sort of therapist-saint-developer-warrior who can work with the OpenStreetMap community without losing their minds.
> What is different with iD is its privileged position
iD's position was earned. I was there, as we worked through months of community process and testing and adjustments to acquire the near-impossible consensus that the OpenStreetMap community desired. Saying that it replaced P2 "in due course" and now has a "privileged position" forgets that the community, not us, made the decision and pressed that 'merge PR' button.
> Note that the current developers and maintainers of iD were not involved at this time, and I'm not even sure of any of the original devs are still with Mapbox.
At the time I contributed my first (‘Mapbox’s first’) PR to iD, the only previous developer was Richard Fairhurst and the codebase was strictly a proof-of-concept. He continued to be involved for a little while and we had very open lines of communication (and still get along to this day)!
Out of the original developers, about 2/4 are still at Mapbox, but the other two don't contribute to it anymore, for the obvious reason: it's utterly thankless, a guaranteed pit of abuse regardless of how you do it. It's hard to find the sort of therapist-saint-developer-warrior who can work with the OpenStreetMap community without losing their minds.
> What is different with iD is its privileged position
iD's position was earned. I was there, as we worked through months of community process and testing and adjustments to acquire the near-impossible consensus that the OpenStreetMap community desired. Saying that it replaced P2 "in due course" and now has a "privileged position" forgets that the community, not us, made the decision and pressed that 'merge PR' button.
> iD's position was earned. I was there, as we worked through months of community process and testing and adjustments to acquire the near-impossible consensus that the OpenStreetMap community desired. Saying that it replaced P2 "in due course" and now has a "privileged position" forgets that the community, not us, made the decision and pressed that 'merge PR' button.
I don't believe I said anything else, particularly given that I was one of the people most strongly lobbying for iD to become the default editor at the time (the rationale was not quite as you put it but that doesn't matter).
And that there is no misunderstanding: nobody is proposing changing that,
I don't believe I said anything else, particularly given that I was one of the people most strongly lobbying for iD to become the default editor at the time (the rationale was not quite as you put it but that doesn't matter).
And that there is no misunderstanding: nobody is proposing changing that,
Excellent recap.
Re: displaying brand logos in the editor and retrieving these online and mainly from Facebook. There are a lot of reasons why this is problematic, not the least that it violates EU privacy regulation without prior consent of the users
I'd like to understand what the issue is here. I'm assuming it isn't just a matter of grabbing a logo (jpg) from Facebook. I assume iD is sending (meta) data about the person who is simply browsing the map to Facebook (i.e. without disclosing that info is being sent or what data is sent)?
Re: displaying brand logos in the editor and retrieving these online and mainly from Facebook. There are a lot of reasons why this is problematic, not the least that it violates EU privacy regulation without prior consent of the users
I'd like to understand what the issue is here. I'm assuming it isn't just a matter of grabbing a logo (jpg) from Facebook. I assume iD is sending (meta) data about the person who is simply browsing the map to Facebook (i.e. without disclosing that info is being sent or what data is sent)?
> I'm assuming it isn't just a matter of grabbing a logo (jpg) from Facebook. I assume iD is sending (meta) data about the person who is simply browsing the map to Facebook (i.e. without disclosing that info is being sent or what data is sent)?
Given the scale of the drama, you'd assume that it was something nefarious, but it isn't. iD loads images for brands from Facebook, and nothing more: here's the relevant code https://github.com/osmlab/name-suggestion-index/blob/66412f0...
Fun story from back in the day: we added 'social share' buttons to iD. Knowing the fundamentals of web security, they were implemented as simple links, with no JavaScript component on the iD side, no 'trackers', nothing of the sort. But some buzz came out about how other Facebook trackers were up to no good, and the OSM community got in a tizzy about how we had installed nefarious tracking devices in the codebase, and after a few days someone recommended that we should implement the buttons as simple links, which is precisely what they were in the first place. That combination of bad faith and nobody reading the code was constant.
Given the scale of the drama, you'd assume that it was something nefarious, but it isn't. iD loads images for brands from Facebook, and nothing more: here's the relevant code https://github.com/osmlab/name-suggestion-index/blob/66412f0...
Fun story from back in the day: we added 'social share' buttons to iD. Knowing the fundamentals of web security, they were implemented as simple links, with no JavaScript component on the iD side, no 'trackers', nothing of the sort. But some buzz came out about how other Facebook trackers were up to no good, and the OSM community got in a tizzy about how we had installed nefarious tracking devices in the codebase, and after a few days someone recommended that we should implement the buttons as simple links, which is precisely what they were in the first place. That combination of bad faith and nobody reading the code was constant.
Everything you do on Facebook is tracked, monitor and used. Everyone knows that. If you start requesting images of fast food chains from Facebook, we all know Facebook is going to analyze and utilize your new found interest in fast food.
The feature literally only pulls images from Facebook's graph api.
It is of course a privacy leak, but there's not actually pushback on addressing that, just not a time machine to go back and fix it already. So there will be a privacy policy notifying users of potential third party data accesses and an option to turn that stuff off.
It is of course a privacy leak, but there's not actually pushback on addressing that, just not a time machine to go back and fix it already. So there will be a privacy policy notifying users of potential third party data accesses and an option to turn that stuff off.
iD, instructs the users browser to fetch an image from graph.facebook.com. The problem is that this tells Facebook that you're interested in a certain brand, if you're logged into Facebook, then FB associates the two. We all know that FB grabs and uses and assembles every bit of information that it can, even repeatidly breaking the law.
Adding a Mothercare shop on OSM shouldn't result in Facebook showing you baby/pregnancy adverts, but here we are.
Adding a Mothercare shop on OSM shouldn't result in Facebook showing you baby/pregnancy adverts, but here we are.
I followed the hyperlinks to some of the discussion. In my opinion, the author’s characterisation doesn’t match the tone of the iD developer.
The author might be right about the lack of community involvement, but imho he takes an overly negative an accusatory stance
The author might be right about the lack of community involvement, but imho he takes an overly negative an accusatory stance
This is a classic osm way of handling something though. They make every smallest disagreement into a giant dramatic deal that needs to be fought to the death. It’s a toxic community that drives out people unwilling to deal with the crap, and that’s a large part of why splits like this happen. No one wants to be on the tagging mailing list because it’s probably full of opinionated curmudgeons just like the rest of the osm mailing lists, and now those same curmudgeons are pissed about being left out of tagging decisions. They’re pushing people away and then complaining when their opinions are no longer centered in the conversation.
Disclaimer: I used to be involved in the mapping space (including osm and Mapbox) and have moved on, despite my love of the work.
Disclaimer: I used to be involved in the mapping space (including osm and Mapbox) and have moved on, despite my love of the work.
Sorry, I can’t agree with your characterization. Obviously nitpickers exist, but the general community can be very helpful. Some people have strange ideas and expect everyone to go along, but everything is up for discussion.
Especially for people coming from cultures where confrontation or directness is frowned upon this leads to the impression that OSM is a fighting ground, but there can be no discussion without disagreement.
Especially for people coming from cultures where confrontation or directness is frowned upon this leads to the impression that OSM is a fighting ground, but there can be no discussion without disagreement.
No need to apologize for disagreeing! I also think the OSM community has a ton of really fantastic people, who are excited to work together to make something really neat. Unfortunately, it's not true across the board, and the tone of this article does a great job of illustrating that.
I edited the code of iD on private project and it's a well done tool
It’s possible to have great execution on a bad idea. I’ve seen people become enamored with their own beautiful idea that made no sense for the reality of the situation. They get invested in it and you can’t talk sense to them.
My favorite description of this is “coherent but wrong”.
My favorite description of this is “coherent but wrong”.
[deleted]
tl;dr: This appears to be about the OpenStreetMap project's online editor app, and disagreement over how to proceed with future development.
Apparently there is a more or less forked alternative editor that may be used instead of the default, although there is no hint how.
Apparently there is a more or less forked alternative editor that may be used instead of the default, although there is no hint how.
Sounds like an argument about creative control. If someone was facilitating my project and I felt like the were trying to commandeer it instead, there would be some hard feelings.
Makes me wonder how the Docker people feel. And how the LXC people feel.
Makes me wonder how the Docker people feel. And how the LXC people feel.
It sounds as if OSM is, at its core, just a database with APIs (I do seem to vaguely recall there wasn't even an official web interface) and thus the decentralized gatekeeping is a point of contention.