Ask HN: Why Did the Semantic Web Fail?
We were promised utopia but it never gained any traction.
9 comments
I think it was probably just too much extra work to expect wide adoption of the standards. It could never be strictly enforced by the time it was laid out, and the lack of enforcement meant it was a classic chicken-and-egg problem where you couldn't expect people to spend a ton of effort setting something up when no one else was doing it.
It turns out though, that improvements in NLP mean there is still hope for a slightly different incarnation of the Semantic Web. There are a lot of companies out there now, like Diffbot [1] or Google [2], that are providing semantic searching of web data with structured results. It's not exactly the same as the concept of the Semantic Web, which would've required content publishers to tag their own data, but it does move towards the Semantic Web goal of making web data processable by machines.
[1] https://www.diffbot.com/
[2] https://developers.google.com/knowledge-graph/
It turns out though, that improvements in NLP mean there is still hope for a slightly different incarnation of the Semantic Web. There are a lot of companies out there now, like Diffbot [1] or Google [2], that are providing semantic searching of web data with structured results. It's not exactly the same as the concept of the Semantic Web, which would've required content publishers to tag their own data, but it does move towards the Semantic Web goal of making web data processable by machines.
[1] https://www.diffbot.com/
[2] https://developers.google.com/knowledge-graph/
This seems very true. Also I think the standard was just too ahead of the tools available at the time. Within the last 18 months it seems CI/CD has taken off, and obviously that is just anecdotal evidence from my life bubble that doesn’t reflect what ppl around the world have been doing for years, but that is something I would attribute to “the tools were finally good enough”. Especially when you consider that CI at least as a concept dates back to the late 1990’s with XP.
I wouldn't go so far as saying it has outright failed - it has made inroads in some areas, such as biblio databases and metadata in academic publishing in general. HTML metadata, microformats, and schema.org sees lots of use in SEO. Its applications weren't just as wide as initially envisioned, simply because giving away your data in an open format isn't a money-maker, in contrast to putting up a website displaying data along with ads and tracking.
> We were promised utopia
That sentence reveals another aspect (a misunderstanding IMHO): that "semantic web" and XML was shovelled onto you by a clueless academic ivory-tower elite, an anti-establishment narrative instrumental in the rise of HTML5, WHATWG and the post-standards world we live in today.
What kindof sucks with W3C's semantic web tech stack is that it captured a lot, yet was impractical, incomplete, and delivered very little. Even TBL's SOLID stack for the dweb uses semweb "creatively" for discovering graph URLs offered by a remote website for replication, by enumerating files/URLs with regexp filtering and similar mechanisms. Semweb has for a long time captured logic programming and description logic; only recently are Prolog and Datalog re-discovered as much more practical means for implementing graph and logic databases.
> We were promised utopia
That sentence reveals another aspect (a misunderstanding IMHO): that "semantic web" and XML was shovelled onto you by a clueless academic ivory-tower elite, an anti-establishment narrative instrumental in the rise of HTML5, WHATWG and the post-standards world we live in today.
What kindof sucks with W3C's semantic web tech stack is that it captured a lot, yet was impractical, incomplete, and delivered very little. Even TBL's SOLID stack for the dweb uses semweb "creatively" for discovering graph URLs offered by a remote website for replication, by enumerating files/URLs with regexp filtering and similar mechanisms. Semweb has for a long time captured logic programming and description logic; only recently are Prolog and Datalog re-discovered as much more practical means for implementing graph and logic databases.
I studied semantic web for a semester during college and I found it was too complicated and confusing. I remember stuff like RDF, Owl, ontologies, sparql, triple stores and others. But I never truly understood how was all connected. Besides, from what I remember, these tools were never production-ready since they came from academia. Although graph databases are gaining a lot of traction these days so there still might be some space for semantic meaning in some projects.
When rococode says 'It turns out though, that improvements in NLP mean there is still hope for a slightly different incarnation of the Semantic Web.'
maybe the reason it failed is just another example of metacrap https://people.well.com/user/doctorow/metacrap.htm
aside from that my take is that it failed at two different points - first point original RDF specs and push to add RDF, second point SPARQL.
First point reasons for failure (in no particular order) -
1. The RDF community were sort of jerks - I can't count how many times people came on XML-DEV or other communities talking about the difficulties of representing RDF in XML and were dismissed with haughty versions of you don't need XML to represent RDF. This was not the most winning attitude to get new people onboard.
2. RDF is hard to represent in XML and almost every tutorial that showed you how to work with RDF did it in context of XML. At the time XML was the runaway train of standards success, and it seemed like the idea was to attach the RDF car to it in any way they could. But the two things did not really attach well.
3. Web development then, as now, has been about having to deal with lots of changes all the time, anything that requires a steep learning curve, has no immediate benefit, and is hard to do right is at a significant disadvantage.
4. About the syntax and mapping of RDF to XML, at the time I was building a product that was supposed to handle media transformations between many XML inputs and RDF was a problem for me. I remember another of the commonly said things at the time was you don't need to worry about writing RDF because your tools will do all that for you, well I was writing the tools and I hated it.
So for these various reasons, and the metacrap one, RDF did not take off.
SPARQL not succeeding was actually somewhat surprising to me, when it first came out I thought it had potential, but I did think it was going against some things that had quite the traction at the time -
1. I would often see tutorials with these Sparql/Linked Data URIS had all the search query in them and looked awful, this was when everyone was very excited about REST and making theirs urls meaningful to humans - the examples given just looked bad in relation to the aesthetic current at the time.
2. RDF was already thought of as a failed technology - once a technology fails it is really hard to get management to invest in a second coming.
3. The bloom was off the rose for W3C at the time, this was when WHATWG was being formed, another strike against from a managerial standpoint I'd think.
4. Maybe the strains of Web Development are such that it cannot handle more than one big standard arriving at a time - this is just a feeling on my part but SPARQL and HTML5 were coming at about the same time (or is my recollection off - I think both started in 2008-9) maybe there's just too much work for people to do that.
Ok, these are of course various expressions of my personal feelings as to why it didn't work, but given our inability to really test the issue I suppose the same will apply to anyone else's ideas on the matter.
aside from that my take is that it failed at two different points - first point original RDF specs and push to add RDF, second point SPARQL.
First point reasons for failure (in no particular order) -
1. The RDF community were sort of jerks - I can't count how many times people came on XML-DEV or other communities talking about the difficulties of representing RDF in XML and were dismissed with haughty versions of you don't need XML to represent RDF. This was not the most winning attitude to get new people onboard.
2. RDF is hard to represent in XML and almost every tutorial that showed you how to work with RDF did it in context of XML. At the time XML was the runaway train of standards success, and it seemed like the idea was to attach the RDF car to it in any way they could. But the two things did not really attach well.
3. Web development then, as now, has been about having to deal with lots of changes all the time, anything that requires a steep learning curve, has no immediate benefit, and is hard to do right is at a significant disadvantage.
4. About the syntax and mapping of RDF to XML, at the time I was building a product that was supposed to handle media transformations between many XML inputs and RDF was a problem for me. I remember another of the commonly said things at the time was you don't need to worry about writing RDF because your tools will do all that for you, well I was writing the tools and I hated it.
So for these various reasons, and the metacrap one, RDF did not take off.
SPARQL not succeeding was actually somewhat surprising to me, when it first came out I thought it had potential, but I did think it was going against some things that had quite the traction at the time -
1. I would often see tutorials with these Sparql/Linked Data URIS had all the search query in them and looked awful, this was when everyone was very excited about REST and making theirs urls meaningful to humans - the examples given just looked bad in relation to the aesthetic current at the time.
2. RDF was already thought of as a failed technology - once a technology fails it is really hard to get management to invest in a second coming.
3. The bloom was off the rose for W3C at the time, this was when WHATWG was being formed, another strike against from a managerial standpoint I'd think.
4. Maybe the strains of Web Development are such that it cannot handle more than one big standard arriving at a time - this is just a feeling on my part but SPARQL and HTML5 were coming at about the same time (or is my recollection off - I think both started in 2008-9) maybe there's just too much work for people to do that.
Ok, these are of course various expressions of my personal feelings as to why it didn't work, but given our inability to really test the issue I suppose the same will apply to anyone else's ideas on the matter.
[deleted]
This is a good write-up on this exact question: https://twobithistory.org/2018/05/27/semantic-web.html
A few years ago I got really deep into semantic web. As a software engineer I had major issues understanding all the standards, jargon, and even value proposition. Complexity is not necessarily the issue here, but on top of that I found it difficult to provide a reasonable business justification to my manager.