I want to learn D3. I don’t want to learn Observable. Is that ok?(talk.observablehq.com)
talk.observablehq.com
I want to learn D3. I don’t want to learn Observable. Is that ok?
https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-want-to-learn-observable-is-that-ok/1957/2
92 comments
Something is wrong when we expect high quality libraries for free. The authors are highly skilled individuals with mortgages to pay and children to feed. Not sure what the answer is. If coupling Observable with d3 is such an answer, then more power to them. Related, I've stumbled upon this blurb from Codemirror6. Perhaps another answer.
If you are using CodeMirror commercially, there is a social expectation that you help fund its maintenance.
https://codemirror.net/6
If you are using CodeMirror commercially, there is a social expectation that you help fund its maintenance.
https://codemirror.net/6
There are countless high quality libraries available for free.
[deleted]
d3 for example.
There really aren’t. As soon as you need something t specific, which is literally all the time in software development, you generally have less than 5 useful implementations, and that’s being generous.
If you break up countless libraries in very small and specific categories, you can start to count them, yes.
> Something is wrong when we expect high quality libraries for free.
Expecting free libraries is natural because the world is built on free libraries and free software. There's models to be paid within this of course (the largest by establishing expertise and then selling support). On the contrary, expecting everyone to pay for libraries is a model proven to have a much more limited lifespan (which also has implications on the developer's career).
Expecting free libraries is natural because the world is built on free libraries and free software. There's models to be paid within this of course (the largest by establishing expertise and then selling support). On the contrary, expecting everyone to pay for libraries is a model proven to have a much more limited lifespan (which also has implications on the developer's career).
My developer career has been great focusing mostly on software products that consumers care to pay for.
In fact, if I learned something from free beer libraries is that it was a mistake to fall down that rabbit hole.
In fact, if I learned something from free beer libraries is that it was a mistake to fall down that rabbit hole.
Bait and switch is not acceptable. If they want to make money from D3, they should charge for D3. What they're doing is disingenuous and has strong similarities to the drug trade. I don't expect people to work for me for free but I do expect people to not tell me they're going to do something for me for free and then try to find a way to charge me for it later. If they want to make a living from D3, they should be upfront about it and charge accordingly.
This remark is really unkind. D3 precedes Observable, and the development path is exactly what someone would take if they wanted to make data visualization accessible - create a base library, and later opinionated tools to work with it.
The harshest way to interpret unavailability of regular JS tutorials should be that the primary authors don't have enough bandwidth or the inclination to work on it, and the community hasn't picked up the slack to make it more accessible to beginners. The authors deserve nothing other than gratitude.
The harshest way to interpret unavailability of regular JS tutorials should be that the primary authors don't have enough bandwidth or the inclination to work on it, and the community hasn't picked up the slack to make it more accessible to beginners. The authors deserve nothing other than gratitude.
Maybe they should not redirect users to observable when clicking on d3js examples.
Why? It’s free software and a free OSS library. Be thankful examples even exist at all.
That’s not really the spirit of open source. Sharing, not charity.
This is about taking something that already existed and repurposing it for a commercial venture.
This is about taking something that already existed and repurposing it for a commercial venture.
I’ve done a lot of open source and don’t get where you’re going.
What was repurposed exactly? Blocks? A deprecated service that nobody volunteered to take over, despite messages asking for that?
Something else?
Was it free before? Is it still free?
What was repurposed exactly? Blocks? A deprecated service that nobody volunteered to take over, despite messages asking for that?
Something else?
Was it free before? Is it still free?
What does free have to redirect to another project. At minimum it should redirect to github examples page.
It redirects to live d3 examples. Free documentation that you can interact with.
That’s like saying someone redirecting documentation to codepen is “redirecting to another project.”
“At minimum” - your entitlement is truly ridiculous. There is no “at minimum” - it’s all free ffs.
That’s like saying someone redirecting documentation to codepen is “redirecting to another project.”
“At minimum” - your entitlement is truly ridiculous. There is no “at minimum” - it’s all free ffs.
It does not need to redirect to observables. It could have lived in d3js itself. I am not sure if i can copy paste it to a html/js page and run.
“At minimum” - your entitlement is truly ridiculous. So just saying a link to github is now not tolerated!
“At minimum” - your entitlement is truly ridiculous. So just saying a link to github is now not tolerated!
Hey I’m not sure what you mean. E.g. “ It could have lived in d3js itself.” doesn’t make sense.
Documentation does exist on GitHub.
Everything in observable is bonus.
Documentation does exist on GitHub.
Everything in observable is bonus.
Creating tutorials is not bait and switch. Any competent web dev can learn from the documentation instead. The Observable tutorials are no different from e.g. Heroku tutorials on Rails, or the usual web tech blog spam Logrocket churns out.
> has strong similarities to the drug trade.
well that's a bit over the top.
well that's a bit over the top.
> > has strong similarities to the drug trade.
> well that's a bit over the top.
“There are only two industries that call their customers ‘users’: illegal drugs and software.” — Edward Tufte
> well that's a bit over the top.
“There are only two industries that call their customers ‘users’: illegal drugs and software.” — Edward Tufte
"Sadly many computer users become computer dealers in order to support their habit."
-- someone's .signature on Usenet around 1992
-- someone's .signature on Usenet around 1992
Observable is conceptually quite simple. Each cell is a bit of JavaScript, and cells are run reactively, i.e. when their free variables change. That is basically it. It's not actually necessary to know much JS to become productive in Observable and it's fairly close to being orthogonal to D3, believe it or not.
Having said that, Observable is not a mature system and there are some traps for the unwary: https://observablehq.com/@tmcw/observable-anti-patterns-and-... This kind of thing would certainly make me think twice about depending on Observable as a platform for a major project. There's a certain confidence in the act of name "anti-patterns" and "code smells" which belies the slightly ad hoc, provisional status of the platform.
And not everything is taken care of elegantly in Observable: making an animation tends to involve more JS control-flow tricks than would be ideal: https://observablehq.com/@mbostock/animation-loops
The idea that Observable makes life more difficult for people (in the service of building a profitable platform) doesn't ring true to me, though. To return to what I said in the first paragraph: Observable is conceptually simple (and most parts of it already have third party open source implementations). D3 is as intimidating as it's ever been, but I think the complaint "now I seem to have to learn Observable as well as D3" is not based on a good faith assessment of the complexity of learning Observable, and shouldn't be seriously entertained. It's in the "petulant" category of complaints about software.
Having said that, Observable is not a mature system and there are some traps for the unwary: https://observablehq.com/@tmcw/observable-anti-patterns-and-... This kind of thing would certainly make me think twice about depending on Observable as a platform for a major project. There's a certain confidence in the act of name "anti-patterns" and "code smells" which belies the slightly ad hoc, provisional status of the platform.
And not everything is taken care of elegantly in Observable: making an animation tends to involve more JS control-flow tricks than would be ideal: https://observablehq.com/@mbostock/animation-loops
The idea that Observable makes life more difficult for people (in the service of building a profitable platform) doesn't ring true to me, though. To return to what I said in the first paragraph: Observable is conceptually simple (and most parts of it already have third party open source implementations). D3 is as intimidating as it's ever been, but I think the complaint "now I seem to have to learn Observable as well as D3" is not based on a good faith assessment of the complexity of learning Observable, and shouldn't be seriously entertained. It's in the "petulant" category of complaints about software.
Hi! I wrote that article.
Basically… the anti-patterns stuff is really not that different from "React anti-patterns". JavaScript supports mutability, a lot of frameworks embrace immutable patterns (like Observable, and React). Using the mutability will lead to bad situations, like mutating in Observable or mutating some state in React.
The same with selecting elements, too. Like - if you're in React (or Vue, or… most component libraries) doing a document.querySelector is no good. You break modularity and the ability to add multiple copies of a component to a screen. Same with Observable. Selecting elements from a necessarily-global DOM is bad.
The gist is: Observable isn't JavaScript but is really close to JavaScript. We could either make it mostly-JavaScript and have some gotchas around mutability and such (but make it a lot friendlier to JS devs, use existing libraries, etc) or make it something stricter and do a 'clean break.' If Observable was way different than JavaScript, this thread about Observable "having lock-in / being too hard" would be 10x bigger :) Such is the balance between practicality and purity.
You can see plenty of other things trying to do this balance. Like React (not quite JavaScript, because of JSX), Svelte (very much not vanilla JavaScript), Elm (pure, idealistic, also its own island because of its purity). Unfortunately you can only really do a pure system with those drawbacks, a practical system with those drabacks, or some combination of the two.
Basically… the anti-patterns stuff is really not that different from "React anti-patterns". JavaScript supports mutability, a lot of frameworks embrace immutable patterns (like Observable, and React). Using the mutability will lead to bad situations, like mutating in Observable or mutating some state in React.
The same with selecting elements, too. Like - if you're in React (or Vue, or… most component libraries) doing a document.querySelector is no good. You break modularity and the ability to add multiple copies of a component to a screen. Same with Observable. Selecting elements from a necessarily-global DOM is bad.
The gist is: Observable isn't JavaScript but is really close to JavaScript. We could either make it mostly-JavaScript and have some gotchas around mutability and such (but make it a lot friendlier to JS devs, use existing libraries, etc) or make it something stricter and do a 'clean break.' If Observable was way different than JavaScript, this thread about Observable "having lock-in / being too hard" would be 10x bigger :) Such is the balance between practicality and purity.
You can see plenty of other things trying to do this balance. Like React (not quite JavaScript, because of JSX), Svelte (very much not vanilla JavaScript), Elm (pure, idealistic, also its own island because of its purity). Unfortunately you can only really do a pure system with those drawbacks, a practical system with those drabacks, or some combination of the two.
[deleted]
Actually, Observable is not JavaScript:
https://observablehq.com/@observablehq/observables-not-javas...
I think it's an amazing piece of technology, hosting some of the best interactive stories on the internet — but I too find it hard to use.
https://observablehq.com/@observablehq/observables-not-javas...
I think it's an amazing piece of technology, hosting some of the best interactive stories on the internet — but I too find it hard to use.
It seems particularly unfortunate for most of the documentation of something that is supposedly a Javascript library to assume you are using something that is not actually quite Javascript.
[deleted]
Tangential: in D3.js 7.0 Bostock switched license from BSD 3-clause to ISC [0].
[0] https://github.com/d3/d3/commit/23b0212743d22e2dea3e20163980...
[0] https://github.com/d3/d3/commit/23b0212743d22e2dea3e20163980...
These licenses are essentially interchangeable. The ISC license just drops a bit of text that was determined to be unnecessary. https://en.wikipedia.org/wiki/ISC_license
Already posted 22 days ago but did not get any traction.
Inspired by https://news.ycombinator.com/item?id=27485123
Inspired by https://news.ycombinator.com/item?id=27485123
I had this exact problem myself two weeks back. All the pre-Observable examples I found were for D3 v4, while D3 is now at v6.
Some great resources in the thread, which I hadn’t found by Googling. Thanks!
Some great resources in the thread, which I hadn’t found by Googling. Thanks!
I’ve figured out how to go the other way - from a JS framework to Observable. I put all the D3 code in a reusable chart [1] which can be copied and pasted into an Observable cell. And have the JS framework handle things like loading data which is modified slightly when transferred to Observable. I don’t want to be too dependent on a proprietary service but it’s nice to share your work on it occasionally.
[1] https://bost.ocks.org/mike/chart/
[1] https://bost.ocks.org/mike/chart/
I've had the goal to learn modern JS and D3 for a few years now. And I struggled with finding a good ressource as well.
Over the past year I've done it finally, and I can recommend the book "D3 for the impatient". It's written for D3 v4.x, but I wanted to write my project with D3 v6, obviously. Luckily, just going for it and googling the error message when I got the inevitable "foo is not a function" was easy enough. Now I'm confident enough with it that just reading the github readme of a new D3 module is usually enough to get me going.
But the API has quite a bit rough edges (like the difference between select and selectAll, the selection.merge(enter) pattern), that a good book is really nice. Especially since online tutorials are a complete mess - because of Observable non-sense and severe API rotting.
Over the past year I've done it finally, and I can recommend the book "D3 for the impatient". It's written for D3 v4.x, but I wanted to write my project with D3 v6, obviously. Luckily, just going for it and googling the error message when I got the inevitable "foo is not a function" was easy enough. Now I'm confident enough with it that just reading the github readme of a new D3 module is usually enough to get me going.
But the API has quite a bit rough edges (like the difference between select and selectAll, the selection.merge(enter) pattern), that a good book is really nice. Especially since online tutorials are a complete mess - because of Observable non-sense and severe API rotting.
There is now selection.join() [1] which means you don’t have to use selection.merge() anymore.
[1] https://observablehq.com/@d3/selection-join
[1] https://observablehq.com/@d3/selection-join
Yes I know! It's a really good improvement. Note that your link nicely proves OP's point ;)
[deleted]
No it doesn't.
You only have to change two lines to vanilla javascript.
document.getElementById("myRoot").appendChild(svg.node());
and remove the yield
Thats it. But apparently you all don't know this, because you don't want to understand D3 or basic DOM APIs beyond ctrl-c/ctrl-v.
document.getElementById("myRoot").appendChild(svg.node());
and remove the yield
Thats it. But apparently you all don't know this, because you don't want to understand D3 or basic DOM APIs beyond ctrl-c/ctrl-v.
I think you may be incorrect regarding its version compatibility. See the author's github repository here:
https://github.com/janert/d3-for-the-impatient
> All examples have been verified to work with this version of the D3 library (D3 Version 5.9.2, downloaded 15. Apr 2019)
So, not quite V6, but a lot more up-to-date than v4.
'D3 for the impatient' is indeed an excellent book and was my entry point into using D3 aswell.
https://github.com/janert/d3-for-the-impatient
> All examples have been verified to work with this version of the D3 library (D3 Version 5.9.2, downloaded 15. Apr 2019)
So, not quite V6, but a lot more up-to-date than v4.
'D3 for the impatient' is indeed an excellent book and was my entry point into using D3 aswell.
If anyone is having this problem, just use the tutorial series' on Observable. They really are great and provide a nice environment for learning. Later, when you want to productionize it or create self-contained applications, nothing's stopping you. A quick google will show you how to build your D3 chart as a React component. You're not learning any non-transferable skills by learning on Observable.
If I'm already writing a comment, I'd like to say that D3 is so awesome and magical! It's surprisingly easy to create nice-looking animated visualizations.
After going through the Observable tutorials for fun I quickly found ways to use it at my employer[0] and create various useful and well-looking animations, you can see one here: https://youtu.be/N9haFA_MwdM
FWIW the tutorial series I used myself was https://observablehq.com/collection/@sxywu/introduction-to-d... , I recommend it very much.
[0]:https://spacelift.io
If I'm already writing a comment, I'd like to say that D3 is so awesome and magical! It's surprisingly easy to create nice-looking animated visualizations.
After going through the Observable tutorials for fun I quickly found ways to use it at my employer[0] and create various useful and well-looking animations, you can see one here: https://youtu.be/N9haFA_MwdM
FWIW the tutorial series I used myself was https://observablehq.com/collection/@sxywu/introduction-to-d... , I recommend it very much.
[0]:https://spacelift.io
[deleted]
I was in this exact same situation a year ago. Learning d3 had always been on my list of things to do but I never got around to it. When I finally found some time I realized that all their resources and tutorials had moved to this heavy interactive thing called "observable" - I was furious because I spent hours trying to find old tutorials. Maybe Observable is a good product for some use cases, but I don't want to deal with it. I want to have plain HTML with plain JS so that I can actually understand what's going on. No layers upon layers of magic. Eventually I gave up.
We need to stop forcing all this unnecessary complexity upon users.
We need to stop forcing all this unnecessary complexity upon users.
For the lower-tech people Observable is targeting (who are in the linked thread), I suspect that having to write tons of Javascript/HTML is actually the "unnecessary complexity" they precisely want to avoid entirely. So Observable is not unnecessary complexity: it's actually a solution to the complex problem they already have.
This kind of discussion you're pointing towards isn't actually a discussion about difficult ideas like "unnecessary complexity," or what is and isn't "complex", or "software magic", as if that was a singular ontological property or category, waiting to be applied. The discussion you're trying to have is ultimately about words, even if you don't think so. If you don't agree on the words with someone else, then trying to have a discussion of ideas, ideas like "unnecessary complexity", is less like a discussion and more like two cars flying past each other on a highway. Your point is just falling on deaf ears, except for everyone who was driving to the same destination already.
Rather than having discussions about that stuff, I think the much more obvious answer pointed at elsewhere -- that Observable provides money for the developers, and D3 does not -- is much more useful as a starting point. And frankly, I think much more plausible. It's a significantly more difficult topic to discuss social issues like monetary support for other people in our nerd culture, than it is to discuss "Is this software complex" -- a thing nerds love to opine over endlessly to avoid actual difficult discussions, as if it had any crisp definition. But it's also significantly more honest and a better reflection of the underlying motives, the words behind the discussion, than zooming 60mph past someone on a completely different topic going the other direction.
I suspect Observable does in fact make money, while D3 doesn't, and so it's understandable that things play out this way, if my assumption were correct. It sucks for highly technical people like myself, or you, who also may not want Observable and are perfectly fine on our own. But talking about software complexity is a red herring; it comes across more as an attempt to fit the world in a neat box of rules, rather than handle it as it is. And that will never leave you satisfied, and always leave you feeling alienated.
This kind of discussion you're pointing towards isn't actually a discussion about difficult ideas like "unnecessary complexity," or what is and isn't "complex", or "software magic", as if that was a singular ontological property or category, waiting to be applied. The discussion you're trying to have is ultimately about words, even if you don't think so. If you don't agree on the words with someone else, then trying to have a discussion of ideas, ideas like "unnecessary complexity", is less like a discussion and more like two cars flying past each other on a highway. Your point is just falling on deaf ears, except for everyone who was driving to the same destination already.
Rather than having discussions about that stuff, I think the much more obvious answer pointed at elsewhere -- that Observable provides money for the developers, and D3 does not -- is much more useful as a starting point. And frankly, I think much more plausible. It's a significantly more difficult topic to discuss social issues like monetary support for other people in our nerd culture, than it is to discuss "Is this software complex" -- a thing nerds love to opine over endlessly to avoid actual difficult discussions, as if it had any crisp definition. But it's also significantly more honest and a better reflection of the underlying motives, the words behind the discussion, than zooming 60mph past someone on a completely different topic going the other direction.
I suspect Observable does in fact make money, while D3 doesn't, and so it's understandable that things play out this way, if my assumption were correct. It sucks for highly technical people like myself, or you, who also may not want Observable and are perfectly fine on our own. But talking about software complexity is a red herring; it comes across more as an attempt to fit the world in a neat box of rules, rather than handle it as it is. And that will never leave you satisfied, and always leave you feeling alienated.
But then again, you could just not use d3 in the first place and write a thin translation from your data structures to svg strings. Maybe use a template engine. You don’t need to cover the whole spec for your use case.
SVG seems really intimidating at first, but for simple (especially non-work) use cases where you only need to do one thing, it's actually really fast and easy to generate minimal SVG directly from data. The learning curve is no steeper than learning a does-everything plotting library, again for simple unchanging usecases.
Prerequisites: knowing how to scale numbers from one range (data range) to another (SVG coordinate range) using multiplication and addition.
Prerequisites: knowing how to scale numbers from one range (data range) to another (SVG coordinate range) using multiplication and addition.
Yes, I ended up not using d3 and using something else instead. But it's a shame because I thought d3 is a great project and a good fit for my use case. I just didn't want to waste my time and energy dealing with Observable.
What did you use and was it up to the task?
Are you sure it's unnecessary complexity?
To someone else, HTML and JavaScript may be "unnecessary" complexity and they want to see the plain C/C++ code of how to render a web page with dynamic animation.
To someone else, HTML and JavaScript may be "unnecessary" complexity and they want to see the plain C/C++ code of how to render a web page with dynamic animation.
d3 and observable is closer to the essence of KISS than "unnecessary complexity".
I find @mbstock's answer slightly disheartening: https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w...
Instead of "we should make this more approachable without buying into Observable" it's "we should spend effort on helping you learn Observable and explaining why you need it".
@sanjalot's final comment is somewhat better https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w...
But instead of links in a forum somewhere to someone else's Observable explaining how to turn Observable's into plain JS code, this info should be front and center on the first page of official docs.
Instead of "we should make this more approachable without buying into Observable" it's "we should spend effort on helping you learn Observable and explaining why you need it".
@sanjalot's final comment is somewhat better https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w...
But instead of links in a forum somewhere to someone else's Observable explaining how to turn Observable's into plain JS code, this info should be front and center on the first page of official docs.
This seems to me like a problem the community can readily solve without demanding that the head of Observable solve it for them. Make a nice neutrally-positioned (i.e. not openly hostile to Observable) site that manages plain JS examples and perhaps you can even get Observable to link to it!
Hopefully I'm not completely out of place asking this here, but does anyone have any suggestions for why a d3 animation inside React might be a bit janky? Looks smooth as butter on my old computer, but for some reason on my better performing computer it looks janky!
I sympathize with the comments, as a software engineer, this comment from Mike really describes my frustration with Observable's runtime:
> Observable notebooks are like spreadsheets, where cells run automatically whenever you edit or values change. That’s not how conventional (imperative) programming languages work, so sadly you can’t simply copy-paste a whole notebook into a vanilla JavaScript application.
However, I do think many others will prefer their reactive runtime; so this is by no means a criticism to Observable, they are doing great work. They are just not targeting JS-purists and that might be the right call.
As a software engineer, I love D3 but don't want to be stuck with a reactive runtime that is not vanilla JavaScript.
I got myself to build https://hal9.ai, an integrated environment to do data analysis based of JavaScript. Is not a D3 learning environment but you can certainly use it for that purpose. If someone is interested in providing feedback or helping with D3 examples, please do so at https://github.com/hal9ai/hal9ai or shoot me an email at javier at hal9.ai
> Observable notebooks are like spreadsheets, where cells run automatically whenever you edit or values change. That’s not how conventional (imperative) programming languages work, so sadly you can’t simply copy-paste a whole notebook into a vanilla JavaScript application.
However, I do think many others will prefer their reactive runtime; so this is by no means a criticism to Observable, they are doing great work. They are just not targeting JS-purists and that might be the right call.
As a software engineer, I love D3 but don't want to be stuck with a reactive runtime that is not vanilla JavaScript.
I got myself to build https://hal9.ai, an integrated environment to do data analysis based of JavaScript. Is not a D3 learning environment but you can certainly use it for that purpose. If someone is interested in providing feedback or helping with D3 examples, please do so at https://github.com/hal9ai/hal9ai or shoot me an email at javier at hal9.ai
I learnt d3 with <script> tags.
I'm glad to hear that others are equally frustrated with the tight coupling between D3 and Observable.
For me, it just wasn't worth the effort of learning D3 AND Observable (especially, not being particularly fond of JS to start with) - in the end, it was too much, and I ended up using a simpler third party javascript charting library instead - which was a real shame, because I really loved what D3 could achieve.
Mike Bostock is clearly a genius, but I think his brain is wired differently from the rest of us mere mortals - and unfortunately, he seems to have the knack of making simple things terribly complicated.
For me, it just wasn't worth the effort of learning D3 AND Observable (especially, not being particularly fond of JS to start with) - in the end, it was too much, and I ended up using a simpler third party javascript charting library instead - which was a real shame, because I really loved what D3 could achieve.
Mike Bostock is clearly a genius, but I think his brain is wired differently from the rest of us mere mortals - and unfortunately, he seems to have the knack of making simple things terribly complicated.
I had no idea that the creator of d3js is also the co-founder of Observable! Now this move makes a lot more sense. I'm starting to believe this may be another case where the creator (rightly so) wanted to find a way to monetize his popular open source project. It's a shame this happened. I don't blame him, but the broken open source business model.
Yep, this is 100% my read.
It’s a damn shame too because d3 is fabulous and I thought he’d already encountered enough wealth through his work with NYT etc.
Surely there’s a way to grow Observable without cannibalizing d3.
It’s a damn shame too because d3 is fabulous and I thought he’d already encountered enough wealth through his work with NYT etc.
Surely there’s a way to grow Observable without cannibalizing d3.
I used D3 v5 only for the force directed graphs part, independently in an artificial life & lambda calculus project. Is maybe too much for such an austere domain, since I saw comments that the result is "unnecessarily flashy". Give me any other visually comparable, but lower level js library for the same purpose and I'll take it. I'm not a programmer, but to paraphrase Mark Twain from A Connecticut Yankee in King Arthur's Court, I felt "I was standing in the awful presence of the Mother of the" functional_programming "language".
It's such a small thing but I can't stand having the output of a cell evaluate above the cell. I get that it's part of the Observable reactive environment, but it breaks what we're used to with the command line and other notebook applications.
I agree, it's confusing enough to make the whole reactive concept less approachable. It'd help if the output didn't look so much like the cell itself, but it's not entirely clear how to differentiate them visually.
[deleted]
The question is about lock-in and choice. You can see on the thread the Observable authors seem almost proud of the fact that their tooling is so good that it couldn't possibly ever be made to be more compatible with everyone else's way of working. In the rapidly moving JS world, this is arrogant and shows poor foresight.
I love learning new tools, but this notebook craze is alarming. When I pay a python contractor I have to specify "a runnable script" or I get a jupyter notebook as a deliverable. I hate to think D3 is going the same way. It's no good to make your tool so hard that it needs a dedicated, incompatible (and morally superior) development environment.
I love learning new tools, but this notebook craze is alarming. When I pay a python contractor I have to specify "a runnable script" or I get a jupyter notebook as a deliverable. I hate to think D3 is going the same way. It's no good to make your tool so hard that it needs a dedicated, incompatible (and morally superior) development environment.
As someone building an in-browser notebook I have a lot of opinions on notebook environments. Notebooks serve different purposes, sometimes the notebook itself is the end-goal because the author is creating an interactive tutorial or explaining a complex concept with a bunch of visualizations. Observable is a fantastic tool for that, and the kind-of-Javascript reactive programming system it is built on is a great fit for that.
Outside of that use-case, I think notebooks are great for the first 20% of the effort that gets 80% of the work done. If it turns out one also needs to do the other 80% of the effort to get the last 20%, it is time to "graduate" away from a notebook. For instance if I am participating in a Kaggle machine learning competition I may train my first models in a Jupyter notebook for quick iteration on ideas, but when I settle onto a more rigid pipeline and infra, I will move to plain Python files that I can test and collaborate on.
This "graduation" from notebook to the "production/serious" environment should be straightforward, which means there shouldn't be too much magic in the notebook without me opting into it. Documentation in my eyes is not so different, I should be able to copy the examples easily into my JS project without knowing specifics of Observable and adapt it to my problem. Saying "don't be lazy and just learn Observable", or "you must learn D3 itself properly to be able to use it anyway" is not helpful. Observable being a closed, walled garden doesn't help: not being able to author notebooks without using their closed source editor is a liability that I can totally understand makes it a non-starter for some companies and individuals.
I think it's ok to plug my own project: It's called Starboard [1] and is truly open source [2]. It's built on different principles: it's hackable, extendable, embeddable, shareable, and easy to check into git (i.e. I try to take what makes the web so great and put that in a notebook environment). You write vanilla JS/ES/Python/HTML/CSS, but you can also import your own more advanced cell types. Here's an example which actually introduces an Observable cell type [3] which is built upon the Observable runtime (which is open source) and an unofficial compiler package [4]. I would be happy for the D3 examples to be expressed in these really-close-to-vanilla JS notebooks, but I doubt I can convince the maintainers to do so.
[1]: https://starboard.gg
[2]: https://github.com/gzuidhof/starboard-notebook
[3]: https://starboard.gg/gz/open-source-observablehq-nfwK2VA
[4]: https://github.com/asg017/unofficial-observablehq-compiler
Outside of that use-case, I think notebooks are great for the first 20% of the effort that gets 80% of the work done. If it turns out one also needs to do the other 80% of the effort to get the last 20%, it is time to "graduate" away from a notebook. For instance if I am participating in a Kaggle machine learning competition I may train my first models in a Jupyter notebook for quick iteration on ideas, but when I settle onto a more rigid pipeline and infra, I will move to plain Python files that I can test and collaborate on.
This "graduation" from notebook to the "production/serious" environment should be straightforward, which means there shouldn't be too much magic in the notebook without me opting into it. Documentation in my eyes is not so different, I should be able to copy the examples easily into my JS project without knowing specifics of Observable and adapt it to my problem. Saying "don't be lazy and just learn Observable", or "you must learn D3 itself properly to be able to use it anyway" is not helpful. Observable being a closed, walled garden doesn't help: not being able to author notebooks without using their closed source editor is a liability that I can totally understand makes it a non-starter for some companies and individuals.
I think it's ok to plug my own project: It's called Starboard [1] and is truly open source [2]. It's built on different principles: it's hackable, extendable, embeddable, shareable, and easy to check into git (i.e. I try to take what makes the web so great and put that in a notebook environment). You write vanilla JS/ES/Python/HTML/CSS, but you can also import your own more advanced cell types. Here's an example which actually introduces an Observable cell type [3] which is built upon the Observable runtime (which is open source) and an unofficial compiler package [4]. I would be happy for the D3 examples to be expressed in these really-close-to-vanilla JS notebooks, but I doubt I can convince the maintainers to do so.
[1]: https://starboard.gg
[2]: https://github.com/gzuidhof/starboard-notebook
[3]: https://starboard.gg/gz/open-source-observablehq-nfwK2VA
[4]: https://github.com/asg017/unofficial-observablehq-compiler
I don't get what the author is complaining about, D3 has great documentation on GitHub; With or without Observable the way to use D3 is pretty much the same.
Yes, the best tutorials for D3 are on Observable; But translating these to pure JavaScript is trivial for anyone who knows some basic javascript syntax.
Those who have zero experience with using JavaScript in the first place should first learn the language, and only then try and attempt at using the more advanced libraries such as D3. It is going to be a real pain to try and get anything done with D3 without understanding some basic JavaScript syntax first.
Yes, the best tutorials for D3 are on Observable; But translating these to pure JavaScript is trivial for anyone who knows some basic javascript syntax.
Those who have zero experience with using JavaScript in the first place should first learn the language, and only then try and attempt at using the more advanced libraries such as D3. It is going to be a real pain to try and get anything done with D3 without understanding some basic JavaScript syntax first.
Not only that but: "You can download the vanilla JavaScript for any notebook by clicking Download Code in the notebook menu."
So... you don't even have to do the trivial transform yourself. Provided those kinds of tools remain available I'm struggling to see the problem.
I think obervable is coming from a good place and committed to making their tech as accessable as possible overall. I'm sure there's plenty of less technical users who would have shied away from just a github homepage that might get into it now.
So... you don't even have to do the trivial transform yourself. Provided those kinds of tools remain available I'm struggling to see the problem.
I think obervable is coming from a good place and committed to making their tech as accessable as possible overall. I'm sure there's plenty of less technical users who would have shied away from just a github homepage that might get into it now.
I have tried just that, and the resulting code is underwhelming, to the tune of 'import antigravity':
<div id="observablehq-chart-6aa5c38e"></div>
<p>Credit: <a href="https://observablehq.com/@d3/contours">Contours by D3</a></p>
<script type="module">
import {Runtime, Inspector} from "https://cdn.jsdelivr.net/npm/@observablehq/runtime@4/dist/runtime.js";
import define from "https://api.observablehq.com/@d3/contours.js?v=3";
new Runtime().module(define, name => {
if (name === "chart") return new Inspector(document.querySelector("#observablehq-chart-6aa5c38e"));
});
</script>The result is: import the open-source Observable runtime and then use it to run the reactive notebook code. What else would you expect?
The reactive notebook code is written to work when run against the runtime. If you want to translate it to run in a different context where there is no reactivity, you are going to need to read the code and modify the parts that depend on that reactivity.
For example, if a cell depends on a variable that was set to be the reactive output of an HTML slider, then you are going to need to put the slider separately into your page, and write the code yourself to hook up the slider’s event handling logic.
This is not too difficult, but the notebook version is typically nicer and more legible.
The reactive notebook code is written to work when run against the runtime. If you want to translate it to run in a different context where there is no reactivity, you are going to need to read the code and modify the parts that depend on that reactivity.
For example, if a cell depends on a variable that was set to be the reactive output of an HTML slider, then you are going to need to put the slider separately into your page, and write the code yourself to hook up the slider’s event handling logic.
This is not too difficult, but the notebook version is typically nicer and more legible.
How much of it this problem is the difference between I want to understand this code vs I want to copy-paste some code to get my plot/chart done?
This problem would exist with productionizing any notebook style code. E.g. Jupyter notebooks.
This problem would exist with productionizing any notebook style code. E.g. Jupyter notebooks.
100% of the problem is people’s desire to copy/paste code directly into their own environment, and then getting frustrated that they can’t do that without putting in a bit of effort to understand the way the Observable runtime works, and then read the code they want to copy and think about it a bit.
Slightly off topic, but is anyone aware of a collection of examples or best practices for using d3 in a Vue application? I just started with d3 and I‘m not sure how to find the sweet spot regarding which DOM-operations are done by d3 and which should be handled by vue (and if that, how).
I'm in this situation and yesterday I replaced all the d3 SVG generation (d3.append, select, data, join, etc) with Vue template syntax (using v-for). So now I'm only using d3 for the x and y axis functions. The code is now much simpler. However, I lost the ability to animate with d3.transition, which I had been contemplating.
The learning curve for d3 is so steep that people are starting to confuse it for observablehq.
This may sound like a weird question -- but is there any reason not to use an older version if you feel comfortable with it?
Mentioned on an earlier thread, but this is part of the reason I started my book "Visualisation with D3.js"[1]... it's early access and I'm trying to find more time to work on it. It's entirely separate from Observable, and it's my way of formalising my D3.js learning journey whilst producing something I can refer to in future.
[1] https://datacrayon.com/shop/product/visualisation-with-d3/
[1] https://datacrayon.com/shop/product/visualisation-with-d3/
As others have pointed out: it's simply because Observable is where they will eventually make money. Both belong to the same author, so their coupling has incentives.
If you could entirely self-host the whole Observable experience it wouldn't be so bad. But you can't.
If you could entirely self-host the whole Observable experience it wouldn't be so bad. But you can't.
ObservableHQ has the tightest feedback loops because of the reactive runtime. It's a better way to program visual things. I prefer non-sequentially executing programs. It's better.
Code changes are local operations, so your running program state is generally preserved . This is an amazing boost for iterative development. This is Smalltalk level productivity boosts
Sucks if you want to do something legacy but I want to move forward with programming and breaking the sequential execution model is the right step IMHO.
Code changes are local operations, so your running program state is generally preserved . This is an amazing boost for iterative development. This is Smalltalk level productivity boosts
Sucks if you want to do something legacy but I want to move forward with programming and breaking the sequential execution model is the right step IMHO.
Here's what's really going on, and what I think the observable team should have said:
d3 was born ~10 years ago, before the rise of vue and react. Today, the community is split into two groups that we shall call "data viz" and "software engineers". The crux of that whole comment thread is that large portions of the d3 API have now been superseded by React/Vue/Svelte, for the software engineering crowd.
You can even see Mike Bostock and the Observable team in places recommending the book and blog post of Amelia Wattenberger. But look at her blog post on using d3 with react: https://wattenberger.com/blog/react-and-d3 If you read that you'll see that the Tl:DR is basically
> Don't use d3 any longer for anything other than the axis objects: use react/vue to build and update the SVG elements.
I suspect that Mike Bostock's (radical) decision to move d3 entirely into the Observable world was a completely rational decision explicitly addressing the fact that he knows that, when it comes to building and updating the DOM, d3 has now been replaced by component-based libraries with HTML-like JSX syntax, i.e. Vue/React etc. At least for software engineers; there is still a community of dataviz people with their center of gravity more in academia and data science who don't use Vue/React and for whom it makes sense to use the entire d3 API (DOM updates as well as utilities like axis functions etc)
I think that it would help if the Observable team just stood up and said that explicitly. Maybe even say on the d3 README:
> If you are a software engineer, bear in mind that things have moved on from 8 years ago and that you may in fact be best served by using utilities from d3 but using React or Vue for your DOM building.
Also, I think that the Observable team did themselves no favours by their slightly evangelical insistance on encouraging people to try Observable because they'll find that they like it once they try. I am a software engineer. I am open minded. I have tried Jupyter many times and I have tried Observable. I cannot use those tools for one simple reason: I use git to navigate my code state. If I am debugging, I make small commits, and use stash and revert and cherry-pick etc to experiment with different code states. I think in git and I work in git. Observable (and Jupyter) are completely antithetical to Git. So I also think that the Observable team would do better to say:
> Look. If you're a professional software engineer who is used to working in a sophiisticated fashion with version control, then honestly this stuff might not be your cup of tea. I'm sure you can see what we're trying to do, and that it's really cool, but we understand if you just want to enter lines of code into a git repo using your text editor. And for you, here's a quick summary/pointer to docs on how to use d3 in that way, with Vue/React, and some pointers on how to make use of the Observable docs for non-Observable users.
Mike Bostock if there's any chance you read this, d3 is one of the most impressive open source libraries I've known and I have been a fan of yours for years! Observable looks amazing and very forward-thinking too and I think I understand that it's aiming to democratize javascript visualization. I hope the above comments came across as constructive. (It would be awesome to have more pointers in the future on ways to make use of portions of the d3 API with other DOM-building APIs such as Vue/React.)
d3 was born ~10 years ago, before the rise of vue and react. Today, the community is split into two groups that we shall call "data viz" and "software engineers". The crux of that whole comment thread is that large portions of the d3 API have now been superseded by React/Vue/Svelte, for the software engineering crowd.
You can even see Mike Bostock and the Observable team in places recommending the book and blog post of Amelia Wattenberger. But look at her blog post on using d3 with react: https://wattenberger.com/blog/react-and-d3 If you read that you'll see that the Tl:DR is basically
> Don't use d3 any longer for anything other than the axis objects: use react/vue to build and update the SVG elements.
I suspect that Mike Bostock's (radical) decision to move d3 entirely into the Observable world was a completely rational decision explicitly addressing the fact that he knows that, when it comes to building and updating the DOM, d3 has now been replaced by component-based libraries with HTML-like JSX syntax, i.e. Vue/React etc. At least for software engineers; there is still a community of dataviz people with their center of gravity more in academia and data science who don't use Vue/React and for whom it makes sense to use the entire d3 API (DOM updates as well as utilities like axis functions etc)
I think that it would help if the Observable team just stood up and said that explicitly. Maybe even say on the d3 README:
> If you are a software engineer, bear in mind that things have moved on from 8 years ago and that you may in fact be best served by using utilities from d3 but using React or Vue for your DOM building.
Also, I think that the Observable team did themselves no favours by their slightly evangelical insistance on encouraging people to try Observable because they'll find that they like it once they try. I am a software engineer. I am open minded. I have tried Jupyter many times and I have tried Observable. I cannot use those tools for one simple reason: I use git to navigate my code state. If I am debugging, I make small commits, and use stash and revert and cherry-pick etc to experiment with different code states. I think in git and I work in git. Observable (and Jupyter) are completely antithetical to Git. So I also think that the Observable team would do better to say:
> Look. If you're a professional software engineer who is used to working in a sophiisticated fashion with version control, then honestly this stuff might not be your cup of tea. I'm sure you can see what we're trying to do, and that it's really cool, but we understand if you just want to enter lines of code into a git repo using your text editor. And for you, here's a quick summary/pointer to docs on how to use d3 in that way, with Vue/React, and some pointers on how to make use of the Observable docs for non-Observable users.
Mike Bostock if there's any chance you read this, d3 is one of the most impressive open source libraries I've known and I have been a fan of yours for years! Observable looks amazing and very forward-thinking too and I think I understand that it's aiming to democratize javascript visualization. I hope the above comments came across as constructive. (It would be awesome to have more pointers in the future on ways to make use of portions of the d3 API with other DOM-building APIs such as Vue/React.)
Recently I needed a simple chart for my startup. I looked online and all I could find was bloated libraries that were increasing the bundle size by many KBs. But then they also started rendering in the browser. Considering that it was possible to cache the request for a full day for all users, I wrote a small library that renders an svg on the server: https://github.com/rugpullindex/svg-line-chart
Chart in action: https://rugpullindex.com
Chart in action: https://rugpullindex.com
I understand the will to bring people to your new project(Observable) using something succesful that you already made(D3), but i think this move will be detrimental to both projects, since everything fundamental for the newcomers is being moved to Observable (Github docs, examples, ...), making the process of learning and integrating the project way more difficult and full of pointless time wasting garbage. This is the exact opposite of open source culture, as an old contributor and advocate of D3 i'm quite pissed. At the same time the big contributors have obvioulsy the right to steer the project in the direction they deem best.
This thread is literally people complaining about the free software used to host free documentation for a free library.
The onus is absolutely not on anyone to improve things for you just because you find them difficult.
Consider how lucky you are that d3 is even documented so well to begin with.
Insane level of entitlement in this thread.
The onus is absolutely not on anyone to improve things for you just because you find them difficult.
Consider how lucky you are that d3 is even documented so well to begin with.
Insane level of entitlement in this thread.
This all seems like exactly the behaviour you would expect if having D3 users is not profitable and having Observable users is profitable. This would explain why the Observable team's explanations don't entirely make sense: the real explanation is "we are making things worse for you because we hope to make money by doing so". It would explain why they're unmoved by users' complaints that something allegedly meant to make things easier for them is making them harder: the real goal is profit, not making things easier.
On the other hand, (1) using Observable is free (at present, at least), and (2) in other respects mbostock has always seemed genuinely interested in providing good things for everyone to use. So I hope the cynical view in the previous paragraph is wrong. But it does seem like it explains what's happening better than "we just want to make things better for everyone, and Observable is so awesome we can't help using it for everything" which is pretty much the official line.