Obsidian Dataview: Turn Obsidian Vault into a database you can query from(github.com)
github.com
Obsidian Dataview: Turn Obsidian Vault into a database you can query from
https://github.com/blacksmithgu/obsidian-dataview
77 comments
Minimal theme w/ Dataview card layout is awesome! I'm just digging into it, but it does make the queries more accessible. Thanks!
Hierarchical notetakers/outliners are becoming a new data structure for end users, specialized in knowledge management. A kind of generalized, expandable spreadsheet.
I expect these tools to include more and more tools for automation based on a reactive programming style (more suited to non-programmers), expanding its function as a warehouse of ideas into a more active role for user-created applications. The NOCODE camp is growing from its VB6-like visual-style roots into environments that look more like an IDE in the style of the browser's "Inspect" window.
The key development that I miss is that of reifying query results, allowing the user to use the data generated by a search as an object in itself. I even have a name for that reified object - a wit would be a collection of data compiled with a common criteria (representing a minimum unit of meaning, is a wordplay on bit being the minimum unit of knowledge), similar to a spreadsheet range. When tools like Obsidian or Logseq start allowing the user to use query results as first-class data, I predict that we will see a revolution in end-user process automation.
I expect these tools to include more and more tools for automation based on a reactive programming style (more suited to non-programmers), expanding its function as a warehouse of ideas into a more active role for user-created applications. The NOCODE camp is growing from its VB6-like visual-style roots into environments that look more like an IDE in the style of the browser's "Inspect" window.
The key development that I miss is that of reifying query results, allowing the user to use the data generated by a search as an object in itself. I even have a name for that reified object - a wit would be a collection of data compiled with a common criteria (representing a minimum unit of meaning, is a wordplay on bit being the minimum unit of knowledge), similar to a spreadsheet range. When tools like Obsidian or Logseq start allowing the user to use query results as first-class data, I predict that we will see a revolution in end-user process automation.
I agree with your issue of reification and I like the concept of a "wit."
I think we're getting there, too. See, for instance, https://github.com/nothingislost/obsidian-query-control — it renders the results of an embedded Obsidian query and you can interact with checkboxes. Likewise, a collaboration between dataview and the Task Collector plugin might let us manipulate different task statuses from a dataview query: https://github.com/ebullient/obsidian-task-collector/issues/...
Exciting times!
I think we're getting there, too. See, for instance, https://github.com/nothingislost/obsidian-query-control — it renders the results of an embedded Obsidian query and you can interact with checkboxes. Likewise, a collaboration between dataview and the Task Collector plugin might let us manipulate different task statuses from a dataview query: https://github.com/ebullient/obsidian-task-collector/issues/...
Exciting times!
> I agree with your issue of reification and I like the concept of a "wit."
Thanks! I had it as a vague idea for several years, though it has been getting a lot more concrete as I've seen new tools adopting similar approaches - but without any of them taking the final step of converting a search result collection into a user-accessible object ready for direct manipulation.
And now that I see tools adopting that style of data management, the concept allows me to see the commonalities among tools with quite different approaches - be it tabular Nocode storage like Airtable or outliners like Roam, Workflowy and the like. Whenever the tool returns a collection from a data gathering process, I see a wit that should be able to be extracted and further processed by the user.
BTW, the main difference between a query result and a wit is that you can store its items in the same structure as the rest of your content in the outliner, and send them to a new process (like spreadsheet cells, but with hierarchical data). Usually, query results are presented through a different interface than the rest of content, but a 'wit' should be homoiconic like Lisp.
(Errata to my grandparent post: a 'bit' is a "minimum unit of data", of course, not of knowledge).
> Exciting times!
You bet! :-D
Thanks! I had it as a vague idea for several years, though it has been getting a lot more concrete as I've seen new tools adopting similar approaches - but without any of them taking the final step of converting a search result collection into a user-accessible object ready for direct manipulation.
And now that I see tools adopting that style of data management, the concept allows me to see the commonalities among tools with quite different approaches - be it tabular Nocode storage like Airtable or outliners like Roam, Workflowy and the like. Whenever the tool returns a collection from a data gathering process, I see a wit that should be able to be extracted and further processed by the user.
BTW, the main difference between a query result and a wit is that you can store its items in the same structure as the rest of your content in the outliner, and send them to a new process (like spreadsheet cells, but with hierarchical data). Usually, query results are presented through a different interface than the rest of content, but a 'wit' should be homoiconic like Lisp.
(Errata to my grandparent post: a 'bit' is a "minimum unit of data", of course, not of knowledge).
> Exciting times!
You bet! :-D
Where has this tool been my whole life? :P Thanks for the link, I didn't know that one.
Yes, this https://youtu.be/Pot9GnHFOVU is the kind of programmable system I'm talking about, creating behavior on top of your stored content with methods specific to the particular collections of data already in your system, and ideally with minimal syntax for transferring data among the different tools.
Yes, this https://youtu.be/Pot9GnHFOVU is the kind of programmable system I'm talking about, creating behavior on top of your stored content with methods specific to the particular collections of data already in your system, and ideally with minimal syntax for transferring data among the different tools.
Super happy this landed, I had the same kind of reaction as you :-)
I’m using obsidian, but I’d move back to OneNote (or even Apple Notes) if it offered inclusions and more flexible tag management and reporting. I feel like MS could do that somewhere down the line.
Can't stand the idea of my personal notes being in the form of a proprietary format though. That's the key advantage for me.
> Hierarchical notetakers/outliners are becoming a new data structure for end users, specialized in knowledge management.
Outliners are used in this area for decades. What we have now is just a new hype-cycle, with new apps, new technologies and a new mix of features.
> When tools like Obsidian or Logseq start allowing the user to use query results as first-class data, I predict that we will see a revolution in end-user process automation.
So, SQL?
Outliners are used in this area for decades. What we have now is just a new hype-cycle, with new apps, new technologies and a new mix of features.
> When tools like Obsidian or Logseq start allowing the user to use query results as first-class data, I predict that we will see a revolution in end-user process automation.
So, SQL?
> What we have now is just a new hype-cycle, with new apps, new technologies and a new mix of features
You contradict yourself. Is it "just a new hype-cycle," or is it "new apps, new technologies, and a new mix of features?" If it's the latter... isn't that substantively different than what we had before?
Outliners have quite the fandom, but my issue with the traditional outliner is that they often trap data in outline form. Sure, most good outliners offer export options. Unfortunately, the second you've taken data from your outliner and exported it, you now have two versions of your data, and you've lost.
Logseq (and Obsidian) have changed this. Finally, a single document _is_ an outline, a backlink, a data source, a webpage, a presentation, and so on... The only thing the user has to do is work with that single page of information.
It's more than hype for me.
You contradict yourself. Is it "just a new hype-cycle," or is it "new apps, new technologies, and a new mix of features?" If it's the latter... isn't that substantively different than what we had before?
Outliners have quite the fandom, but my issue with the traditional outliner is that they often trap data in outline form. Sure, most good outliners offer export options. Unfortunately, the second you've taken data from your outliner and exported it, you now have two versions of your data, and you've lost.
Logseq (and Obsidian) have changed this. Finally, a single document _is_ an outline, a backlink, a data source, a webpage, a presentation, and so on... The only thing the user has to do is work with that single page of information.
It's more than hype for me.
> You contradict yourself. Is it "just a new hype-cycle," or is it "new apps, new technologies, and a new mix of features?"
The concepts are old and have a new hype. The apps are new. But generally, it doesn't matter much for the user whether you have some old-school desktop-app with gtk or Qt, or a "modern" electron app, or some webapp on your own server or some service in the cloud. At the end it's all the same ideas and concepts in different colors and flavors.
> If it's the latter... isn't that substantively different than what we had before?
That entirely depends on who is "we". We which are all users? Or we the old users? Or we the young users? Or we the new users? I'm not saying that there is not development or change, just that this is nothing new, or even a new development. This has going for a long time, it just reached a new cycle and and a new generation of users.
> Outliners have quite the fandom, but my issue with the traditional outliner is that they often trap data in outline form. Sure, most good outliners offer export options. Unfortunately, the second you've taken data from your outliner and exported it, you now have two versions of your data, and you've lost.
This is a fundamental problem of every app and space. But interoperability does exist here and there, it's not just exports.
> Logseq (and Obsidian) have changed this.
Not really? They only established a common foundation for interoperability. Which is also what other apps are doing to some degree. But you still have limitations by everything which is not covered by this foundation. For example, can Logseq handle dataview-syntax? Or generally any obsidian-plugin has it's own special syntax? What about other markdown-editors? How much do they support the special syntax used by obsidian or it's plugins?
True, the markdown-space has now a healthy movement toward interoperability and gains many nice functionalities. But it's not like this is especially new. And all those features also existed before Markdown in some app, in some way or another. Nothing is really new here. What's happening is that a new platform is growing, collection nice stuff from the collective pool of abilities, and a new consolidation is happening. This happens all the time. Sometimes it works, sometimes it dies.
Funny enough, the WWW was one of the first and most successful of this movements. And markdown grew from this as a simplification, to now grew into the same space where WWW started and obsidian now moves toward too, just in a different flavor. This whole "new" movement is just replicating where others were already before in different cloths.
> Finally, a single document _is_ an outline, a backlink, a data source, a webpage, a presentation, and so on...
So has Org-mode 10 years ago, XML 20 Years ago. HTML 30 years ago. More or less, the numbers may be a bit off.
> It's more than hype for me.
Good for you. I'm not question that. Just saying this generation is not the first one having that.
The concepts are old and have a new hype. The apps are new. But generally, it doesn't matter much for the user whether you have some old-school desktop-app with gtk or Qt, or a "modern" electron app, or some webapp on your own server or some service in the cloud. At the end it's all the same ideas and concepts in different colors and flavors.
> If it's the latter... isn't that substantively different than what we had before?
That entirely depends on who is "we". We which are all users? Or we the old users? Or we the young users? Or we the new users? I'm not saying that there is not development or change, just that this is nothing new, or even a new development. This has going for a long time, it just reached a new cycle and and a new generation of users.
> Outliners have quite the fandom, but my issue with the traditional outliner is that they often trap data in outline form. Sure, most good outliners offer export options. Unfortunately, the second you've taken data from your outliner and exported it, you now have two versions of your data, and you've lost.
This is a fundamental problem of every app and space. But interoperability does exist here and there, it's not just exports.
> Logseq (and Obsidian) have changed this.
Not really? They only established a common foundation for interoperability. Which is also what other apps are doing to some degree. But you still have limitations by everything which is not covered by this foundation. For example, can Logseq handle dataview-syntax? Or generally any obsidian-plugin has it's own special syntax? What about other markdown-editors? How much do they support the special syntax used by obsidian or it's plugins?
True, the markdown-space has now a healthy movement toward interoperability and gains many nice functionalities. But it's not like this is especially new. And all those features also existed before Markdown in some app, in some way or another. Nothing is really new here. What's happening is that a new platform is growing, collection nice stuff from the collective pool of abilities, and a new consolidation is happening. This happens all the time. Sometimes it works, sometimes it dies.
Funny enough, the WWW was one of the first and most successful of this movements. And markdown grew from this as a simplification, to now grew into the same space where WWW started and obsidian now moves toward too, just in a different flavor. This whole "new" movement is just replicating where others were already before in different cloths.
> Finally, a single document _is_ an outline, a backlink, a data source, a webpage, a presentation, and so on...
So has Org-mode 10 years ago, XML 20 Years ago. HTML 30 years ago. More or less, the numbers may be a bit off.
> It's more than hype for me.
Good for you. I'm not question that. Just saying this generation is not the first one having that.
I'm afraid you're entirely missing the relevance of the core fact that the syntax and runtime execution of programming languages are humongously difficult to understand for the large majority of computer users. Every time you create a system to automate some part of computer usage that doesn't rely on complex syntax and which executes in a declarative way, the same end result is created through a new concept, even if there existed a different process before that could also create it.
HTML was a step in the right direction (and XML a misstep); before it, you needed to program a GUI API to lay out content intermixed with interactive components. Saying that the web brought nothing new because BBSs existed before it is missing out on the importance of the details in achieving that result. Precisely, "different cloths" is what makes the movement relevant to make it accessible to new users who could never have used the previous system.
You're right in pointing out that interoperability is a key factor in the new batch of tools. The key point that neither Org-mode nor SGML-derived codes had is is to make user-entered content accessible to transformation tools without the need to code the transformation in an imperative language. Spreadsheets had this feature as well, sure, but spreadsheets didn't easily allow for hierarchical content.
The main new aspect that logseq and Obsidian bring to the table is the possibility to backlink to any part of the hierarchical structure. The tools automatically compile all those links as a new data object, which the user can process without creating a script to gather all that data. HTML had the potential to popularize that feature 30 years ago (with a slightly more awkward syntax than Markdown), but browsers never got around to it.
Previous tools would require users to move content to a different tool (say, by copy-pasting the content of your files to a spreadsheet to apply formulas), destroying the possibility of having a central personal knowledge repository. In logseq / Obsidian, you can keep adding multimedia content to your structured knowledge base, and exploit that content without coding at all, or at most by creating simple declarative expressions. What tool before these ones had that possibility before? With hierarchical structures, not just flat structures like the spreadsheet?
HTML was a step in the right direction (and XML a misstep); before it, you needed to program a GUI API to lay out content intermixed with interactive components. Saying that the web brought nothing new because BBSs existed before it is missing out on the importance of the details in achieving that result. Precisely, "different cloths" is what makes the movement relevant to make it accessible to new users who could never have used the previous system.
You're right in pointing out that interoperability is a key factor in the new batch of tools. The key point that neither Org-mode nor SGML-derived codes had is is to make user-entered content accessible to transformation tools without the need to code the transformation in an imperative language. Spreadsheets had this feature as well, sure, but spreadsheets didn't easily allow for hierarchical content.
The main new aspect that logseq and Obsidian bring to the table is the possibility to backlink to any part of the hierarchical structure. The tools automatically compile all those links as a new data object, which the user can process without creating a script to gather all that data. HTML had the potential to popularize that feature 30 years ago (with a slightly more awkward syntax than Markdown), but browsers never got around to it.
Previous tools would require users to move content to a different tool (say, by copy-pasting the content of your files to a spreadsheet to apply formulas), destroying the possibility of having a central personal knowledge repository. In logseq / Obsidian, you can keep adding multimedia content to your structured knowledge base, and exploit that content without coding at all, or at most by creating simple declarative expressions. What tool before these ones had that possibility before? With hierarchical structures, not just flat structures like the spreadsheet?
> I'm afraid you're entirely missing the relevance of the core fact that the syntax and runtime execution of programming languages are humongously difficult to understand for the large majority of computer users.
No, I know that, I just don't think it's a relevant problem for the user group we are talking about.
> Saying that the web brought nothing new because BBSs existed before
I did not say that? And in the first place, BBS competes with other networks like the Internet, not a service running in such a network.
> The main new aspect that logseq and Obsidian bring to the table is the possibility to backlink to any part of the hierarchical structure.
No, they do not. Linking is a fundamental concept of hypertext. And easy linking inside your dataspace, as also automatically listing backlinks, was already established with wikis 20+ years ago. Even hierarchies exists in wiki space for a long time now in several different ways.
> The tools automatically compile all those links as a new data object, which the user can process without creating a script to gather all that data.
What kind of low-level-tools have you used till now that this is your level of knowledge here??
> In logseq / Obsidian, you can keep adding multimedia content to your structured knowledge base, and exploit that content without coding at all
Yes, because someone else does the coding, as always. This is the benefit of a popular ecosystem. But this is not the point of having a scriptable and programmable app. The scripting just enable you to personalize on small levels. The programmability should enable the user and community to extend the program with new abilities, as dataview here is doing it. There are other extensions who enable scripting to certain degrees for the user, as obsidian is not really adding this on it's own.
No app will ever be perfect for everyone. That's why extensions and scripting are beneficial, especially for this space.k
> What tool before these ones had that possibility before? With hierarchical structures, not just flat structures like the spreadsheet?
The constraints again. Why does it matter that obsidian&co. can do some more tricks than their ancestors, like playing videos, when the fundamental purpose and handling is still the same? That's like the old joke of taking something old and just add a clock, to sell it as new. More bells and whistle don't make a new church.
No, I know that, I just don't think it's a relevant problem for the user group we are talking about.
> Saying that the web brought nothing new because BBSs existed before
I did not say that? And in the first place, BBS competes with other networks like the Internet, not a service running in such a network.
> The main new aspect that logseq and Obsidian bring to the table is the possibility to backlink to any part of the hierarchical structure.
No, they do not. Linking is a fundamental concept of hypertext. And easy linking inside your dataspace, as also automatically listing backlinks, was already established with wikis 20+ years ago. Even hierarchies exists in wiki space for a long time now in several different ways.
> The tools automatically compile all those links as a new data object, which the user can process without creating a script to gather all that data.
What kind of low-level-tools have you used till now that this is your level of knowledge here??
> In logseq / Obsidian, you can keep adding multimedia content to your structured knowledge base, and exploit that content without coding at all
Yes, because someone else does the coding, as always. This is the benefit of a popular ecosystem. But this is not the point of having a scriptable and programmable app. The scripting just enable you to personalize on small levels. The programmability should enable the user and community to extend the program with new abilities, as dataview here is doing it. There are other extensions who enable scripting to certain degrees for the user, as obsidian is not really adding this on it's own.
No app will ever be perfect for everyone. That's why extensions and scripting are beneficial, especially for this space.k
> What tool before these ones had that possibility before? With hierarchical structures, not just flat structures like the spreadsheet?
The constraints again. Why does it matter that obsidian&co. can do some more tricks than their ancestors, like playing videos, when the fundamental purpose and handling is still the same? That's like the old joke of taking something old and just add a clock, to sell it as new. More bells and whistle don't make a new church.
> No, I know that, I just don't think it's a relevant problem for the user group we are talking about.
What user group are you talking about? I'm specifically talking about the large majority of users who could never learn to program in a syntax-heavy general-purpose programming language! X-D Those people nevertheless have a need to build their own userland applications, and no current tool serves them well; but a Nocode programmable knowledge database could do it.
> No, they do not. Linking is a fundamental concept of hypertext. And easy linking inside your dataspace, as also automatically listing backlinks, was already established with wikis 20+ years ago.
And as I explained, no end-user tool has exploited that possibility to allow end users to access those automated links programatically with nocode tools (like for example with spreadsheet-like declarative expressions). The browser missed the chance by having no persistent storage and only allowing javascript as their programming language, and wikis never were nocode-friendly. Yet modern tools are just beginning to do that on top of a personal knowledge base.
> What kind of low-level-tools have you used till now that this is your level of knowledge here??
I beg you pardon? I've programmed all the way down from building my own flip-flops with electric transistors up to non-linear mathematical solvers for combinatorial optimization. I don't see the relevance, since low-level-tools will never be adequate development platforms for end-users.
> Yes, because someone else does the coding, as always.
No. Logseq / Obsidian have tools that allow end users to build their own coding through templates and queries. The kind of coding is different from scripting, though; it's more like creating semi-automated workflows that put all relevant information in front of the user, which then proceeds to transform it with direct manipulation. What tool in the past has allowed end-users to create that kind of workflows from end-user templates and arbitrary queries, other than non-hierarchical spreadsheets?
> The constraints again. Why does it matter that obsidian&co. can do some more tricks than their ancestors
The constraints are there for a reason, and you don't seem to grasp why they are important. The spreadsheet , and spreadsheets are typically not automated through scripting and extensions - using those is normally considered a failure of the spreadsheet model, because you're processing data outside the mental model of the spreadsheet as a programming environment i.e. cheating. Most End-User Development tools rely on the assumption that users can't program at all, so all automation concepts must be presented to them through alternate syntax and interactions, quite different from classic coding.
A spreadsheet-like development environment based on outliners as its data structure, rather than the tables of a spreadsheet, should also be programmable by end users without relying on imperative scripts external to the tool. Logseq is creating the basis for that storage model, which some people are starting to exploit the kind of semi-automated workflows I talked about above.
> like playing videos
That's a red-herring, not a main point. I mentioned multimedia merely because programmable tools tend to be text only, forcing users to use separate tools for rich content, or import specific libraries to handle multimedia data as programmable objects (rather than editing them as native interactive content like they would on the GUI). The point is that end users should be able to treat all kinds of content equally with their tools, not being tied to the Unix shell constraint that all content is moved between processes as text streams or binary blobs.
What user group are you talking about? I'm specifically talking about the large majority of users who could never learn to program in a syntax-heavy general-purpose programming language! X-D Those people nevertheless have a need to build their own userland applications, and no current tool serves them well; but a Nocode programmable knowledge database could do it.
> No, they do not. Linking is a fundamental concept of hypertext. And easy linking inside your dataspace, as also automatically listing backlinks, was already established with wikis 20+ years ago.
And as I explained, no end-user tool has exploited that possibility to allow end users to access those automated links programatically with nocode tools (like for example with spreadsheet-like declarative expressions). The browser missed the chance by having no persistent storage and only allowing javascript as their programming language, and wikis never were nocode-friendly. Yet modern tools are just beginning to do that on top of a personal knowledge base.
> What kind of low-level-tools have you used till now that this is your level of knowledge here??
I beg you pardon? I've programmed all the way down from building my own flip-flops with electric transistors up to non-linear mathematical solvers for combinatorial optimization. I don't see the relevance, since low-level-tools will never be adequate development platforms for end-users.
> Yes, because someone else does the coding, as always.
No. Logseq / Obsidian have tools that allow end users to build their own coding through templates and queries. The kind of coding is different from scripting, though; it's more like creating semi-automated workflows that put all relevant information in front of the user, which then proceeds to transform it with direct manipulation. What tool in the past has allowed end-users to create that kind of workflows from end-user templates and arbitrary queries, other than non-hierarchical spreadsheets?
> The constraints again. Why does it matter that obsidian&co. can do some more tricks than their ancestors
The constraints are there for a reason, and you don't seem to grasp why they are important. The spreadsheet , and spreadsheets are typically not automated through scripting and extensions - using those is normally considered a failure of the spreadsheet model, because you're processing data outside the mental model of the spreadsheet as a programming environment i.e. cheating. Most End-User Development tools rely on the assumption that users can't program at all, so all automation concepts must be presented to them through alternate syntax and interactions, quite different from classic coding.
A spreadsheet-like development environment based on outliners as its data structure, rather than the tables of a spreadsheet, should also be programmable by end users without relying on imperative scripts external to the tool. Logseq is creating the basis for that storage model, which some people are starting to exploit the kind of semi-automated workflows I talked about above.
> like playing videos
That's a red-herring, not a main point. I mentioned multimedia merely because programmable tools tend to be text only, forcing users to use separate tools for rich content, or import specific libraries to handle multimedia data as programmable objects (rather than editing them as native interactive content like they would on the GUI). The point is that end users should be able to treat all kinds of content equally with their tools, not being tied to the Unix shell constraint that all content is moved between processes as text streams or binary blobs.
> What user group are you talking about?
The kind of people who use such tools. The kind of people who can lookup DSLs to create dataviews, or copy&paste snippets to make up stuff. Something like a knowledge-system is not used by children or someone similar.
> Those people nevertheless have a need to build their own userland applications,
No, they do not. Not every user around the world has this demand. Most user don't have this demand. We are talking here about a minority of quite special usecases. Most people are not even able to mange their own files, and you mean they will use a complex knowledge-managment-system? People are even challenged with a simple todo-list or note-app. No way will they use something more complex.
> and no current tool serves them well; but a Nocode programmable knowledge database could do it.
No, it's not. Nocode is not magic. You still need a certain level of competence, understanding and motivation to make something with it. And whatever interface you present to the users, be it GUI or text or whatever, there always will be a certain level of complexity that not any random user can master. And this is good, we already see with apple and google what harm we have from too much simplification to cater to the uninteressted.
> No. Logseq / Obsidian have tools that allow end users to build their own coding through templates and queries. The kind of coding is different from scripting, though; it's more like creating semi-automated workflows that put all relevant information in front of the user, which then proceeds to transform it with direct manipulation. What tool in the past has allowed end-users to create that kind of workflows from end-user templates and arbitrary queries, other than non-hierarchical spreadsheets?
Many? What? This is also a kind of scripting, and there is an endless amount of tools enabling people to do that style of scripting. Starting with shellscripts, batchscripts, all the strange scripting-engines. Heck, emacs and has this style as their main purpose established. The data science-sector has a very long history of building such tools on a more professional level and integrating data-views. Wikis and databases have a long history of scripting in the knowledge managment-space. The likes of obsidian are just the next iteration, there are not fundamentally new.
> and spreadsheets are typically not automated through scripting and extensions - using those is normally considered a failure of the spreadsheet model
No, they are? What? People have build all kind of crap with VBA for a very long time. There is a reason why this garbage is still around, getting ported to everywhere. And let's not start with the plugins or how extensions for MS Office are called these days.
> using those is normally considered a failure of the spreadsheet model, because you're processing data outside the mental model of the spreadsheet as a programming environment
Maybe that's our bubbles understanding. Seems the rest of the world doesn't give much here. Limiting yourself to a model is nonsens anyway. No model is ever perfect and satisfying every user.
> That's a red-herring, not a main point. I mentioned multimedia merely because programmable tools tend to be text only, forcing users to use separate tools for rich content.
It's not like obsidian is doing more with them, just because it can play them. Until it understand them can make them accessable for the rest of the system it's remains some fancy gadget.
The kind of people who use such tools. The kind of people who can lookup DSLs to create dataviews, or copy&paste snippets to make up stuff. Something like a knowledge-system is not used by children or someone similar.
> Those people nevertheless have a need to build their own userland applications,
No, they do not. Not every user around the world has this demand. Most user don't have this demand. We are talking here about a minority of quite special usecases. Most people are not even able to mange their own files, and you mean they will use a complex knowledge-managment-system? People are even challenged with a simple todo-list or note-app. No way will they use something more complex.
> and no current tool serves them well; but a Nocode programmable knowledge database could do it.
No, it's not. Nocode is not magic. You still need a certain level of competence, understanding and motivation to make something with it. And whatever interface you present to the users, be it GUI or text or whatever, there always will be a certain level of complexity that not any random user can master. And this is good, we already see with apple and google what harm we have from too much simplification to cater to the uninteressted.
> No. Logseq / Obsidian have tools that allow end users to build their own coding through templates and queries. The kind of coding is different from scripting, though; it's more like creating semi-automated workflows that put all relevant information in front of the user, which then proceeds to transform it with direct manipulation. What tool in the past has allowed end-users to create that kind of workflows from end-user templates and arbitrary queries, other than non-hierarchical spreadsheets?
Many? What? This is also a kind of scripting, and there is an endless amount of tools enabling people to do that style of scripting. Starting with shellscripts, batchscripts, all the strange scripting-engines. Heck, emacs and has this style as their main purpose established. The data science-sector has a very long history of building such tools on a more professional level and integrating data-views. Wikis and databases have a long history of scripting in the knowledge managment-space. The likes of obsidian are just the next iteration, there are not fundamentally new.
> and spreadsheets are typically not automated through scripting and extensions - using those is normally considered a failure of the spreadsheet model
No, they are? What? People have build all kind of crap with VBA for a very long time. There is a reason why this garbage is still around, getting ported to everywhere. And let's not start with the plugins or how extensions for MS Office are called these days.
> using those is normally considered a failure of the spreadsheet model, because you're processing data outside the mental model of the spreadsheet as a programming environment
Maybe that's our bubbles understanding. Seems the rest of the world doesn't give much here. Limiting yourself to a model is nonsens anyway. No model is ever perfect and satisfying every user.
> That's a red-herring, not a main point. I mentioned multimedia merely because programmable tools tend to be text only, forcing users to use separate tools for rich content.
It's not like obsidian is doing more with them, just because it can play them. Until it understand them can make them accessable for the rest of the system it's remains some fancy gadget.
So, we're talking about entirely different groups of users then, there's no wonder that we're talking past each other, and that you're missing everything innovative about this new line of tools.
> No, they do not. Not every user around the world has this demand. Most user don't have this demand. We are talking here about a minority of quite special usecases. Most people are not even able to mange their own files, and you mean they will use a complex knowledge-management-system?
There I have to strongly disagree. There's a whole academic field of study, End-User Development, dedicated to prove that idea wrong. Every computer user I know is able to use spreadsheets to create their own data types tailored for their domain-specific use cases, some times quite complex ones. That they can't use scripting languages is a failure of those languages, not of users.
And yes, users are able to handle complex knowledge management systems when they're built from the ground up with the right models, instead of relying on arcane legacy metaphors from the 1960s, such as hierarchical filesystems - which are good for developers, not so much for end users. Our knowledge about digital content organization has evolved a lot since those early pioneer times.
With the proper tools, of course they'll be able to create new software artifacts, which of course they want and need - the same way they want and use new apps built by others. If they could create those themselves, why wouldn't they build their own?
> Many? What? This is also a kind of scripting, and there is an endless amount of tools enabling people to do that style of scripting.
True, but you're missing out the possibility of enabling people to create automation without any scripting at all. There aren't many tools for that other than spreadsheets, which have bad integration with the rest of the computing environment.
>Wikis and databases have a long history of scripting in the knowledge managment-space. The likes of obsidian are just the next iteration, there are not fundamentally new.
I recommend you to learn about End-User Development,[1] to see all the ways these tools can be expanded beyond classic scripting, to see what you're missing out, that other people in this thread are aware and you're not. The end-user computing landscape is going very soon to be transformed, flooded with the kind of tools that end users can build for themselves, for the very same reason that the "VBA and MS Office plugins garbage" are still around; just this time, we finally have a chance to create a way better platform for those tools, which doesn't rely on outdated plugins and low-powered scripting languages.
"Watch what I do"[2] and "Your wish is my command" [3] are the seminal works with loots of good examples. They're a bit dated, but some of their best methods have still not found their way to practical common-use tools (but modern outliners could change that very soon).
[1] https://en.wikipedia.org/wiki/End-user_development
[2] https://archive.org/details/watchwhatido00alle/page/n5/mode/...
[3] https://web.media.mit.edu/~lieber/PBE/Your-Wish/
> No, they do not. Not every user around the world has this demand. Most user don't have this demand. We are talking here about a minority of quite special usecases. Most people are not even able to mange their own files, and you mean they will use a complex knowledge-management-system?
There I have to strongly disagree. There's a whole academic field of study, End-User Development, dedicated to prove that idea wrong. Every computer user I know is able to use spreadsheets to create their own data types tailored for their domain-specific use cases, some times quite complex ones. That they can't use scripting languages is a failure of those languages, not of users.
And yes, users are able to handle complex knowledge management systems when they're built from the ground up with the right models, instead of relying on arcane legacy metaphors from the 1960s, such as hierarchical filesystems - which are good for developers, not so much for end users. Our knowledge about digital content organization has evolved a lot since those early pioneer times.
With the proper tools, of course they'll be able to create new software artifacts, which of course they want and need - the same way they want and use new apps built by others. If they could create those themselves, why wouldn't they build their own?
> Many? What? This is also a kind of scripting, and there is an endless amount of tools enabling people to do that style of scripting.
True, but you're missing out the possibility of enabling people to create automation without any scripting at all. There aren't many tools for that other than spreadsheets, which have bad integration with the rest of the computing environment.
>Wikis and databases have a long history of scripting in the knowledge managment-space. The likes of obsidian are just the next iteration, there are not fundamentally new.
I recommend you to learn about End-User Development,[1] to see all the ways these tools can be expanded beyond classic scripting, to see what you're missing out, that other people in this thread are aware and you're not. The end-user computing landscape is going very soon to be transformed, flooded with the kind of tools that end users can build for themselves, for the very same reason that the "VBA and MS Office plugins garbage" are still around; just this time, we finally have a chance to create a way better platform for those tools, which doesn't rely on outdated plugins and low-powered scripting languages.
"Watch what I do"[2] and "Your wish is my command" [3] are the seminal works with loots of good examples. They're a bit dated, but some of their best methods have still not found their way to practical common-use tools (but modern outliners could change that very soon).
[1] https://en.wikipedia.org/wiki/End-user_development
[2] https://archive.org/details/watchwhatido00alle/page/n5/mode/...
[3] https://web.media.mit.edu/~lieber/PBE/Your-Wish/
> Outliners are used in this area for decades. What we have now is just a new hype-cycle, with new apps, new technologies and a new mix of features.
To say that what we have now is nothing but a new hype cycle while the Zettelkasten method was only widely exposed within the last couple of decades, and was not utilized in the design of past Outliners, shows you don't really understand what's actually changed.
To say that what we have now is nothing but a new hype cycle while the Zettelkasten method was only widely exposed within the last couple of decades, and was not utilized in the design of past Outliners, shows you don't really understand what's actually changed.
Zettelkasten is nothing new and just hype. Only shows that you just don't know this space, and it's history. Obviously there is improvement, but it's not in methods or data-structures, it's with technology and implementations. We are now at the point where we can move fast with implementations and glue much more features together.
Then please, illuminate me. Because the implications of Luhmann's use of the Zettelkasten weren't widely available till the last decade, and weren't available in software until the same period.
So what exactly don't I know? Genuinely.
So what exactly don't I know? Genuinely.
Luhmann methods are known since the 80s, at least in europe. And I think I saw the first digital implementation around 2006 or 2008. Looking it up, the first(?) Digital Zettelkasten after Luhmann method was in 2004 already at version 2.5, so it's likely quite older. But dates are not important here.
Luhmanns method is simply not that special, nor unique. Many scientists and artist used similar methods and tools. He is mostly just replicating what was common at the time, but with some twist. And in the 80s or 90s is would be really awesome, probably even somewhat groundbreaking. But today? It's just another graph-system which you find in wikis or hypertext for decades now. And most people are not even using the benefits of his number-system, just calling everything zettelkasten which is just a wiki otherwise, leaving even less of his originality. Though, in digital space it make less sense anyway, so not a big lose.
But for whatever reason, there now this hype from people who seem to have know knowledge about those space or it's history, or even really understand the terms, thinking Luhmanns method is called Zettelkasten...
Luhmanns method is simply not that special, nor unique. Many scientists and artist used similar methods and tools. He is mostly just replicating what was common at the time, but with some twist. And in the 80s or 90s is would be really awesome, probably even somewhat groundbreaking. But today? It's just another graph-system which you find in wikis or hypertext for decades now. And most people are not even using the benefits of his number-system, just calling everything zettelkasten which is just a wiki otherwise, leaving even less of his originality. Though, in digital space it make less sense anyway, so not a big lose.
But for whatever reason, there now this hype from people who seem to have know knowledge about those space or it's history, or even really understand the terms, thinking Luhmanns method is called Zettelkasten...
> Outliners are used in this area for decades.
What outliners do you know which allow end-users to feed their data into formulas for processing it without using general-purpose programming languages? I'm genuinely interested, as I've been exploring these tools for years and I haven't found any that meets my requirements.
> So, SQL
How many end-users without any programming experience are able to create moderately complex SQL queries?
With a data-centric outliner tool, they'll be able to decompose those into several simple graphical queries.
What outliners do you know which allow end-users to feed their data into formulas for processing it without using general-purpose programming languages? I'm genuinely interested, as I've been exploring these tools for years and I haven't found any that meets my requirements.
> So, SQL
How many end-users without any programming experience are able to create moderately complex SQL queries?
With a data-centric outliner tool, they'll be able to decompose those into several simple graphical queries.
> What outliners do you know which allow end-users to feed their data into formulas for processing it without using general-purpose programming languages?
Bit of a pointless constraint, the talk is about outliners, not no-code-datamangment. Which tool today does this even offer on a useful level?
But you can look at leo editor (https://leoeditor.com), which is active for 20+ years, fully scriptable and extendable. Though, it's a hot piece of garbage for laymen. It's offers a bunch of features and plugins even for non-coders, but I'm not sure it would satisfy you for this area, if you can't code. Obsidian is probably better than, even if it has more limitations at the moment.
But I'm not sure if there ever is a tool which will satisfy everyone with just a no-code-approach.
Bit of a pointless constraint, the talk is about outliners, not no-code-datamangment. Which tool today does this even offer on a useful level?
But you can look at leo editor (https://leoeditor.com), which is active for 20+ years, fully scriptable and extendable. Though, it's a hot piece of garbage for laymen. It's offers a bunch of features and plugins even for non-coders, but I'm not sure it would satisfy you for this area, if you can't code. Obsidian is probably better than, even if it has more limitations at the moment.
But I'm not sure if there ever is a tool which will satisfy everyone with just a no-code-approach.
> Bit of a pointless constraint, the talk is about outliners, not no-code-datamangment.
The point I'm making is that those are both aspects of the same substrate, and modern knowledge tools are exploring the gap between those two - as well as expanding them in new directions.
Yes, certainly Leo is one of the closest tools on which to build a system like that, although it still relies too much on Python programming (and its usability leaves a lot to be desired, even if the concepts are solid). I expect that the next popular tool in that space will grow either from online notebooks like Jupyter or from desktop linked note-takers like logseq.
(BTW I can code, but I don't want to; remembering the syntax details of the needed APIs does not come natural to me, so it's usually not worth the effort). I still believe that it is possible to grow a programming tool similar to Excel but with structured hierarchical content, without requiring coding more complex than spreadsheet formulas and filters.
The point I'm making is that those are both aspects of the same substrate, and modern knowledge tools are exploring the gap between those two - as well as expanding them in new directions.
Yes, certainly Leo is one of the closest tools on which to build a system like that, although it still relies too much on Python programming (and its usability leaves a lot to be desired, even if the concepts are solid). I expect that the next popular tool in that space will grow either from online notebooks like Jupyter or from desktop linked note-takers like logseq.
(BTW I can code, but I don't want to; remembering the syntax details of the needed APIs does not come natural to me, so it's usually not worth the effort). I still believe that it is possible to grow a programming tool similar to Excel but with structured hierarchical content, without requiring coding more complex than spreadsheet formulas and filters.
> and modern knowledge tools are exploring the gap between those two - as well as expanding them in new directions.
Do they? As I see it, they make it worse. We have all kind of specialized languages. Each function coming with it's own limited DSL and API. Often with poor documentation. Is this actually better?
Yes, each language has on it's own less complexity, but all in all they have more problems and limitations than a proper general purpose language with a good API. I don't think that this is really a good future. But it's true that it might lower the bar for new people.
> BTW I can code, but I don't want to; remembering the syntax details of the needed APIs does not come natural to me
So the problem is not the language, but the support? More autocompletion, documentation, automation, etc. That's a concern of mine too. Things are complex, but there is no support for the casual users. Actually I think if obsidian had been build ontop of VS Code and gained instant support from all the fancy developer-tools, it would gain significant benefits.
Another solution could be integrating an interface like Node-RED offers it. This seems to be quite accessible for casual experts.
> I still believe that it is possible to grow a programming tool similar to Excel
And Excel has good access for new users?
Do they? As I see it, they make it worse. We have all kind of specialized languages. Each function coming with it's own limited DSL and API. Often with poor documentation. Is this actually better?
Yes, each language has on it's own less complexity, but all in all they have more problems and limitations than a proper general purpose language with a good API. I don't think that this is really a good future. But it's true that it might lower the bar for new people.
> BTW I can code, but I don't want to; remembering the syntax details of the needed APIs does not come natural to me
So the problem is not the language, but the support? More autocompletion, documentation, automation, etc. That's a concern of mine too. Things are complex, but there is no support for the casual users. Actually I think if obsidian had been build ontop of VS Code and gained instant support from all the fancy developer-tools, it would gain significant benefits.
Another solution could be integrating an interface like Node-RED offers it. This seems to be quite accessible for casual experts.
> I still believe that it is possible to grow a programming tool similar to Excel
And Excel has good access for new users?
> Do they? As I see it, they make it worse.
That may be because your development needs are well served already if you're a developer, so you won't see the value in such system. For people who rely on others to build every single piece of automation they need, being able to create their own small automatisms would be priceless.
> all in all they have more problems and limitations than a proper general purpose language with a good API
End users don't need a general purpose language with a good API - at least not as someting that they will handle directly. They need small, self-contained domain-specific languages that they can expand in areas where they fall short, but without the full power of a general programming system (which would overcomplicate the development environment and provide too many opportunities to shoot themselves in the foot).
> So the problem is not the language, but the support?
For me yes, partially, because I already know programming. I could work with a programming environment with complex syntax that autocompleted the required syntax to match my thoughts (something like GitHub copilot works in that direction). But that's still two points of complexity above the system I envision as possible.
Something like the classic Hypercard could be a good starting point for the mental model of such tools. A simple data model, integrated storage, and interactive processes using a programming language with minimal syntax. Expand it with hierarchical storage, expanded integration with other systems and a modern declarative language, and you're set.
Someone today pointed me to a system like this[1], Lepiter, which looks a lot like the solution end users could manage to create on their own on top of their knowledge systems: like a browser with separate cells, where each cell can run a different runtime, hyperlinked and with a simple storage mechanism.
[1] https://lepiter.io/feenk/introducing-lepiter--knowledge-mana...
> More autocompletion, documentation, automation, etc.
Those are good things but not enough. What end users are lacking the most is the possibility to create their own automated processes; most tools only allow them to choose among a library of predefined ones, without allowing them to grow the library.
> Another solution could be integrating an interface like Node-RED offers it. This seems to be quite accessible for casual experts.
Unfortunately, graph-based flow visual tools have proven to be useful only for a small subset of automation tasks largely consisting on data transformations using pre-defined functions. The most promising results in End-User Development point out that reactive, rule-based runtime environments like the spreadsheet seem to be better abstractions for allowing users to define their own computations.
> And Excel has good access for new users?
Yes, spreadsheets have a low learning curve and high ceiling. And compared to imperative programming languages, they're like walking up the slope of your house vs. extreme rock climbing.
That may be because your development needs are well served already if you're a developer, so you won't see the value in such system. For people who rely on others to build every single piece of automation they need, being able to create their own small automatisms would be priceless.
> all in all they have more problems and limitations than a proper general purpose language with a good API
End users don't need a general purpose language with a good API - at least not as someting that they will handle directly. They need small, self-contained domain-specific languages that they can expand in areas where they fall short, but without the full power of a general programming system (which would overcomplicate the development environment and provide too many opportunities to shoot themselves in the foot).
> So the problem is not the language, but the support?
For me yes, partially, because I already know programming. I could work with a programming environment with complex syntax that autocompleted the required syntax to match my thoughts (something like GitHub copilot works in that direction). But that's still two points of complexity above the system I envision as possible.
Something like the classic Hypercard could be a good starting point for the mental model of such tools. A simple data model, integrated storage, and interactive processes using a programming language with minimal syntax. Expand it with hierarchical storage, expanded integration with other systems and a modern declarative language, and you're set.
Someone today pointed me to a system like this[1], Lepiter, which looks a lot like the solution end users could manage to create on their own on top of their knowledge systems: like a browser with separate cells, where each cell can run a different runtime, hyperlinked and with a simple storage mechanism.
[1] https://lepiter.io/feenk/introducing-lepiter--knowledge-mana...
> More autocompletion, documentation, automation, etc.
Those are good things but not enough. What end users are lacking the most is the possibility to create their own automated processes; most tools only allow them to choose among a library of predefined ones, without allowing them to grow the library.
> Another solution could be integrating an interface like Node-RED offers it. This seems to be quite accessible for casual experts.
Unfortunately, graph-based flow visual tools have proven to be useful only for a small subset of automation tasks largely consisting on data transformations using pre-defined functions. The most promising results in End-User Development point out that reactive, rule-based runtime environments like the spreadsheet seem to be better abstractions for allowing users to define their own computations.
> And Excel has good access for new users?
Yes, spreadsheets have a low learning curve and high ceiling. And compared to imperative programming languages, they're like walking up the slope of your house vs. extreme rock climbing.
OmniOutliner has had some kind of formulas for a very long time, however they work like an actual spreadsheet, where each outline row can have additional number and formula columns.
I don't think it's all that useful, outside of some basic "making a shopping/packing/project list" type calculations. It's worth checking out if you're interested in the history though.
I don't think it's all that useful, outside of some basic "making a shopping/packing/project list" type calculations. It's worth checking out if you're interested in the history though.
Thanks, I'll take a look at it.
Whoa, very weird to see my plugin hit HN! Dataview was initially borne as "Notion tables for Obsidian", but quickly became a way to do flexible data annotation + querying after I realized how powerful metadata can be. Viewing your notes as "text + data that can easily be accessed" can really alter how you think about note taking and knowledge management.
There is a fair bit left to do for the plugin (export to raw Markdown, section-level metadata), but I hope you find it useful!
There is a fair bit left to do for the plugin (export to raw Markdown, section-level metadata), but I hope you find it useful!
thank you for your work on Dataview! I use it as a read later list in combination with a Mac Shortcut I wrote + https://github.com/hauselin/obsidian-alfred
It has completely replaced Pocket/Instapaper for me.
It has completely replaced Pocket/Instapaper for me.
If you like this idea, but prefer something more self-contained, Tiddlywiki is an excellent option, as it runs in-browser from a single HTML file. Its core approach is to treat notes as a queryable database, and its markup supports dynamic content based on queries of that DB. Because its UI is constructed from the same DB, the same techniques that allow you to construct a dynamic view from the DB also allow you to construct custom UIs for that view, including changing properties/attributes of items in the DB.
That's all pretty abstract. A simple example: it allows you to construct a tabbed view of several todo lists based on tags and fields and adjust priority and status from that single UI.
It's been available for many years, but has never been trendy, despite its unique design and strengths. I recommend giving it a spin!
That's all pretty abstract. A simple example: it allows you to construct a tabbed view of several todo lists based on tags and fields and adjust priority and status from that single UI.
It's been available for many years, but has never been trendy, despite its unique design and strengths. I recommend giving it a spin!
> It's been available for many years, but has never been trendy, despite its unique design and strengths. I recommend giving it a spin!
I was a heavy TiddlyWiki user in the past but moved away from it precisely because of its singe-HTML file design. Modern browsers like chrome made it virtually impossible to save an edited page. Not TW's fault per se, but ruined my experience with it. There is a way to install TW-5 with node.js as a directory tree but my ship has sailed. Now I am a happy obsidian user.
I was a heavy TiddlyWiki user in the past but moved away from it precisely because of its singe-HTML file design. Modern browsers like chrome made it virtually impossible to save an edited page. Not TW's fault per se, but ruined my experience with it. There is a way to install TW-5 with node.js as a directory tree but my ship has sailed. Now I am a happy obsidian user.
Yes, that aspect of the design makes it feel like a toy, which is a shame. I've written my own tooling for saving to bridge this gap. It's a Python script that provides a response to an OPTIONS call so Tiddlywiki's native WebDAV support allows seamless saving. It also provides automatic backups and an index page to serve multiple wikis. I've used it for my last two jobs and in my personal life and I'm really happy with it. I hope to make it available publicly in case others find it useful, just haven't gotten around to it.
I use tiddlywiki (with the nodejs setup) regularly. It's great until your wiki grows too large, at which point you need to wait ~10s for it to load even with modern hardware.
I'd be very curious to hear about the sizes and use cases where you're seeing this. My understanding was that single file wikis could scale to a hundred megabytes and beyond, and I thought the nodejs version was more scalable than that due to lazy loading tiddlers. My wikis tend to be less than 20mb since I mostly use plain text for everything.
I do make use of the `backlinks` query extensively, and I use saqimtiaz's wonderful `streams` [1] plugin to recreate a Roam-like experience. This setup leads to a large amount of small-sized tiddlers (2.5k and counting), which might add to the overhead.
Not to mention downloading and parsing 20mb-worth of html+js each time you open up the wiki can be quite slow.
As for lazy loading --- when enabled, it breaks the indexing (e.g. finding backlinks) and searching because these are done by the js inside the browser too. I only lazy-load images and binaries because they are not indexable anyway.
[1]: https://github.com/saqimtiaz/streams
Not to mention downloading and parsing 20mb-worth of html+js each time you open up the wiki can be quite slow.
As for lazy loading --- when enabled, it breaks the indexing (e.g. finding backlinks) and searching because these are done by the js inside the browser too. I only lazy-load images and binaries because they are not indexable anyway.
[1]: https://github.com/saqimtiaz/streams
Can Tiddlywiki be used by a team with simultaneous edits?
Not at all, AFAIK. Completely single-user in my experience.
You can. But you need to host it with a nodejs server.
Logseq allows query via datomic: https://discuss.logseq.com/t/advanced-queries-and-the-databa...
Minor correction, I don't think Datomic is used. I believe it uses a variation of DataScript and in turn DataLog for queries. Whereas Datomic is a DB which is also based on and uses DataLog queries.
Either way, super powerful.
Refs: https://en.m.wikipedia.org/wiki/Datalog https://github.com/tonsky/datascript https://www.datomic.com/
Either way, super powerful.
Refs: https://en.m.wikipedia.org/wiki/Datalog https://github.com/tonsky/datascript https://www.datomic.com/
Quite right, thanks for the corrections and links!
It's great, but I wish there was some more interesting stuff you could do with the results of that query, other than show an outline or table.
The inline programmability of DataView and the Grid views etc are pretty killer. Unfortunately Obsidian is not for me, so I hope Logseq catches up.
The inline programmability of DataView and the Grid views etc are pretty killer. Unfortunately Obsidian is not for me, so I hope Logseq catches up.
Obsidian is amazing. Dataview is transformative, but it's not alone; the Excalidraw plugin is also a game-changer in its own right, enabling notes in drawings and drawings in notes... just incredibly powerful for more visually-oriented users.
Just a shameless suggestion: if anyone is concerned about the security implications of using the Obsidian Sync paid addon, I’ve been having good results with Syncthing instead. It supports peer-to-peer sync and is FOSS.
I've done this too, but on iOS Syncthing doesn't work so well. Resorted to running a Windows VM as a bridge between iCloud and Syncthing.
I have just used iCloud for everything, and it works great, even on Windows (note that you cannot use the Microsoft Store app, which is garbage).
I guess I fit solidly inside the 5GB free tier and probably will for quite a while.
I would use Dropbox but the app feels quite heavy and it wasn’t very easy to set up on iPhone.
I guess I fit solidly inside the 5GB free tier and probably will for quite a while.
I would use Dropbox but the app feels quite heavy and it wasn’t very easy to set up on iPhone.
Sadly no iCloud client for Linux though, so that's where Syncthing comes in!
what's wrong with git?
Works on your phone and works in real time as opposed to needing to make a commit every time you change a note.
What's wrong with syncthing?
I've been using it for a few months and it's pretty cool. It allows me to do stuff like:
* list all unfinished tasks grouped by some custom metadata ( project, etc.)
* have note templates which contains a dynamic query for the history of the same type of note for historical context
* use it in conjunction with Obsidian charts to make custom charts ( e.g. number of meetings with tag/metadata/mention of X)
* list all unfinished tasks grouped by some custom metadata ( project, etc.)
* have note templates which contains a dynamic query for the history of the same type of note for historical context
* use it in conjunction with Obsidian charts to make custom charts ( e.g. number of meetings with tag/metadata/mention of X)
How would you manage tasks with this?
Currently, I'm using Evernote and its new Tasks feature is very good, even though it could be better.
I cannot move from Evernote to Obsidian if I don't solve the issue of managing tasks effectively from with Obsidian, as I currently do with Emacs.
Currently, I'm using Evernote and its new Tasks feature is very good, even though it could be better.
I cannot move from Evernote to Obsidian if I don't solve the issue of managing tasks effectively from with Obsidian, as I currently do with Emacs.
I moved from Emacs org-mode to Obsidian with the help of the obsidian-tasks plugin, which lets you write queries for tasks similarly to Dataview. I used a few of those queries to build up an Agenda view that matched my custom agenda buffer in org-mode. I put together a little write up here: https://davi.sh/blog/2022/01/obsidian-one/
Great!
I use the Tasks plugin for this. Tasks are created by creating markdown lists. You can put tags in with hashtags and due dates in with (and this bit I dislike, even though there's a command for it) the calendar emoji followed by a date.
E.g. (note HN strips out emoji)
E.g.
There are other plugins you can use, like Dataview which is the subject of this post (but means each Todo needs to be its own note), or the Todoist plugin, but Tasks is the one which works for me.
E.g. (note HN strips out emoji)
- [] Write documentation for Foobar #docs #personxyz (cal emoji) 2022-05-22
E.g.
```tasks
not done
description does not include #deferred
due before today
path does not include Templates
```
There are other plugins you can use, like Dataview which is the subject of this post (but means each Todo needs to be its own note), or the Todoist plugin, but Tasks is the one which works for me.
I add a tag, which filters it into a table. Pretty rudimentary but it meets my needs.
So many great features, but all these systems are missing the one true feature I really need, Link types. I see that Dataview gives some ability to have them, but it requires a very specific format that is pretty distinct from how the notes would be taken.
can you elaborate? How the ideal implementation would look like.
My use case would be more simple than others, but I am not super well versed in the actual current syntax of Obsidian. I like the idea of this syntax https://forum.obsidian.md/t/add-support-for-link-types/6994/... as it would be most flexible. But a simple [[Page Name]]{type:child} could be good, all I really need is a means of filtering linked documents by their link type (in a graph view/visualization) and ideally a method to query and display those. Dataview gives the second, though the first seems to require the link type to be a first class citizen.
I understand that it is a complex thing though, and not something easy to just throw in
I understand that it is a complex thing though, and not something easy to just throw in
you can use the supercharged links plugin and careful control of it's class types to achieve that, it's how I did it. I'm largely using schema.org types
Obsidian is quickly shaping up to be the best thing ever to happen to note taking in a long time. I don't know how I could live without it!
I wish it were FOSS, or had a roadmap to become FOSS once the whole world already owns a licensed copy, like everyone should, but even though I'm normally a major FOSS enthusiast, I'm just happy this software exists at all, in any form.
I will definitely be making use of this dataview extension!
I wish it were FOSS, or had a roadmap to become FOSS once the whole world already owns a licensed copy, like everyone should, but even though I'm normally a major FOSS enthusiast, I'm just happy this software exists at all, in any form.
I will definitely be making use of this dataview extension!
Honestly this is one of the handful of plugins I hate trying to live without.
Tag wrangler is the other.
Tag wrangler is the other.
Before Obsidian there was ConntextedText [1]. Few people knew it even back then, but it is quite powerful, packed with common wiki features like linking, attributes, querying (also display results as a table), transclusion, page graph visualisation, python integration, etc.
[1]: https://www.connectedtext.com/
[1]: https://www.connectedtext.com/
Dataview is pretty nice, but last time I played with it, I missed a bit interactivity. By which I mean working with the results on the fly, like sorting them inplace, selecting individual entries and move them to new results. Is this possible now, or in progress?
You can sort Dataview results using this plugin: https://github.com/alexandru-dinu/obsidian-sortable
Ah, thanks. Makes sense that other extensions will also work for this.
This one is still in the manual-install phase, but you might like this plugin: https://github.com/tomaszkiewicz/obsidian-database-plugin
This looks really nice.
Is there a timeline when it will be installable through obsidian?
should be pretty soon tbh, code review is almost done
Moving dataview entries between data views is fundamentally at odds with the paradigm in which dataview operates. The point is that it's ephemeral data that's entirely derived from the state of your notes.
Maybe there are some basic queries that could support "two-way binding" of this sort, but it would be fiendishly complex and ultimately better solved by a different tool altogether
Maybe there are some basic queries that could support "two-way binding" of this sort, but it would be fiendishly complex and ultimately better solved by a different tool altogether
> Moving dataview entries between data views is fundamentally at odds with the paradigm in which dataview operates.
It doesn't need to be between dataviews. Just temporary filter the view, or export it in a regular static table. This is just about enhancing the View, not changing the concept.
Which opens the question, can dataview also take tables as data-source?
> Maybe there are some basic queries that could support "two-way binding" of this sort
I'm not talking about Two-way-bindings. Though this would be awesome. And I don't think it would be that hard to implement in general, depening on how exactly obsidian works internally with dataview.
It doesn't need to be between dataviews. Just temporary filter the view, or export it in a regular static table. This is just about enhancing the View, not changing the concept.
Which opens the question, can dataview also take tables as data-source?
> Maybe there are some basic queries that could support "two-way binding" of this sort
I'm not talking about Two-way-bindings. Though this would be awesome. And I don't think it would be that hard to implement in general, depening on how exactly obsidian works internally with dataview.
Thats a really cool idea
Next we’ll do email!
I sometimes wonder if learning how to use Dataview inside of Obsidian is too technical to get mass adoption, compared to WYSIWYG tools like Notion that let you build databases with nice UI filtering. But the performance of Dataview is so much better, and the control you have over queries so much more granular, that it’s worth the learning curve. The Obsidian community is also incredibly good at helping new users learn, which makes me feel optimistic.
To help make things a bit more user-friendly I have been building an Obsidian theme called Minimal[1] which allows you to take Dataview tables and display them in a card layout[2]. I really enjoy that layout for certain types of information, while having all the underlying power of Dataview.
[1]: https://github.com/kepano/obsidian-minimal
[2]: https://minimal.guide/Block+types/Cards