Rete: JavaScript framework for visual programming and creating node editor(github.com)
github.com
Rete: JavaScript framework for visual programming and creating node editor
https://github.com/retejs/rete
61 comments
I am developing a generic data flow programming tool, it is only view and the model or engine can be added on top of it.
If you want to use it I am more then happy to help: https://github.com/fibo/flow-view
If you want to use it I am more then happy to help: https://github.com/fibo/flow-view
Do you have a working online example? The links don't seem to work (404 or blank), and I'd like to see whether I should try it if I at some time get to write a dataflow graph config tool sitting on top of timely-dataflow. Especially because I expect to use it for a sizable event processing graph.
Thank you for the feedback, I restored the online example here: http://g14n.info/flow-view/examples/basic/
I am also working on an engine that is able to run javascript code: https://github.com/fibo/dflow
Currently I am working on a rewrite and I will use flow-view to create custom nodes, for example nodes that have a canvas or a graph inside (images a SinOsc node with a graphic preview). The engine is able to order nodes by level and run them usinng the graph order, it is not an event based engine but every graph compiles a JS function.
I am also working on an engine that is able to run javascript code: https://github.com/fibo/dflow
Currently I am working on a rewrite and I will use flow-view to create custom nodes, for example nodes that have a canvas or a graph inside (images a SinOsc node with a graphic preview). The engine is able to order nodes by level and run them usinng the graph order, it is not an event based engine but every graph compiles a JS function.
Not sure if it fits the bill exactly but did you look at VCV Rack?
https://vcvrack.com/
https://vcvrack.com/
I've downloaded vcvrack, but not actually tried it out yet. It looks awesome. However, the project I'm working on is a web-based synth using the HTML5 Web Audio API. It's built using Tone.js and NexusUI.js, so vcvrack isn't really applicable, as far as I can tell at least!
Thanks for the recommendation though. :)
Thanks for the recommendation though. :)
I was going to recommend VCV Rack as well :-)
Sounds cool! Will the project be open-source by chance?
Relatedly, what are your thoughts on Tone.js?
https://tonejs.github.io/
Relatedly, what are your thoughts on Tone.js?
https://tonejs.github.io/
Yes, it will be floss. I've still got a way to go to fully flesh it out, but I'm getting there. Here's a screenshot of what I have so far - https://i.imgur.com/bEBIBFF.png. The sequencer, effects rack to the right, keyboard, pad on the left, wave selection, oscilloscope, level meter, master volume and bottom two dial racks (dry/wet levels) are all working. The rest is yet to be plumbed in, but I've added it in to try and get some idea of the final layout.
At present, the sequence of connections for each effect are pre-determined, but I'm working with the idea that a button will flip the synth around so you can patch the units in any order you wish. Another idea is to start with an empty rack and have you add them in and move them around however you wish too, so you could have more than one of each type in the chain. Very much still a work-in-progress!
I'm actually using Tone.js for this project. So far I'm finding it very useful.
At present, the sequence of connections for each effect are pre-determined, but I'm working with the idea that a button will flip the synth around so you can patch the units in any order you wish. Another idea is to start with an empty rack and have you add them in and move them around however you wish too, so you could have more than one of each type in the chain. Very much still a work-in-progress!
I'm actually using Tone.js for this project. So far I'm finding it very useful.
I've used jsPlmb (years ago) and it worked well.
Neat looking project but maybe the naming is off because of possible confusion with Rete Networks (one of the coolest computer science algorithms and data structures to optimize expert system rules engines for efficiently handling very large numbers of production rules).
[deleted]
Apparently nothing to do with the Rete algorithm (fwiw). [1]
[1] https://en.wikipedia.org/wiki/Rete_algorithm
[1] https://en.wikipedia.org/wiki/Rete_algorithm
Looks really nice! Looking forward to playing with it a bit.
Shameless plug: I built something similar as a part of my workflow platform for JVM (for Java and Clojure):
https://github.com/mikub/titanoboa
The main focus of my tool is enterprise integration and data processing, but I still love to play and experiment with the ideas of developing code directly in browser and how to visualize its execution flow. So I built a live coding / visualization also as a part of my tool. This becomes non-trivial if you apply it to languages other then js (so browser integration becomes a bit harder).
I also think the main challenge here is the server-side part of things, especially when you make such solution distributed - i.e. how you achieve high availability and assure everything gets executed (at least) once even in case of failover, how you cater for multitenancy while achieving sufficient security/isolation between users etc. edit:spelling/formatting
Shameless plug: I built something similar as a part of my workflow platform for JVM (for Java and Clojure):
https://github.com/mikub/titanoboa
The main focus of my tool is enterprise integration and data processing, but I still love to play and experiment with the ideas of developing code directly in browser and how to visualize its execution flow. So I built a live coding / visualization also as a part of my tool. This becomes non-trivial if you apply it to languages other then js (so browser integration becomes a bit harder).
I also think the main challenge here is the server-side part of things, especially when you make such solution distributed - i.e. how you achieve high availability and assure everything gets executed (at least) once even in case of failover, how you cater for multitenancy while achieving sufficient security/isolation between users etc. edit:spelling/formatting
See also: https://github.com/jagenjo/litegraph.js
man, growing up with Maya and Houdini, making a node editor and dataflow engine was probably my ultimate dream
Shameless shill:
We've been building a set of tools to make software development more visual, kind of like this. There are some examples on our site here, https://refinery.io
More specifically, you still write code and you visually connect pieces of code together. Think "pipelines of transformations" in a functional programming way. Then you click "deploy" and have it all running on AWS Lambda with rich logging and other goodies.
A lot my inspiration comes from being a former game dev who has used Scratch and UE4's Blueprint engine. I've personally found that it's difficult to get the abstraction right in a way that keeps complexity in check (as others in this thread have commented). That's why you still write code in Refinery -- you're just visually chaining microservice calls.
Would love to hear feedback + thoughts, if anybody is interested in sharing. free at refinery.io is a good email for me. :)
More specifically, you still write code and you visually connect pieces of code together. Think "pipelines of transformations" in a functional programming way. Then you click "deploy" and have it all running on AWS Lambda with rich logging and other goodies.
A lot my inspiration comes from being a former game dev who has used Scratch and UE4's Blueprint engine. I've personally found that it's difficult to get the abstraction right in a way that keeps complexity in check (as others in this thread have commented). That's why you still write code in Refinery -- you're just visually chaining microservice calls.
Would love to hear feedback + thoughts, if anybody is interested in sharing. free at refinery.io is a good email for me. :)
We started with trying to create something like this when we started Dark (https://darklang.com). We ultimately moved away from this style of programming, as we couldn't figure out a way to make it feel "code-y" enough for people to want to write programs in it.
I'm super interested in what folks want to use this for?
I'm super interested in what folks want to use this for?
Yeah I like the idea of visual programming but it's not gonna replace everyday dev work in my opinion.
I would like to see more apps for tablets that introduce visual programming into everyday workflows. If you haven't seen the work of Ink & Switch check out these essays on end user programming [0] and an app for basically working on ideas [1], combine these and you get a fluid interface with an easy way of doing some computation (the visual programming bit) which is something I would love to use.
But visual programming gets messy if you attempt to do something complex and I don't know if there is any way around this. This then is where there needs to be programs with strong types that can compose with each other, like the example explained in this post [2].
These concepts are all things which anyone using a computer could use, not something just resticted to people who are already "developers", and I think we have to get into that mindset for this type of software to become more successful.
[0] https://www.inkandswitch.com/end-user-programming.html
[1] https://www.inkandswitch.com/muse-studio-for-ideas.html
[2] https://pchiusano.github.io/2013-09-10/type-systems-and-ux-e...
I would like to see more apps for tablets that introduce visual programming into everyday workflows. If you haven't seen the work of Ink & Switch check out these essays on end user programming [0] and an app for basically working on ideas [1], combine these and you get a fluid interface with an easy way of doing some computation (the visual programming bit) which is something I would love to use.
But visual programming gets messy if you attempt to do something complex and I don't know if there is any way around this. This then is where there needs to be programs with strong types that can compose with each other, like the example explained in this post [2].
These concepts are all things which anyone using a computer could use, not something just resticted to people who are already "developers", and I think we have to get into that mindset for this type of software to become more successful.
[0] https://www.inkandswitch.com/end-user-programming.html
[1] https://www.inkandswitch.com/muse-studio-for-ideas.html
[2] https://pchiusano.github.io/2013-09-10/type-systems-and-ux-e...
In theory one could structure any program this way right? In practice I've seen it work mostly for "creative programming", like in Max/MSP [1] or Unity3d [2]. NiFi [3] seems to be successful too, but kinda niche (and looks very daunting for general, simple problems).
I like the idea of having programs structured as "producer" and "consumer" nodes, with nodes being as close to pure functions as possible. But it seems like the graphical representation indeed gets in the way.
Perhaps the idea could still work without the visualization part? Each node a single file, listing inputs (maybe some sinks too) at the top. Still a graph, but users wouldn't _necesarily_ need to visualize it all the time.
1: https://cycling74.com/products/max
2: https://unity.com/shader-graph
3: https://nifi.apache.org/
I like the idea of having programs structured as "producer" and "consumer" nodes, with nodes being as close to pure functions as possible. But it seems like the graphical representation indeed gets in the way.
Perhaps the idea could still work without the visualization part? Each node a single file, listing inputs (maybe some sinks too) at the top. Still a graph, but users wouldn't _necesarily_ need to visualize it all the time.
1: https://cycling74.com/products/max
2: https://unity.com/shader-graph
3: https://nifi.apache.org/
Creator or litegraph here.
Visual programming works very well when you have data flowing, but it doesnt work when you have loops and branching. But it is nice to support graphs for some tasks, it really speeds up some processes.
Visual programming works very well when you have data flowing, but it doesnt work when you have loops and branching. But it is nice to support graphs for some tasks, it really speeds up some processes.
>but it doesnt work when you have loops and branching
We managed to make visual programming work well with loops [1] and more or less OK with branching [2]. The trick is to look at loops from a functional perspective (i.e. as iter/map), rather than from imperative.
[1] https://easymorph.com/learn/iterations.html
[2] https://easymorph.com/learn/conditional-workflows.html
We managed to make visual programming work well with loops [1] and more or less OK with branching [2]. The trick is to look at loops from a functional perspective (i.e. as iter/map), rather than from imperative.
[1] https://easymorph.com/learn/iterations.html
[2] https://easymorph.com/learn/conditional-workflows.html
I think the ultimate test is whether it can be self hosted. eg. can the program be used to create the program itself!? Or what problem does it solve?!?
Both loops and branching/dispatching are examples of control flow, which is essentially independent from the 'data flow' default of most "visual programming" approaches. You can actually show both in the same visualization, but only in a well-defined "sub-diagram" sense. That is, a "block" in a data-flow visualization can be defined by a flowchart (showing control flow), and vice versa.
The dual problem is that flowcharts (in the traditional, control-flow-only sense) cannot show fork-join parallelism or fixed point constructions, while both are readily expressible by data flow.
The dual problem is that flowcharts (in the traditional, control-flow-only sense) cannot show fork-join parallelism or fixed point constructions, while both are readily expressible by data flow.
I like PraxisLive[1] because it combines both code and nodes. Reconnecting nodes in different ways it's ideal for experimenting with different implementations or trying out different orders when applying filters (which works both for audio and graphics effects). It also facilitates changing values in real time without stopping the program, which is perfect when developing time-based software.
[1] https://praxislive.org/
[1] https://praxislive.org/
I also think visual coding works for high level abstractions rather than low-level procedural style workflows. E.g. for ML a visual flow could be import some data, train a model, deploy model to cloud. Orange Data Mining is an example of visual programming at the right level of abstraction but stops short of including enough modules to be useful in common data science workflows:
https://orange.biolab.si/
https://orange.biolab.si/
The general approach seems to work well for NodeRed and Unity? I think it's good to get to a point where we can use layered infrastructure to experiment more easily with this paradigm. I have tried to build something for mobile, partially inspired by Roocky's Boots, but I guess my assumption that drawing would be more accessible for mobile than typing didn't really hold (see flowgrid.org if interested).
Does that mean the Dark in Action on the homepage is outdated? Or is the UI something that carried over from the original design? I know there are "code-y" elements in that video and all the auto-complete features that are demoed are fantastic.
No, it's pretty current. The "code" in Dark is textual (it's not bytes on disk, but it's very far from nodes and edges). Dark is intended to be very code-y - although we're not fully there just yet coding in Dark mostly much feels like coding (discounting having to use a different editor).
Related: Node-RED - Low-code programming for event-driven applications
https://nodered.org/
https://nodered.org/
There's also http://nodes.io/ which is in alpha release state the last thing I heard.
Owner of this project should check out visual programming in Unreal Engine 4 it’s completely production ready and has shipped in many AAA titles.
I absolutely love the blueprint editor in UE4. It enables me to rapidly hammer out ideas and test new things. Not only that, but it helped me learn a lot about how to go about doing things in UE4 so when I started writing CPP I already had an understanding for where to put what and how to do it.
Easily the first visual programming language I used that actually sold me on the idea. Most of my experiences have been with systems that claim visual programming, but then require you to jump top code almost immediately. UE4 blueprints are sufficient to create the majority of a game.
WebbleWorld 3 is at once similar, and different: https://github.com/truemrwalker/wblwrld3
with several academic papers and a book about the concept. Professor Yuzuru Tanaka pioneered the concept of Meme Media.
Awesome project. Reminds me of my days of using Apache NiFi. https://nifi.apache.org/
I’d be curious if there was a node interface framework for React (Rete is Vue-oriented, which is not made obvious until you follows installation docs).
Rete has a react based component thing too[1], is that jus5 for custom components.
[1] https://github.com/retejs/react-render-plugin
[1] https://github.com/retejs/react-render-plugin
Sweet, I stand corrected!
It is increasingly important that we publish components in a framework/library agnostic manner. Not only do we have a lot of duplication of component libraries in general, but we also hit the landmines of "Angular-oriented", "Vue-oriented", "React-oriented"... Hopefully, we will ser more full fkedged component libraries built with Web Components, which can integrate with any framework or library.
You don't have to wait for web components though. If the authors of said libraries split it up in two parts, the main logic and framework/library integration, the sharing would be possible today already.
I think tools like xState gets this right, you can glance the package listing here: https://github.com/davidkpiano/xstate/tree/master/packages
Then you just include core + framework X integration, and if you need a new integration, you create it new and use it together with core.
I think tools like xState gets this right, you can glance the package listing here: https://github.com/davidkpiano/xstate/tree/master/packages
Then you just include core + framework X integration, and if you need a new integration, you create it new and use it together with core.
Have you tried Adafruit's make code? https://makecode.adafruit.com/ Visual programming for javascript with options to interact with their Circuit Playground hardware.
It's from Microsoft, with flavors for Mincraft and Legostorm too, https://www.microsoft.com/en-us/makecode .
It's, of course, not an entire ecosystem for everything, but they do a lot of visual things that may be worth evaluating. I like the colors/shapes of the code pieces, and the ability to switch between the visual code editor and the plain js.
It's from Microsoft, with flavors for Mincraft and Legostorm too, https://www.microsoft.com/en-us/makecode .
It's, of course, not an entire ecosystem for everything, but they do a lot of visual things that may be worth evaluating. I like the colors/shapes of the code pieces, and the ability to switch between the visual code editor and the plain js.
Really cool! Is there a way to export a data representation of your graph, like in json or something? Would there be a standard for that, like some control flow representation?
Node-red? Great visual workflow system based on MQTT, but you can also program UIs in it (node-red-dashboard), cluster process stuff with plugins, and even program this stuff on mobile (https://github.com/alexisread/noreml).
It can import/export json representations of the diagrams/code.
I've found mermaid (and vue-mermaid) useful for just the workflow representation as the diagrams are compact, though the lines don't curve like in other libraries
I've found mermaid (and vue-mermaid) useful for just the workflow representation as the diagrams are compact, though the lines don't curve like in other libraries
Looks like editor.toJSON() exports the current graph.
I tried it in the dev consolve, on this page: https://rete.js.org/#/components
The exported object has properties id (string) and nodes (array). Each node is shaped like:
I tried it in the dev consolve, on this page: https://rete.js.org/#/components
The exported object has properties id (string) and nodes (array). Each node is shaped like:
{
"id": 1,
"data": {
"name": "rete",
"importance": 1
},
"inputs": {
"pkg": {
"connections": [
{
"node": 2,
"output": "pkg",
"data": {
"pins": []
}
}
]
}
},
"outputs": {
"pkg": {
"connections": [
{
"node": 3,
"input": "pkg",
"data": {
"pins": [
{
"x": 306.4013957327592,
"y": -1661.933328730452
}
]
}
}
]
}
},
"position": [
-509.99078981156725,
-241.02053426307492
],
"name": "Package"
}Use BPM/BPMN? (Business process modelling)
Not answering you question directly, but it’s a way of doing something similar.
A whole set of enterprise tools do things like this, and I’ve worked a bit with bpm workflow engines that kind of does this in a standardized way.
> BPMN models are expressed by simple diagrams constructed from a limited set of graphical elements. For both business users and developers, they simplify understanding of business activities' flow and process. BPMN's four basic element categories are:
- Flow objects: Events, activities, gateways
- Connecting objects: Sequence flow, message flow, association
- Swim lanes: Pool, lane
- Artifacts: Data object, group, annotation
https://en.wikipedia.org/wiki/Business_Process_Model_and_Not...
You can use web tools to draw processes and, execute them on a compliant engine:
https://demo.bpmn.io/
Camunda comes to mind, as an open source alternative, but if nodejs is your thing there a a couple of BPM engines in the works, eg:
https://github.com/paed01/bpmn-engine
As with anything graphical based - if it doesn't fit the purpose exactly, it will become a pain in the end.
Not answering you question directly, but it’s a way of doing something similar.
A whole set of enterprise tools do things like this, and I’ve worked a bit with bpm workflow engines that kind of does this in a standardized way.
> BPMN models are expressed by simple diagrams constructed from a limited set of graphical elements. For both business users and developers, they simplify understanding of business activities' flow and process. BPMN's four basic element categories are:
- Flow objects: Events, activities, gateways
- Connecting objects: Sequence flow, message flow, association
- Swim lanes: Pool, lane
- Artifacts: Data object, group, annotation
https://en.wikipedia.org/wiki/Business_Process_Model_and_Not...
You can use web tools to draw processes and, execute them on a compliant engine:
https://demo.bpmn.io/
Camunda comes to mind, as an open source alternative, but if nodejs is your thing there a a couple of BPM engines in the works, eg:
https://github.com/paed01/bpmn-engine
As with anything graphical based - if it doesn't fit the purpose exactly, it will become a pain in the end.
I remember I had to do one project at my university. I hated it. I still wonder if there is a successful product out there which I made through bpm. I doubt it.
Same here, actually... well, not uni, but as a consultant and employee.
It’s “ok” if you own the whole lot, but I have yet to see a hand-over in an enterprise setting work out well, and bring speed and agility.
In many ways, for an enterprise, json and web api’s is a good enough abstraction to work with the machines.
Tangential to this - Netflix made Conductor as a kind of in-house workflow engine, and uber built Cadence, if you want more codified control.
Wheels keep getting re-invented! :)
Tangential to this - Netflix made Conductor as a kind of in-house workflow engine, and uber built Cadence, if you want more codified control.
Wheels keep getting re-invented! :)
I've had building something like this in the back of my mind for a while. Free idea (the reason I wanted this): use this to build an interface to crypto smart contracts (e.g. Ethereum). Payment processing seems similar to signal processing (which these node editors are good at, e.g. Max/MSP, Unity shader graph).
Anyone invest their time learning something like this and have it pay off? Interested in success stories I suppose.
(Edit - I mean something more like nodes.io)
(Edit - I mean something more like nodes.io)
NodeRed. I’m pretty good at it by now. We use it in our hackspace for home automation and at work for prototypes.
Link with video: https://nodered.org/
It's an IBM side-project built for IoT.
That's how it started, but it's now an independent project of the OpenJS Foundation. And it isn't limited to IoT - it can be used for any event driven application.
(Disclaimer: I'm the creator and project lead of Node-Red)
(Disclaimer: I'm the creator and project lead of Node-Red)
I made anither comment about the blueprint editor in UE4.
It helped me both rapidly prototype things and learn UE4 design patterns / practices. Some of the core functionality and perf related things I'll bang out in cpp but for most things I stick to blueprints.
It helped me both rapidly prototype things and learn UE4 design patterns / practices. Some of the core functionality and perf related things I'll bang out in cpp but for most things I stick to blueprints.
The only functionality I’m actually looking for is the ability to connect and disconnect elements using virtual cables (that flex and look good, just like those in rete). It’s for a software synth patchbay (like those used in the Reason digital audio workstation software). So, ultimately, even though this library is impressive, it isn’t quite the right fit for my project.
If anyone can recommend an alternative that focuses purely on cabling, I’d love to hear about it!