Roll your own JavaScript runtime, pt. 3(deno.com)
deno.com
Roll your own JavaScript runtime, pt. 3
https://deno.com/blog/roll-your-own-javascript-runtime-pt3
46 comments
roll your own runtime does not imply to me `import runtime from "runtime";` but this seems to just be "include deno and use deno"?
This is using Deno as the glue between Rust and V8; it leaves many things to the end-user, such as stdlib implementation. This isn't quite the same as building it from scratch, but it's still building a good chunk of the runtime I think.
[deleted]
When/why would you want to do this? I went to pt. 1 but it didn't seem clear.
The content of the series describes creating something like the beginnings of a Deno alternative, upon which the reader could fully recreate Deno or Node.js. It seems to me that the core idea presented is "a JavaScript/TypeScript interface to Rust". The thing that most interests me about something like this is not making yet another alternative to Deno or Node.js, but the potential for adding a scripting language to an application or framework written in Rust. I'm thinking like Python scripting of Blender, Lua as a scripting/modding layer for a game engine, scripting of Tiled with JS, an Electron alternative using GTK, your own browser.
When you want to add a sandboxed scripting language to any system, you might implement something like this. For example, a multi-tenant web proxy that lets users script functions for transforming requests, or a game that lets users script the user interface elements, would be two valid use cases.
sounds useful for sandboxing
There's hundreds of programs that can be scripted with embedded Lua. For example, nginx web server, or neovim editor or awesomewm window manager. Having an embedded scripting language gives these systems enormous flexibility & utility.
This series is effectively about doing the same, but embedded Deno instead of Lua. It's showing how to launch Deno runtime, and how to seed those runtime with various hooks to access & manipulate the program it's embedded in.
This series is effectively about doing the same, but embedded Deno instead of Lua. It's showing how to launch Deno runtime, and how to seed those runtime with various hooks to access & manipulate the program it's embedded in.
This is not any kind of "rolling your own JS runtime". This is "use our framework to use v8 from rust". What an obnoxious title.
v8 is a JS runtime. JSC is a JS runtime. SpiderMonkey is a JS runtime.
All of these are embeddable and have usable APIs. If all you are doing, is linking to a JS runtime, and then using it, you aren't "rolling your own runtime".
If you want to roll your own JS runtime go look at how the LibJS folk in serenity did it - they did it without corporate backing and despite that I believe LibJS is fairly complete even with the new draft language features, albeit lacking a decade or so of performance optimizations.
v8 is a JS runtime. JSC is a JS runtime. SpiderMonkey is a JS runtime.
All of these are embeddable and have usable APIs. If all you are doing, is linking to a JS runtime, and then using it, you aren't "rolling your own runtime".
If you want to roll your own JS runtime go look at how the LibJS folk in serenity did it - they did it without corporate backing and despite that I believe LibJS is fairly complete even with the new draft language features, albeit lacking a decade or so of performance optimizations.
Yeah unfortunately as pessimistic as it seems I must agree, this title is disappointing to me. I'd love to see a series discussing the ins and outs of even just tiny segments of implementing ECMA262, as there's quite a lot of interesting nuance that has come up over the years.
I guess the difference is runtime vs perhaps interpreter, but it's definitely ambiguous.
I guess the difference is runtime vs perhaps interpreter, but it's definitely ambiguous.
You might find this interesting: https://github.com/svaarala/duktape/blob/master/doc/compiler...
V8, JSC, etc have an interpreter (or JIT, or what have you), but they also include everything else: the GC, the standard library, the standard and core objects. e.g. the full runtime.
Adding a couple of functions to an existing runtime is not rolling your own runtime.
Adding a couple of functions to an existing runtime is not rolling your own runtime.
[deleted]
I expected some interesting content and left with a "import v8: Cargo edition" 'tutorial' is this really a good look for the ailing Deno?
Yes, this is more like "Roll your own JavaScript environment"
JS runtime != JS engine
Runtime: - Chrome - Deno - Node - Bun
Engine: - V8 - JSC - SpiderMonkey - LibJS
Runtime: - Chrome - Deno - Node - Bun
Engine: - V8 - JSC - SpiderMonkey - LibJS
Specifically, a runtime (IMO) is the set of variables and their associated functionality that exist in the global scope, or are expected to be importable in a predictable and standardized way.
Part 2 of this series, for instance, shows the instantiator of the V8 engine (via deno_core, a runtime-less wrapper as explained in Part 1) implementing a simplified fetch API with some trivial JS and the bulk of the logic in Rust: https://deno.com/blog/roll-your-own-javascript-runtime-pt2#i... - this would then be available to any code executing in their custom JS environment.
This is useful even outside of creating a full Node-style implementation; for instance, Cloudflare created a locked-down runtime (a reasonable subset of the browser runtime) for v8 isolates for their workers: https://developers.cloudflare.com/workers/runtime-apis/
It's super cool to know this stuff as it lets you consider (when appropriate!) using JS as a way to accept Turing-complete user-submitted logic rather than just accepting, say, JSON configs, while limiting the surface with which it can interact with your system.
Part 2 of this series, for instance, shows the instantiator of the V8 engine (via deno_core, a runtime-less wrapper as explained in Part 1) implementing a simplified fetch API with some trivial JS and the bulk of the logic in Rust: https://deno.com/blog/roll-your-own-javascript-runtime-pt2#i... - this would then be available to any code executing in their custom JS environment.
This is useful even outside of creating a full Node-style implementation; for instance, Cloudflare created a locked-down runtime (a reasonable subset of the browser runtime) for v8 isolates for their workers: https://developers.cloudflare.com/workers/runtime-apis/
It's super cool to know this stuff as it lets you consider (when appropriate!) using JS as a way to accept Turing-complete user-submitted logic rather than just accepting, say, JSON configs, while limiting the surface with which it can interact with your system.
> Specifically, a runtime (IMO) is the set of variables and their associated functionality that exist in the global scope, or are expected to be importable in a predictable and standardized way.
All of which is provided by V8 in this example. The global object, math object, "Object" in general, Arrays, etc are all the runtime. All this tutorial series on embedding V8 is doing is instantiating an existing runtime environment and adding using the APIs to insert a few new APIs.
All together you might say you've got a custom runtime as you have the baseline "JS runtime" + some new APIs and that's clearly a new runtime environment that is distinct from the JS environment on a web page, vs. the one in a worker, vs. in node, etc. But that is at best "extending a runtime", not "rolling your own".
Using the embedding APIs provided by a JS runtime as documented and intended, is not "rolling your own". By that definition I could make an app, embed a WebView, and claim I rolled my own browser runtime, which I would hope is more clearly absurd. Or I could "roll my own Command-line" by reading a string from a user, prepending some commands, and then passing it to system().
All of which is provided by V8 in this example. The global object, math object, "Object" in general, Arrays, etc are all the runtime. All this tutorial series on embedding V8 is doing is instantiating an existing runtime environment and adding using the APIs to insert a few new APIs.
All together you might say you've got a custom runtime as you have the baseline "JS runtime" + some new APIs and that's clearly a new runtime environment that is distinct from the JS environment on a web page, vs. the one in a worker, vs. in node, etc. But that is at best "extending a runtime", not "rolling your own".
Using the embedding APIs provided by a JS runtime as documented and intended, is not "rolling your own". By that definition I could make an app, embed a WebView, and claim I rolled my own browser runtime, which I would hope is more clearly absurd. Or I could "roll my own Command-line" by reading a string from a user, prepending some commands, and then passing it to system().
Not to forget the engine Charka: https://github.com/chakra-core/ChakraCore
The only bit of the old Edge that had its source emancipated. It was/is? quite performant.
The only bit of the old Edge that had its source emancipated. It was/is? quite performant.
That's a weird usage of the word "runtime" then. To me, the definition is closer to Wikipedia's definition of "runtime system" [0] and includes any interpreters, JITs, and any other language elements present while a program is running.
[0]: https://en.wikipedia.org/wiki/Runtime_systemIt’s Javascript. Everything is weird - from the equality coercion to the package management to left padding text.
Left pad is built into the language now the community has shown we can’t be trusted with anything else.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
The runtime includes the engine (interpreter for example) and an environment, which could also include a standard library, bindings to the outside world and more.
For example, in the case of node, it's a runtime built around the v8 engine, using libuv for bindings.
For example, in the case of node, it's a runtime built around the v8 engine, using libuv for bindings.
No.
The runtime is the combination of the execution engine, the garbage collector, and the environment.
V8, JSC, SpiderMonkey, LibJS, etc all provide all of those.
They all have APIs that let you (the embedder) a few new objects and functions, as it would be a useless embedding API otherwise. But that's all you're doing: adding some glorified callbacks to an existing runtime. You are in no way "rolling your own JS runtime".
To break it down:
Execution Engine: the part of the runtime that evaluates JS, an interpreter, jit, or some combination. That would be "Ignition" and "TurboFan" in V8, "LLInt" and "FTL" in JSC, "WarpMonkey" in SpiderMonkey.
Garbage Collector: the part of the runtime that supports object allocation and reclamation. "Orinoco" in V8, but not sure if given a separate marketing name in other runtimes.
The environment: this is all the builtin objects and functionality, things like the global object, the regex engine (note it's not a regex runtime because all it does find the start and stop sections of matches, the embedding environment is responsible for everything else), all those core things like the Object, Array, Math, Number, etc types, and all of their implementations and runtime functions. The Execution engine does not need any of those to be implemented, as to the engine there is essentially no distinction between those builtin things and anything else written in JS.
When you embed V8, JSC, LibJS, etc you are getting a full runtime, that can do a huge amount. You _might_ choose to use there APIs to add some new objects or or functions, but what you are doing is negligible, and certainly not "a runtime".
The runtime is the combination of the execution engine, the garbage collector, and the environment.
V8, JSC, SpiderMonkey, LibJS, etc all provide all of those.
They all have APIs that let you (the embedder) a few new objects and functions, as it would be a useless embedding API otherwise. But that's all you're doing: adding some glorified callbacks to an existing runtime. You are in no way "rolling your own JS runtime".
To break it down:
Execution Engine: the part of the runtime that evaluates JS, an interpreter, jit, or some combination. That would be "Ignition" and "TurboFan" in V8, "LLInt" and "FTL" in JSC, "WarpMonkey" in SpiderMonkey.
Garbage Collector: the part of the runtime that supports object allocation and reclamation. "Orinoco" in V8, but not sure if given a separate marketing name in other runtimes.
The environment: this is all the builtin objects and functionality, things like the global object, the regex engine (note it's not a regex runtime because all it does find the start and stop sections of matches, the embedding environment is responsible for everything else), all those core things like the Object, Array, Math, Number, etc types, and all of their implementations and runtime functions. The Execution engine does not need any of those to be implemented, as to the engine there is essentially no distinction between those builtin things and anything else written in JS.
When you embed V8, JSC, LibJS, etc you are getting a full runtime, that can do a huge amount. You _might_ choose to use there APIs to add some new objects or or functions, but what you are doing is negligible, and certainly not "a runtime".
So Node is not a JS runtime?
It is but the Node developers developers didn't roll their own JavaScript runtime.
V8 provides a JS runtime.
Node uses the V8 C++ APIs to add additional functions and objects to that runtime.
So Node has a specific JS runtime environment, just as browsers have a specific JS runtime environment (and Workers have another), etc.
All of these environments are extending the runtime environment provided by their underlying JS runtime, they're not all providing their own stdlib implementations, they're not providing their own implementations of arrays, objects, global object, etc.
Node uses the V8 C++ APIs to add additional functions and objects to that runtime.
So Node has a specific JS runtime environment, just as browsers have a specific JS runtime environment (and Workers have another), etc.
All of these environments are extending the runtime environment provided by their underlying JS runtime, they're not all providing their own stdlib implementations, they're not providing their own implementations of arrays, objects, global object, etc.
> v8 is a JS runtime. JSC is a JS runtime. SpiderMonkey is a JS runtime.
V8, JSC and SpiderMonkey have no way to interact with the outside world. Without some supporting structure, running JavaScript does anything other than being an over-engineered space heater. It's that supporting structure — the runtime — that allows you to do anything useful.
Visual Studio Redistributables for C++ is a famous case of a runtime that doesn't involve an interpreter.
V8, JSC and SpiderMonkey have no way to interact with the outside world. Without some supporting structure, running JavaScript does anything other than being an over-engineered space heater. It's that supporting structure — the runtime — that allows you to do anything useful.
Visual Studio Redistributables for C++ is a famous case of a runtime that doesn't involve an interpreter.
You're right an interpreter is not a runtime. However V8, JSC, SM, ... aren't just providing you an execution engine. E.g. they are providing all the core objects, APIs, etc that are equivalently provided by the "Visual Studio Redistributables for C++" which you seem happy to call a runtime.
The lack of direct IO is irrelevant. You can take any of these libraries, and execute arbitrary JS, and then display the output (Serenity's spreadsheet uses LibJS for equation cells IIRC). E.g. JS that runs and uses the "runtime environment" to do things.
Your particular use case may benefit from exposing some additional APIs to JS, and all these libraries allow you to do that. But exposing, for example, printf to a full JS runtime environment does not mean you've made a runtime. The belief that for something to be a "runtime" it must have built in IO routines me that no generally usable scripting environment could be a runtime.
The lack of direct IO is irrelevant. You can take any of these libraries, and execute arbitrary JS, and then display the output (Serenity's spreadsheet uses LibJS for equation cells IIRC). E.g. JS that runs and uses the "runtime environment" to do things.
Your particular use case may benefit from exposing some additional APIs to JS, and all these libraries allow you to do that. But exposing, for example, printf to a full JS runtime environment does not mean you've made a runtime. The belief that for something to be a "runtime" it must have built in IO routines me that no generally usable scripting environment could be a runtime.
JS has improved a lot over the years but I am not sure I would want it as the default scripting language for applications. JS is really only useful because it's the language you access the browser's API with. If you are creating a different kind of application and therefore don't need the DOM or other browser things then why JS at all? Having worked with Lua for some game mods, it seems like a better choice for an integrated language. Maybe a Rust inspired language would make a better fit for a modern DSL... insert XKCD comic about standards proliferation.
Lua is a better integrated language simply because that path is intended by its developers and directly documented, and its runtime is designed to be easy to couple to C. But nothing about its semantics make it better for that task than javascript. They are extremely similar except lua doesn't have stuff you would almost always want like basic string manipulation tools, regex, any array operations more sophisticated than loop.
You can argue you don't need those things, but their lack isn't a feature either. If you're not coupling to C's memory model or if for some reason someone has already done the work of building and debugging the integration... you gain nothing by using lua over js.
I've done professional work in lua and it's highly overrated imo. For integrated languages Tcl or janet is a better choice, unless your use case is extremely extremely simple and then forth is a better choice. The only time I would choose lua is if I'm integrating with C, the integration language tasks are very simple but also requires coroutines. Lua's coroutines are genuinely good.
You can argue you don't need those things, but their lack isn't a feature either. If you're not coupling to C's memory model or if for some reason someone has already done the work of building and debugging the integration... you gain nothing by using lua over js.
I've done professional work in lua and it's highly overrated imo. For integrated languages Tcl or janet is a better choice, unless your use case is extremely extremely simple and then forth is a better choice. The only time I would choose lua is if I'm integrating with C, the integration language tasks are very simple but also requires coroutines. Lua's coroutines are genuinely good.
Because Worse is Better [1]
Lua is technically superior, but JavaScript is practically superior due to its ubiquitous platform the web browser.
Similar but different to C "beating" Lisp.
[1] https://en.wikipedia.org/wiki/Worse_is_better
Lua is technically superior, but JavaScript is practically superior due to its ubiquitous platform the web browser.
Similar but different to C "beating" Lisp.
[1] https://en.wikipedia.org/wiki/Worse_is_better
I strongly disagree. Lua somehow maintains a positive reputation, but it’s extremely quirky and weird in many ways. It has some Perl-esque rough edges (#array for length), and some “let’s be different” design decisions that are counterintuitive and unnecessary (arrays starting at the 1 index instead of 0).
JavaScript as it is today is a vastly more ergonomic and fluid experience, and I am saying this as someone that has written tons of both.
The embedding experience with Lua is likely much more form fit than JavaScript (sandboxing, etc). I’m strictly speaking about syntax and language features.
JavaScript as it is today is a vastly more ergonomic and fluid experience, and I am saying this as someone that has written tons of both.
The embedding experience with Lua is likely much more form fit than JavaScript (sandboxing, etc). I’m strictly speaking about syntax and language features.
I don't want to start a 0-index war, but there is a great argument I read once and wave to share. Indexing can be tough as two different things, selecting or offsetting. When selecting you start from the first (1) option. When offsetting you move zero (0) from the first option.
0-index arrays express offsets, 1-index arrays express index selection. In C (and family) arrays start at zero because they offset, `a[i]` literally means `* (a+i)` (you actually can write `i[a]`, it will work and be valid). For other language selecting 1 base indexing is not a weird option, I'd argue.
0-index arrays express offsets, 1-index arrays express index selection. In C (and family) arrays start at zero because they offset, `a[i]` literally means `* (a+i)` (you actually can write `i[a]`, it will work and be valid). For other language selecting 1 base indexing is not a weird option, I'd argue.
If you ever need an ally for 1-indexed, you can point people to PostgreSQL.
JS is also ergonomically superior compared to Lua.
I love Lua but tbh I accept JS as the default scripting language, simply because it's ubiquitous, most runtimes are faster than or on par with Lua, and not harder to embed than Lua with stuff like quickjs. Also I don't think from language design perspective there's fundamental difference beyond syntax, metatable is pretty similar to prototype chain, JS has much more stuff you can argue it's bloated compare to Lua but you don't really pay for what you don't use.
Does Lua have the ability to launch very low cost sandboxed instances?
With Deno I know I can quickly spawn lots of isolated instances at very low overhead.
If I really want to push it far, there's tools for loading frozen snapshots that have lots of code loaded.
I've never seen similar from Lua & I'm not sure it exists. Seems like a huge advantage.
With Deno I know I can quickly spawn lots of isolated instances at very low overhead.
If I really want to push it far, there's tools for loading frozen snapshots that have lots of code loaded.
I've never seen similar from Lua & I'm not sure it exists. Seems like a huge advantage.
Creating a new lua context is pretty cheap, but I don't know how it compares to Deno / v8. It should be much lighter weight, but not sure if Deno / v8 does some special thing to speed up the use case you talked about.
The submission itself talks about creating & loading snapshots, which is sort of a Checkpoint Restore In Userspace (CRIU), allowing one to create a new context with a lot of existing libraries already loaded very very quickly.
>With Deno I know I can quickly spawn lots of isolated instances at very low overhead.
Err..new to Deno - how does one do this ?
Err..new to Deno - how does one do this ?
I think you might be underestimating browser compatibility.
There are a lot of browser API's. They're portable, designed with security in mind (at least somewhat), usually well documented, reasonably easy to understand, and people already know them. So, even if it's not a browser, you may prefer JavaScript so you don't reinvent the relevant API's and there's less to learn.
Deno itself is a good example of this.
There are a lot of browser API's. They're portable, designed with security in mind (at least somewhat), usually well documented, reasonably easy to understand, and people already know them. So, even if it's not a browser, you may prefer JavaScript so you don't reinvent the relevant API's and there's less to learn.
Deno itself is a good example of this.
Speaking of Lua, does anyone know of a Lua implementation with filesystem access that is smaller than (or as small as) QuickJS (~800KB)?
Last time I compiled standard Lua I think it was only 300-400KB. It has all the cross platform file manipulation capabilities that C does. I was using TCC though, and it does seem to generate small (unoptimized) binaries.
> If you are creating a different kind of application and therefore don't need the DOM or other browser things then why JS at all?
Because I like TypeScript and don't like Lua.
JavaScript, I don't like. It's in the same bucket to me as Python, Ruby, and Lua - things I might have to use sometimes but I'm holding my nose. But TypeScript? I like TypeScript and want to use it more.
Because I like TypeScript and don't like Lua.
JavaScript, I don't like. It's in the same bucket to me as Python, Ruby, and Lua - things I might have to use sometimes but I'm holding my nose. But TypeScript? I like TypeScript and want to use it more.
I made a demo integrating deno and sqlite into a quasi-"lambda" runtime in 200 lines for a rust meetup talk, if anyone is interested https://github.com/tbillington/rust_serverless_runtime.