Cloudflare Workers: Run JavaScript Service Workers at the Edge(blog.cloudflare.com)
blog.cloudflare.com
Cloudflare Workers: Run JavaScript Service Workers at the Edge
https://blog.cloudflare.com/introducing-cloudflare-workers/
133 comments
Daww, thanks!
I wish I'd included more images and diagrams in the post, but I'm generally terrible at coming up with those.
I also wish we'd been able to enable it in production for everyone immediately... but with a change this big we need to be cautious.
I wish I'd included more images and diagrams in the post, but I'm generally terrible at coming up with those.
I also wish we'd been able to enable it in production for everyone immediately... but with a change this big we need to be cautious.
> I didn't notice a single typo
JavaScript is spelled incorrect all over the place.
Awesome feature and post!
JavaScript is spelled incorrect all over the place.
Awesome feature and post!
That would be "incorrectly".... be careful with criticism.
You're right. Fixed.
(I habitually type it "Javascript" but it is indeed supposed to be "JavaScript".)
(I habitually type it "Javascript" but it is indeed supposed to be "JavaScript".)
Hey all! This is my project at Cloudflare.
(You may remember me as the tech lead of Sandstorm.io and Cap'n Proto.)
Happy to answer questions!
(You may remember me as the tech lead of Sandstorm.io and Cap'n Proto.)
Happy to answer questions!
This is really interesting. I have an idea where this could be helpful to the Plex user community. Recently Plex added a header that blocks the page from being iframed (X-Frame-Options).
Would doing something like this, obviously replacing example.com with their own domain.com, replace the offending header?
Would doing something like this, obviously replacing example.com with their own domain.com, replace the offending header?
addEventListener('fetch', event => {
let request = event.request;
if (request.headers.has('X-Frame-Options')) {
let newHeaders = new Headers(request.headers);
newHeaders.set('X-Frame-Options', 'ALLOW-FROM https://example.com/');
event.respondWith(fetch(request, {headers: newHeaders}));
}
// Use default behavior.
return;
});Yes.
Of course, you could only apply it to your own server.
Also, you would want to think carefully about clickjacking attacks (where someone puts your site in an invisible iframe and tricks people into clicking on it). The X-Frame-Options header was probably added to prevent clickjacking.
Of course, you could only apply it to your own server.
Also, you would want to think carefully about clickjacking attacks (where someone puts your site in an invisible iframe and tricks people into clicking on it). The X-Frame-Options header was probably added to prevent clickjacking.
Of course this would be the user's personally hosted server. Typically hidden behind a password and loaded in some sort of HTPC manager like Organizr.
According to reports, Plex' intention was to prevent clickjacking, which is perfectly reasonable but left many of their users from being able to use their Plex servers within the HTPC managers.
According to reports, Plex' intention was to prevent clickjacking, which is perfectly reasonable but left many of their users from being able to use their Plex servers within the HTPC managers.
Congratulations on the announce - it seems like a game changer of a feature, and one I look forward to testing.
Will Cloudflare be curating a list of useful worker scripts? I imagine there will be certain usecases that get a lot of attention (e.g. ESI)
Do requests made from the API go through Cloudflare's usual pipeline, or do they go straight to the backend? In short, will we need to manage the cache ourselves?
And finally, does this change Cloudflare's role as a "dumb" intermediary?
Will Cloudflare be curating a list of useful worker scripts? I imagine there will be certain usecases that get a lot of attention (e.g. ESI)
Do requests made from the API go through Cloudflare's usual pipeline, or do they go straight to the backend? In short, will we need to manage the cache ourselves?
And finally, does this change Cloudflare's role as a "dumb" intermediary?
> Will Cloudflare be curating a list of useful worker scripts?
Better than that, we plan to unify this with Cloudflare Apps, so people can publish a reusable script as an app, and other people can then "install" that app onto their site with a click.
> Do requests made from the API go through Cloudflare's usual pipeline, or do they go straight to the backend? In short, will we need to manage the cache ourselves?
The worker sits in front of the cache, so subrequests made by calling fetch() will go through the usual caching logic.
Eventually we also plan to expose the "Cache" API from standard Service Workers to allow you to manipulate the cache directly for advanced use cases, but this shouldn't be necessary for most people.
> And finally, does this change Cloudflare's role as a "dumb" intermediary?
That sounds like a policy question, which isn't my department. ;)
Better than that, we plan to unify this with Cloudflare Apps, so people can publish a reusable script as an app, and other people can then "install" that app onto their site with a click.
> Do requests made from the API go through Cloudflare's usual pipeline, or do they go straight to the backend? In short, will we need to manage the cache ourselves?
The worker sits in front of the cache, so subrequests made by calling fetch() will go through the usual caching logic.
Eventually we also plan to expose the "Cache" API from standard Service Workers to allow you to manipulate the cache directly for advanced use cases, but this shouldn't be necessary for most people.
> And finally, does this change Cloudflare's role as a "dumb" intermediary?
That sounds like a policy question, which isn't my department. ;)
Hey Kenton, should've guessed you were behind this. Lovely design, great work.
I'd be curious to learn more about the implementation (did you lift the existing SW implementation from blink somehow, or reimplement it)?
I'd be curious to learn more about the implementation (did you lift the existing SW implementation from blink somehow, or reimplement it)?
PPS - it is probably too late for this, but one of the last things I worked on at Google was Gin:
https://chromium.googlesource.com/chromium/src.git/+/lkgr/gi...
A bindings layer for v8 that was specifically intended to make implementing web-style APIs outside of Blink easier. At the time at least, refactoring things like SW out of Blink was ~impossible.
https://chromium.googlesource.com/chromium/src.git/+/lkgr/gi...
A bindings layer for v8 that was specifically intended to make implementing web-style APIs outside of Blink easier. At the time at least, refactoring things like SW out of Blink was ~impossible.
Doh, indeed, wish I had seen that earlier. V8's raw API is... tricky.
Darn, I wish you'd seen it too :). Sounds like you came to similar conclusion I did -- using C++ templates to stamp out v8::FunctionTemplate and ObjectTemplate instances.
I was pretty happy about how Gin turned out and always thought it would be nice to put it on Github where it would be easier for people to reuse.
But at the time Chromium still didn't allow all the modern C++ features, so Gin had to depend on base::Callback, which made it hard to extract cleanly.
I was pretty happy about how Gin turned out and always thought it would be nice to put it on Github where it would be easier for people to reuse.
But at the time Chromium still didn't allow all the modern C++ features, so Gin had to depend on base::Callback, which made it hard to extract cleanly.
Hey Aaron!
I looked a bit at the code in Chrome but determined that it was too attached to Chrome infrastructure that didn't make sense in our use case. Also, there are a lot of parts of Service Workers that don't make any sense outside the browser, so it looked like it would be pretty hairy trying to pull those apart. So, we're building our own implementation (a lot of which is still in-progress, of course).
I actually built a little V8 API binding shim using template and macro metaprogramming that I really like, which allows us to write classes in natural C++ and trivially export them to Javascript. We've been filling in the various non-V8-builtin APIs using this.
We're using libkj's event loop, HTTP library, and other utility code. (KJ is the C++ framework library that is in the process of spinning out of Cap'n Proto. Yeah, I may be suffering NIH, but I think it's worked well.)
I looked a bit at the code in Chrome but determined that it was too attached to Chrome infrastructure that didn't make sense in our use case. Also, there are a lot of parts of Service Workers that don't make any sense outside the browser, so it looked like it would be pretty hairy trying to pull those apart. So, we're building our own implementation (a lot of which is still in-progress, of course).
I actually built a little V8 API binding shim using template and macro metaprogramming that I really like, which allows us to write classes in natural C++ and trivially export them to Javascript. We've been filling in the various non-V8-builtin APIs using this.
We're using libkj's event loop, HTTP library, and other utility code. (KJ is the C++ framework library that is in the process of spinning out of Cap'n Proto. Yeah, I may be suffering NIH, but I think it's worked well.)
Pretty awesome stuff, will echo that this is one of the cleanest feature pages I've seen.
I'd love to hear more about your evaluation of Lua. LuaJIT is so blazingly fast(and small!) that I'm sure it'd be some pretty significant compute savings.
What sandbox solutions did you look into? Separate lua states, just overriding ENV/setfenv() or something completely different?
I'd love to hear more about your evaluation of Lua. LuaJIT is so blazingly fast(and small!) that I'm sure it'd be some pretty significant compute savings.
What sandbox solutions did you look into? Separate lua states, just overriding ENV/setfenv() or something completely different?
We love Lua and LuaJIT -- we use them extensively here.
But for running third-party code, we need to everything in our power to reduce the risk of a compromise.
Every sandbox (including V8) has bugs, and security is about risk management. With scrutiny, the low-hanging fruit is found and the risk of further bugs steadily decreases. At the end of the day, no Lua sandboxing mechanism has had anywhere near the scrutiny of V8. It's a totally unfair chicken-and-egg problem: to get scrutiny you need usage, but to get usage you need scrutiny. But, it is what it is. :/
But for running third-party code, we need to everything in our power to reduce the risk of a compromise.
Every sandbox (including V8) has bugs, and security is about risk management. With scrutiny, the low-hanging fruit is found and the risk of further bugs steadily decreases. At the end of the day, no Lua sandboxing mechanism has had anywhere near the scrutiny of V8. It's a totally unfair chicken-and-egg problem: to get scrutiny you need usage, but to get usage you need scrutiny. But, it is what it is. :/
Cool, appreciate the candid response.
I think there's definitely a compelling reason to use JS both from a developer comfort perspective and the fact that it's a pretty battle-tested path.
It sounds like anything short of a full-blown container(with all the overhead that brings) wouldn't be sufficient to cover the security concerns. I'd love to see Lua battle-harded a bit more in that area so was just curious if you had any new learnings.
I think there's definitely a compelling reason to use JS both from a developer comfort perspective and the fact that it's a pretty battle-tested path.
It sounds like anything short of a full-blown container(with all the overhead that brings) wouldn't be sufficient to cover the security concerns. I'd love to see Lua battle-harded a bit more in that area so was just curious if you had any new learnings.
Are you able to talk about the pricing structure? Is this going to be per request, total cpu time, etc?
edit. In another reply you said that pricing hasn't been finalised which is understandable. We've got a few use cases which CF Workers would be ideal for but we'd be looking at 10-15k requests per minutes which could get expensive if pricing is per request.
edit. In another reply you said that pricing hasn't been finalised which is understandable. We've got a few use cases which CF Workers would be ideal for but we'd be looking at 10-15k requests per minutes which could get expensive if pricing is per request.
We are taking input on what the use cases are in order to come up with a pricing model that makes sense. Definitely reach out with what you're trying to do and we'll take it into account as we're designing how it's priced.
The blog post mentions WebAssembly support in V8, but I can't get it to work in the playground. Coming later?
Yes, we intentionally disabled it for now because the WebAssembly API allows dynamic code loading at runtime. We want all code deployed on Cloudflare to go through our standard deployment process so that we have copies of all of it e.g. for forensic purposes. Also, when we integrate with Cloudflare Apps, that code needs to be moderated.
What we'll do at some point is allow you to provide a WASM blob along-side your script, which will be loaded separately and then exposed to your script probably as a global variable.
What we'll do at some point is allow you to provide a WASM blob along-side your script, which will be loaded separately and then exposed to your script probably as a global variable.
Makes sense.
Hi Kenton - awesome new feature!
In the blog post you talk about the trade-offs of using JavaScript vs other options like containers. I thought you might be interested in this comparison of using JavaScript vs other sandboxing options.
https://blog.acolyer.org/2017/08/22/javascript-for-extending...
> Hosts of V8 can multiplex applications by switching between contexts, just as conventional protection is implemented in the OS as process context switches… A V8 context switch is just 8.7% of the cost of a conventional process context switch… V8 will allow more tenants, and it will allow more of them to be active at a time at a lower cost.
In the blog post you talk about the trade-offs of using JavaScript vs other options like containers. I thought you might be interested in this comparison of using JavaScript vs other sandboxing options.
https://blog.acolyer.org/2017/08/22/javascript-for-extending...
> Hosts of V8 can multiplex applications by switching between contexts, just as conventional protection is implemented in the OS as process context switches… A V8 context switch is just 8.7% of the cost of a conventional process context switch… V8 will allow more tenants, and it will allow more of them to be active at a time at a lower cost.
Yes, that sounds like exactly the conclusions we came to. :)
Will there be any limits on memory/execution time etc like with AWS Lambda?
Yes. Currently you get at most 50ms CPU time per request, and memory usage of your Javascript VM is limited to 128MB. These numbers may change before launch.
In practice, though, a typical script probably uses much less than 1ms of CPU time per request and probably needs only a couple megs of RAM. Because we're applying limits on the level of a V8 Isolate, not on an OS process, the limits go a lot further.
Keep in mind that Cloudflare Workers are not intended to be a place where you do "bulk compute", but rather where you run little bits of code that benefit from being close to users.
In practice, though, a typical script probably uses much less than 1ms of CPU time per request and probably needs only a couple megs of RAM. Because we're applying limits on the level of a V8 Isolate, not on an OS process, the limits go a lot further.
Keep in mind that Cloudflare Workers are not intended to be a place where you do "bulk compute", but rather where you run little bits of code that benefit from being close to users.
I would very much like to see a pay-as-you-go cloud pricing model that allows you to pay to go over these limits (within reason.) For the use case I'm envisaging, filtering largish buffers of Cap'N Proto data (I'm tickled that both of these things are your babies!) I can use streaming fetch API to stay within the memory limit, but the 50ms limit would really be a hard wall that would cause requests to fail on larger amounts of data. I'd be happy to pay in those cases for extra execution time.
I'd love to hear more about your use case (especially since it involves Cap'n Proto!). E-mail me? (kenton at cloudflare)
Thanks, I should have specified I meant in comparison to Lambda@Edge on CloudFront which this seems quite similar in terms of the use cases.
What happens if the limits are reached? Does the end user get an error or does the request proceed as it would without the script?
Currently, an error, because default handling might not be correct in many cases -- imagine a script implementing access control or expanding an HTML template. But I'd be open to the idea of letting the script call, like, `event.fallbackOk()` upfront to specify that default behavior as a fallback is safe.
Note that the 50ms is CPU time, not wall time. This is actually a pretty huge amount of time for handling one request.
Note that the 50ms is CPU time, not wall time. This is actually a pretty huge amount of time for handling one request.
Oh yeah, BTW: If you're an experienced systems engineer interested in working on a young codebase written in ultra-modern C++, let me know (kenton at cloudflare).
Point of curiosity: why did you go with C++ instead of Rust?
V8's API is C++. While bindings exist in other languages, they don't always expose everything and don't always keep up with V8's frequent API changes. I felt it was essential that we bind closely to V8, being able to use all of its features, and that we be able to track closely to the latest stable release.
Also, I personally have decades of experience in C++ and wasn't sure this should be the project where I work on learning Rust.
That said, I love Rust and highly encourage more people to use it.
Also, I personally have decades of experience in C++ and wasn't sure this should be the project where I work on learning Rust.
That said, I love Rust and highly encourage more people to use it.
That’s about what I expected; thanks.
Just for reference for people reading this: rust-bindgen can produce bindings to C++ code these days, though that gets you an unsafe library; you’d want to build another thin layer on top of it to nail down the ownership semantics so that you can provide a safe interface.
Just for reference for people reading this: rust-bindgen can produce bindings to C++ code these days, though that gets you an unsafe library; you’d want to build another thin layer on top of it to nail down the ownership semantics so that you can provide a safe interface.
bindgen’s C++ support is better than it used to be, but still pretty limited, e.g. no support for inline functions, or their moral equivalent, methods defined within a class definition (which most C++ code has a ton of). And even if it works, to quote the documentation:
> using [C++] types in Rust will be nowhere near as nice as using them in C++. You will have to manually call constructors, destructors, overloaded operators, etc yourself.
source: https://github.com/rust-lang-nursery/rust-bindgen/blob/maste...
> using [C++] types in Rust will be nowhere near as nice as using them in C++. You will have to manually call constructors, destructors, overloaded operators, etc yourself.
source: https://github.com/rust-lang-nursery/rust-bindgen/blob/maste...
Thanks. Sometimes I feel like the "just use X" folks don't really think about the problem at hand, not to mention how trivializing it can be. And this is coming from a Rust fan and someone who uses bindgen.
https://www.neon-bindings.com/ is written by Dave Herman, you might find it interesting to play with :)
Is there any way to store state?
So, there is the HTTP cache. It turns out a lot of storage use cases are actually answered by the cache. E.g. say you need some lookup table that changes every five minutes -- make it cacheable, max-age=300, and have your worker load it from cache.
Of course, that's not the answer to everything. We don't plan to offer other storage in v1 but we are definitely thinking about it for the future.
Of course, that's not the answer to everything. We don't plan to offer other storage in v1 but we are definitely thinking about it for the future.
Obligatory transformation of cloud to you-know-what:
https://cloudflareworkers.com/#9bdc354e936c05a4a1d7df7eb0d7f...
https://cloudflareworkers.com/#9bdc354e936c05a4a1d7df7eb0d7f...
https://cloudflareworkers.com/#1f3220d90979ded0b1f5413db0eac... this fixes the styling issues :D
Kudos for re-using an existing API when one was already available in the same language for a very similar usecase.
Any time you commit to someone else's API -- whether it's an actual industry standard, or simply some de facto widely used paradigm -- you incur risks; conversely, now that you're a vested participant, consider being involved in the future of the spec so it can evolve where it needs to meeting emerging needs around its new uses.
Any time you commit to someone else's API -- whether it's an actual industry standard, or simply some de facto widely used paradigm -- you incur risks; conversely, now that you're a vested participant, consider being involved in the future of the spec so it can evolve where it needs to meeting emerging needs around its new uses.
Absolutely! One reason we wanted to get this out in public before it's ready is so we can properly engage with the spec writers (and the V8 team).
That said, I am amazed by how well the spec fits as-is. I don't usually like other people's API designs but in this case I think they did a really good job, and I've been pleased not to have to think about API design myself.
That said, I am amazed by how well the spec fits as-is. I don't usually like other people's API designs but in this case I think they did a really good job, and I've been pleased not to have to think about API design myself.
Paging HN user johansch from his comment on the Cloudflare Apps mitm JavaScript injection discussion 3 months ago:
johansch: All I want is my code running on your nodes all around the world with an end-to-end ping that is less than 10 ms to the average client
dsl: Akamai Edge Compute is what they are asking for
https://news.ycombinator.com/item?id=14650025
johansch: All I want is my code running on your nodes all around the world with an end-to-end ping that is less than 10 ms to the average client
dsl: Akamai Edge Compute is what they are asking for
https://news.ycombinator.com/item?id=14650025
This is fantastic. Fastly is great but hard to use with Varnish VCL, and no other CDN had any real scripting capabilities. The service worker API is also a lot better than serverless cloud functions or lambda@edge.
Great product announcement post! I especially liked the Q&A section's reasons for not choosing alternatives.
Is the lack of maturity also the reason for not choosing something like vm2 for NodeJS https://github.com/patriksimek/vm2
Is the lack of maturity also the reason for not choosing something like vm2 for NodeJS https://github.com/patriksimek/vm2
I actually didn't know about that one.
Looking briefly, it looks like it's based on creating separate contexts, but not separate isolates. Contexts within the same isolate can be reasonably secure (it's how Chrome sandboxes iframes from their parent frames, after all), but they still share a single heap and must run on the same thread. Isolates can run on separate threads. We prefer to give each script its own isolate, so that one script using a lot of CPU does not block other scripts. We also want to be able to kill a script that does, say, "while(true) {}".
So yeah, it looks like a neat library but it probably wouldn't suit our needs.
Looking briefly, it looks like it's based on creating separate contexts, but not separate isolates. Contexts within the same isolate can be reasonably secure (it's how Chrome sandboxes iframes from their parent frames, after all), but they still share a single heap and must run on the same thread. Isolates can run on separate threads. We prefer to give each script its own isolate, so that one script using a lot of CPU does not block other scripts. We also want to be able to kill a script that does, say, "while(true) {}".
So yeah, it looks like a neat library but it probably wouldn't suit our needs.
This sounds like a really powerful feature. I love how it uses JavaScript, which makes it much more approachable for web developers.
My experience with Service Worker APIs hasn't been very positive, although I don't have any suggestions for ways it could be improved, so I apologize for the non-constructive feedback. Maybe after using it more I'll change my mind. I recognize that everyone involved is likely working hard to provide an API that's capable of handling a wide range of problems, many of which I likely haven't even considered.
Here's a more actionable complaint: fetch doesn't support timeout or abortion. I have a hard time understanding how this isn't a problem for more people. Say what you will about XMLHttpRequest, at least it supports these basic features. As an end-user, I always find it absolutely infuriating when things hang forever because a developer forgot to handle failure cases.
I'd love it if you published a locally runnable version. Aside from making it easier to configure and experiment, it would give me peace of mind to know that I could continue to use the same configuration if Cloudflare decided to terminate my service.
My experience with Service Worker APIs hasn't been very positive, although I don't have any suggestions for ways it could be improved, so I apologize for the non-constructive feedback. Maybe after using it more I'll change my mind. I recognize that everyone involved is likely working hard to provide an API that's capable of handling a wide range of problems, many of which I likely haven't even considered.
Here's a more actionable complaint: fetch doesn't support timeout or abortion. I have a hard time understanding how this isn't a problem for more people. Say what you will about XMLHttpRequest, at least it supports these basic features. As an end-user, I always find it absolutely infuriating when things hang forever because a developer forgot to handle failure cases.
I'd love it if you published a locally runnable version. Aside from making it easier to configure and experiment, it would give me peace of mind to know that I could continue to use the same configuration if Cloudflare decided to terminate my service.
Abortable fetch actually just recently became a thing, as in the Github issue was closed 9 days ago. More info:
https://developers.google.com/web/updates/2017/09/abortable-...
We'll be implementing soon.
https://developers.google.com/web/updates/2017/09/abortable-...
We'll be implementing soon.
Could this be used for HTTP Push across domains? I would love an easy way to take advantage of HTTP Push for assets hosted on S3 without routing requests to my origin server.
Yes, your worker can make subrequests to other domains, and serving some assets out of S3 is a use case we specifically want to enable.
I haven't looked into how specifically to expose HTTP Push in the API but that certainly seems like something we should support.
I haven't looked into how specifically to expose HTTP Push in the API but that certainly seems like something we should support.
Please, add support HTTP/2 push!
This is very, very awesome work. My team has been working on enabling the PRPL pattern[1] and differential serving on a few platforms, and edge caching has been a problem. We've tried to use Vary: User-Agent, but that leads to low cache hits. This API would let us to much smarter UA sniffing at the edge.
From there we just need to parse some responses like JS and HTML, to be able to push their sub-resources, for an instant speedup and great caching for fine-grained deployments.
[1]: https://developers.google.com/web/fundamentals/performance/p...
This is very, very awesome work. My team has been working on enabling the PRPL pattern[1] and differential serving on a few platforms, and edge caching has been a problem. We've tried to use Vary: User-Agent, but that leads to low cache hits. This API would let us to much smarter UA sniffing at the edge.
From there we just need to parse some responses like JS and HTML, to be able to push their sub-resources, for an instant speedup and great caching for fine-grained deployments.
[1]: https://developers.google.com/web/fundamentals/performance/p...
Http/2 push is interesting here. Beyon static assists I wonder about read API orchestration or equivalent of graghQL on edge. I.e a single request comes in and it's broken apart against catchable resources and pushed back to the client from cache where possible.
That's actually one of the canonical use cases we thought about while designing. It seems splitting graphql queries into cacheable chunks is a common need.
So in a way this is similar to for example AWS Lambda ? It can process incoming http requests in many ways ? Fascinating idea.
Is there any indication on price level ? And what about runtime duration ?
Is there any indication on price level ? And what about runtime duration ?
It's actually somewhat different. AWS Lambda is intended to act as your origin server. Generally your Lambda functions run in a small number of locations, not necessarily close to users.
Cloudflare Workers will run in all of Cloudflare's 117 (and growing) locations. The idea is that you'd put code in a worker if you need the code to run close to the user. You might want that to improve page load speed, or to reduce bandwidth costs (don't have to pay for the long haul), or because you want to augment Cloudflare's feature set, or a number of other reasons. But, generally, you would not host your whole service on this. (Well, you could, but it's not the intent.)
We haven't nailed down pricing yet, but we've worked hard to create the most efficient possible design so that we can make pricing attractive.
Cloudflare Workers will run in all of Cloudflare's 117 (and growing) locations. The idea is that you'd put code in a worker if you need the code to run close to the user. You might want that to improve page load speed, or to reduce bandwidth costs (don't have to pay for the long haul), or because you want to augment Cloudflare's feature set, or a number of other reasons. But, generally, you would not host your whole service on this. (Well, you could, but it's not the intent.)
We haven't nailed down pricing yet, but we've worked hard to create the most efficient possible design so that we can make pricing attractive.
Can I use this together with CloudFlare streaming to open up stream ingestion based on some HMAC signed parameters?
To give regular end-user the rights to e.g. 'record one video of up to X seconds and Y mbps under ID "some-uuid"'
I've been waiting for a proper Content Ingestion Network for ages by now... if the CloudFlare video team ever wants to talk to someone who hand-rolled their own single-node version of this I'd be more than willing to share my experiences.
To give regular end-user the rights to e.g. 'record one video of up to X seconds and Y mbps under ID "some-uuid"'
I've been waiting for a proper Content Ingestion Network for ages by now... if the CloudFlare video team ever wants to talk to someone who hand-rolled their own single-node version of this I'd be more than willing to share my experiences.
That's a really interesting idea!
If you added this with RTMP+WebRTC ingestion support all of these [1] companies would probably jump on the opportunity... and dozens more would start up in that space ;-)
[1] http://interviewingsoftware.com/
[1] http://interviewingsoftware.com/
So, more like https://aws.amazon.com/lambda/edge/ ?
That is correct. However, the design is very different. Lambda@Edge is, from what I can tell, based on running a full Node.js process in a process-level sandbox for each customer, whereas we are embedding V8 directly.
What are you embedding v8 into exactly? What happens if I figure out a buffer overflow in v8? What do I break out into? Is it solely process isolation? Is it in a container? A VM?
> What are you embedding v8 into exactly?
C++ code.
Yeah, I know...
> What happens if I figure out a buffer overflow in v8?
You report it to Google for $15,000. ;)
More seriously, though, this is something we've spent a huge amount of effort worrying about. There are many layers of sandboxing and mitigation, including an incredibly tight seccomp filter, namespaces, cgroups, ASLR, architectural changes to keep sensitive secrets away from risky processes, etc. But we still worry and will continue to add more safeguards.
C++ code.
Yeah, I know...
> What happens if I figure out a buffer overflow in v8?
You report it to Google for $15,000. ;)
More seriously, though, this is something we've spent a huge amount of effort worrying about. There are many layers of sandboxing and mitigation, including an incredibly tight seccomp filter, namespaces, cgroups, ASLR, architectural changes to keep sensitive secrets away from risky processes, etc. But we still worry and will continue to add more safeguards.
We haven't nailed down pricing yet
How do you store scripts in each edge ? Since in 1 of 117 datacenters, you'll need all scripts from all customers ready to go/execute.
Same as any other site setting, all of which need to be pushed to every location. We have existing infrastructure for this that propagates changes everywhere within a couple seconds.
[deleted]
Have you guys considered side channels?
Seems like whenever there's co-execution (VMs, JavaScript, etc) there seem to be side channel leakages.
Seems like whenever there's co-execution (VMs, JavaScript, etc) there seem to be side channel leakages.
We're certainly aware of them, but haven't spent a lot of time focused on this issue yet. Of course, the issue exists on all forms of shared compute. So if you're going to do crypto, you'd better make it constant-time. Which is... not easy in Javascript. (But we will provide the WebCrypto API, which might help.)
There is a theoretical solution that we might be able to explore at some point: If compute is deterministic -- that is, always guaranteed to produce the same result given the same input -- then it can't possibly pick up side channels. It's possible to imagine a Javascript engine that is deterministic. The fact that Javascript is single-threaded helps here. In concrete terms, this would mean hooking Date.now() so that it stays constant during continuous execution, only progressing between events.
That said, this is just a theory and there would be lots of details to work out.
There is a theoretical solution that we might be able to explore at some point: If compute is deterministic -- that is, always guaranteed to produce the same result given the same input -- then it can't possibly pick up side channels. It's possible to imagine a Javascript engine that is deterministic. The fact that Javascript is single-threaded helps here. In concrete terms, this would mean hooking Date.now() so that it stays constant during continuous execution, only progressing between events.
That said, this is just a theory and there would be lots of details to work out.
> There is a theoretical solution that we might be able to explore at some point: If compute is deterministic -- that is, always guaranteed to produce the same result given the same input -- then it can't possibly pick up side channels.
Doesn't this require the timing and interleaving with other processes also be deterministic? ...which seems hard to guarantee with modern CPUs, async IO, and shared execution.
Doesn't this require the timing and interleaving with other processes also be deterministic? ...which seems hard to guarantee with modern CPUs, async IO, and shared execution.
If you don't provide (real) time as an input to the program, then non-determinism of time does not affect determinism of the program. At least, in theory. It's definitely a long way from there to practice.
That's true -- but "the program" is also any code using yours as a middle-step, which is the case of a network service is a huge number of people. So anything that requires total program control rather than subroutine enforced safety is a nonstarter.
The timing may occur outside of your control, and then statistical profiling used. If the timing of a reponse to a network request has a time component that depends on shared load, then you have a side-channel.
The timing may occur outside of your control, and then statistical profiling used. If the timing of a reponse to a network request has a time component that depends on shared load, then you have a side-channel.
I read the article, but I'm not sure what this technology is enabling really. Can it be thought of as a new player in the lambda/serverless category? Does it have any advantages to using other serverless stacks like aws lambda or azure?
The advantage is that the code runs close to your users. 90% of the people on Earth are within 10ms of a Cloudflare location, so by pushing your code out to the edge you can make your site faster. You also save on the cost of transmitting data around the world.
I guess it's mod_rewrite in js
Seems like this plus the Cloudflare Apps could yield some interesting projects.
definitely. We plan on exposing the workers to app developers later this year.
Yup. :-)
This seems like it has a ton of potential. I'm already thinking of the possibilities for my own Cloudflare-proxied sites.
I hope it's not priced too harshly. Hopefully an added monthly flat rate rather than per-request pricing?
I hope it's not priced too harshly. Hopefully an added monthly flat rate rather than per-request pricing?
Sounds like this could be a pretty trivial way to load balance React prerendering. As long as the react code fetches all data in 1 call it should be at least as efficient as doing it all in Nodejs on your server.
Relevant example using Preact https://cloudflareworkers.com/#87ec550da87ab3f7de756a4979c00...
Could this be generalized to work across frameworks with e.g. https://github.com/prerender/prerender?
- Is it "Cloudflare Workers" or "Cloudflare Service Workers"?
A "Cloudflare Worker" is JavaScript you write that runs on Cloudflare's edge.
A "Cloudflare Service Worker" is specifically a worker which handles
HTTP traffic and is written against the Service Worker API.
Not really, because once there are workers other than Service Workers, they'll have their own names. To be clear, a Service Worker is one kind of Worker. At the moment it's the only kind, but we could introduce others in the future. For instance, maybe we'd introduce a "DNS Worker" that responds to DNS requests.
Also note we didn't invent these terms. "Workers" and "Service Workers" are W3C standards.
Also note we didn't invent these terms. "Workers" and "Service Workers" are W3C standards.
Ok. If you're going to have a DNS worker, wouldn't it make more sense to call the http one the HTTP worker? Isn't DNS a service?
We called it a Service Worker because the API follows the W3C standard Service Worker spec:
https://www.w3.org/TR/service-workers-1/
https://developer.mozilla.org/en-US/docs/Web/API/Service_Wor...
I agree that the naming is not completely intuitive, but these are the names that exist and so we're using them.
I'm not aware of any similar spec for DNS workers so I figure we'll use whatever name we want there.
https://www.w3.org/TR/service-workers-1/
https://developer.mozilla.org/en-US/docs/Web/API/Service_Wor...
I agree that the naming is not completely intuitive, but these are the names that exist and so we're using them.
I'm not aware of any similar spec for DNS workers so I figure we'll use whatever name we want there.
Oh my. Is there any form of persistence, present or planned?
If there is - this means that there (eventually) will be a way to have logs from the edge servers. I'm just thinking about a worker that would collect the requests and responses data in some circular buffer, and try to push it to the origin server. Eventually, the data will get through, so no CDN-returned 52x ("web server is not responding" etc) errors would go unnoticed.
If there is - this means that there (eventually) will be a way to have logs from the edge servers. I'm just thinking about a worker that would collect the requests and responses data in some circular buffer, and try to push it to the origin server. Eventually, the data will get through, so no CDN-returned 52x ("web server is not responding" etc) errors would go unnoticed.
We won't offer writable storage at the edge in v1, but you could always make a background request to push these events back to some log service you maintain (on separate infrastructure from your main service to make simultaneous outages unlikely). Note that you can make background requests that happen after the main response has already returned (but still subject to the per-request CPU limits, etc.).
We're thinking about how to do writable storage, but it's tricky, since one thing we don't want is for developers to have to think about the fact that their code is running in hundreds of locations.
We're thinking about how to do writable storage, but it's tricky, since one thing we don't want is for developers to have to think about the fact that their code is running in hundreds of locations.
So, can I use this to do this:
Render your SPA (different index.html) when a login cookie is set and otherwise render your landing page (yet another index.html)? - Such that that my http://example.com can always be cached (unless it needs to hit the server where the same logic is implemented).
And in general, how do you manage your landing page vs. your SPA?
Render your SPA (different index.html) when a login cookie is set and otherwise render your landing page (yet another index.html)? - Such that that my http://example.com can always be cached (unless it needs to hit the server where the same logic is implemented).
And in general, how do you manage your landing page vs. your SPA?
This is really interesting; kudos to Cloudflare for launching what seems like a cool thing!
Also, agree with other commentators here; nicely-written blog post!
Also, agree with other commentators here; nicely-written blog post!
Very nice. Are there plans to expand on it? For example, some way to allow state to be kept at the edge as well.
Great addition to your services.
I really want to see the pricing.
Also I'm quite excited to see how this will play along with the apps concept. Do you plan also to introduce a monetisation system for the apps? It will be a good incentive then for developers to get some nice income.
As a matter of fact, the monetization part already exists: Cloudflare Apps is a thing today, including paid apps. Today, though, they are limited to injecting client-side code. Once Cloudflare Workers are integrated with Apps, they'll be able to include server-side code too. https://www.cloudflare.com/apps/
PS. There's an investment fund targeting Cloudflare App developers: https://www.cloudflare.com/fund/
PS. There's an investment fund targeting Cloudflare App developers: https://www.cloudflare.com/fund/
This is really cool! I wonder if this will become the new "S3 API" and get everyone to copy^H^H^H^H implement it.
Also a little concerned about writing anything substantial without an onprem version!
Also a little concerned about writing anything substantial without an onprem version!
Yes, it's my hope that Service Workers will be recognized broadly as the standard API for this.
FWIW I've seen several different folks saying they're working on implementing a Service Workers API shim on top of Node, which would be a great way to get an on-prem version, and should not be very hard to do.
FWIW I've seen several different folks saying they're working on implementing a Service Workers API shim on top of Node, which would be a great way to get an on-prem version, and should not be very hard to do.
This is excellent news! Now I wish AWS Lambda and alternatives also supported Service Worker API (Google Cloud Functions has express like API).
How will billing work for this? It seems like I could conceivably write an entire application that just runs as a Service Worker.
Is this similar to Amazon's Lambda@Edge?
Sounds like it. I think Cloudflare's edge (no pun intended) is going to be their 110+ locations to host this. As opposed to Amazon's 93 I believe? (If all cloudfront spots support lambda@edge that is) https://aws.amazon.com/cloudfront/details/
I would say at 117 edge locations vs 93 Cloudflare will have lower latency, but most notable in markets that most companies care least about. Probably of more importance is by using embedded V8 as opposed to a process-level sandbox running Node.JS, the Cloudflare service worker will have a much faster startup time in all locations. I seem to recall from tests in Lambda that we're looking at 40-50ms, which is actually substantial given how close these locations often are to the user. But the biggest edge in my books is CloudFront bandwidth is ridiculously expensive and Cloudflare doesn't charge (directly) for bandwidth. Bandwidth often being the single biggest line item in your AWS bill if you get a lot of traffic.
fyi from a different thread: https://news.ycombinator.com/item?id=15365314
offtopic: can cloudflare be used as a CDN for just assets? The cloudflare docs talk only about using it as CDN for an entire site.
Technically, yes, but as I understand it, they frown upon it and could potentially terminate your account. See this section of their ToS (https://www.cloudflare.com/terms/):
>SECTION 10: LIMITATION ON NON-HTML CACHING
>You acknowledge that Cloudflare’s Service is offered as a platform to cache and serve web pages and websites and is not offered for other purposes, such as remote storage. Accordingly, you understand and agree to use the Service solely for the purpose of hosting and serving web pages as viewed through a web browser or other application and the Hypertext Markup Language (HTML) protocol or other equivalent technology. Cloudflare’s Service is also a shared web caching service, which means a number of customers’ websites are cached from the same server. To ensure that Cloudflare’s Service is reliable and available for the greatest number of users, a customer’s usage cannot adversely affect the performance of other customers’ sites. Additionally, the purpose of Cloudflare’s Service is to proxy web content, not store data. Using an account primarily as an online storage space, including the storage or caching of a disproportionate percentage of pictures, movies, audio files, or other non-HTML content, is prohibited. You further agree that if, at Cloudflare’s sole discretion, you are deemed to have violated this section, or if Cloudflare, in its sole discretion, deems it necessary due to excessive burden or potential adverse impact on Cloudflare’s systems, potential adverse impact on other users, server processing power, server memory, abuse controls, or other reasons, Cloudflare may suspend or terminate your account without notice to or liability to you.
I think in practice, unless you have a very popular website or are abusing it in some way, they probably wouldn't care or even notice. But you'd still be taking a gamble.
(I don't work at Cloudflare though so take whatever I say with a grain of salt.)
>SECTION 10: LIMITATION ON NON-HTML CACHING
>You acknowledge that Cloudflare’s Service is offered as a platform to cache and serve web pages and websites and is not offered for other purposes, such as remote storage. Accordingly, you understand and agree to use the Service solely for the purpose of hosting and serving web pages as viewed through a web browser or other application and the Hypertext Markup Language (HTML) protocol or other equivalent technology. Cloudflare’s Service is also a shared web caching service, which means a number of customers’ websites are cached from the same server. To ensure that Cloudflare’s Service is reliable and available for the greatest number of users, a customer’s usage cannot adversely affect the performance of other customers’ sites. Additionally, the purpose of Cloudflare’s Service is to proxy web content, not store data. Using an account primarily as an online storage space, including the storage or caching of a disproportionate percentage of pictures, movies, audio files, or other non-HTML content, is prohibited. You further agree that if, at Cloudflare’s sole discretion, you are deemed to have violated this section, or if Cloudflare, in its sole discretion, deems it necessary due to excessive burden or potential adverse impact on Cloudflare’s systems, potential adverse impact on other users, server processing power, server memory, abuse controls, or other reasons, Cloudflare may suspend or terminate your account without notice to or liability to you.
I think in practice, unless you have a very popular website or are abusing it in some way, they probably wouldn't care or even notice. But you'd still be taking a gamble.
(I don't work at Cloudflare though so take whatever I say with a grain of salt.)
Wow, thanks for that info! Our enterprise account uses CF as a CDN for medium sized (~50MB) video files and I just assumed that they were ok with everyone doing that. I guess not.
I guess the answer for the OP then is, get an enterprise account and you can do whatever you want.
I guess the answer for the OP then is, get an enterprise account and you can do whatever you want.
CDNs dont make a distinction. It's a reverse proxy so any HTTP request can be cached. Cloudflare has a list of standard filetypes supported: https://support.cloudflare.com/hc/en-us/articles/200172516-W...
They do have rules on large files though and prefer you use it for typical web content like text and images.
They do have rules on large files though and prefer you use it for typical web content like text and images.
This is really cool...but why?
I thought of a few use cases...
A more granular way to block requests than what cloudflare provides (IP, BGP AS, country)
Tokenizing credit cards at the edge if you have a payment provider that supports that, and are using CF's PCI compliant environment.
Injecting a unique ID into requests for log correlation. You get collisions if you do this via browser javascript for various reasons. I'm assuming v8 has a better Math.random(), or that you would at least be able to find a workaround since it's in one stack.
I'm sure there's more.
A more granular way to block requests than what cloudflare provides (IP, BGP AS, country)
Tokenizing credit cards at the edge if you have a payment provider that supports that, and are using CF's PCI compliant environment.
Injecting a unique ID into requests for log correlation. You get collisions if you do this via browser javascript for various reasons. I'm assuming v8 has a better Math.random(), or that you would at least be able to find a workaround since it's in one stack.
I'm sure there's more.
Also some good use cases. :)
I worked for an analytics company at one point and we generated impression IDs server side (you'd be surprised how broken client-side JS randomness can be) and included them in our loader. If we had been using CloudFlare I would have put our loader directly into this service worker so that we could shave a few more milliseconds off our intial response time, so that we could get to our cached JS faster.
So yeah, if you care about milliseconds :)
So yeah, if you care about milliseconds :)
Ah, that makes more sense. Thanks!
Can’t you already do this with Fastly and Varnish?
Also, this feature is pretty cool!