Cloudflare Pub/Sub: Programmable MQTT Message Brokers(blog.cloudflare.com)
blog.cloudflare.com
Cloudflare Pub/Sub: Programmable MQTT Message Brokers
https://blog.cloudflare.com/announcing-pubsub-programmable-mqtt-messaging/
51 comments
At least on my team at Cloudflare, great Terraform support is a basically required part of any API change. We have an official Terraform provider (https://registry.terraform.io/providers/cloudflare/cloudflar...), and nearly every API feature is provisionable using that provider.
> Is there some sort of "innovation week" type thing going on right now?
Yes. "Platform Week 2022": https://blog.cloudflare.com/platform-week-2022/
Yes. "Platform Week 2022": https://blog.cloudflare.com/platform-week-2022/
[deleted]
Platform Week
Cloudflare is really fast putting the pieces together to become a modern simpler AWS. By focusing on edge networks and programmability from the start they've been able to outmaneuver AWS in product development and pricing.
AWS today is stable, reliable but expensive and rigid. A lot of their products feel like they've been taken out of the oven 15 minutes too early. AppSync, API Gateway, CloudFormation and others all feel lethargic, stagnant or not fit for purpose in the current world.
I don't think AWS is in a position to compete with Cloudflare's strategy without massively slashing bandwidth prices and improving their product development. They also aren't really a credible edge platform. Lambda@Edge is really bad, CloudFront is slow and terrible compared to Akamai, Fastly and Cloudflare (and Amazon's retail business doesn't even use it -- because it sucks).
Super interesting to watch. ($NET shareholder)
AWS today is stable, reliable but expensive and rigid. A lot of their products feel like they've been taken out of the oven 15 minutes too early. AppSync, API Gateway, CloudFormation and others all feel lethargic, stagnant or not fit for purpose in the current world.
I don't think AWS is in a position to compete with Cloudflare's strategy without massively slashing bandwidth prices and improving their product development. They also aren't really a credible edge platform. Lambda@Edge is really bad, CloudFront is slow and terrible compared to Akamai, Fastly and Cloudflare (and Amazon's retail business doesn't even use it -- because it sucks).
Super interesting to watch. ($NET shareholder)
They have no non-function compute though, which is arguably AWS’s core feature.
It's also the things that's easiest to migrate away from to a cheap-and-performant VPS or dedicated server
I guess from my perspective that's also why I want it in my cloud. I'm willing to pay for managed services, but don't want to be using services I can't easily migrate away from.
Great take, I agree. Thoughts on a Fastly or Cloudflare acquisition by one of the tech giants?
If Akamai were smart they'd buy Fastly yesterday, because they're rapidly falling behind. Cloudflare is harder to see because of antitrust concerns. With Fastly's current position, they should sell, and someone should buy them for the network and progress on edge compute alone.
[deleted]
Pub/Sub, Workers, KV, Durable Objects, R2... yes, these all sounds promising. Awesome tool for developers. But there is one huge obstacle in the middle that prevents most to actually try this stack.
Logs and Account Access control.
Cloudflare is a very powerful platform, you can't let everyone be Admin. If you run anything, you need to be able to see logs it to ship it somewhere.
For some reason, Cloudflare thinks that these are enterprise feature. I got news for you, these are basic features for anyone who wants to build more than just a simple script for individual use.
You work so hard to build these features but let only a small subset of users really do anything meaningful with it
Logs and Account Access control.
Cloudflare is a very powerful platform, you can't let everyone be Admin. If you run anything, you need to be able to see logs it to ship it somewhere.
For some reason, Cloudflare thinks that these are enterprise feature. I got news for you, these are basic features for anyone who wants to build more than just a simple script for individual use.
You work so hard to build these features but let only a small subset of users really do anything meaningful with it
We just announced something called the Workers Analytics Engine today: https://blog.cloudflare.com/workers-analytics-engine/
More coming soon!
Happy to chat more - jpl at cloudflare dot com
More coming soon!
Happy to chat more - jpl at cloudflare dot com
Yes, fully agree, just to give Fred access to develop a worker in dev environment we have to grant them access to the whole account, it's batshit insane. We created acmecorpx.com for dev and put this on a seperate CF account but it still sucks
Honest question, not a snark attack. But would it be possible to build those things on top of Cloudflare itself, esp the logging? For access control, you would have to have a separate process controlling the CF api for configuration.
Not so simple when a lot of the tooling that facilitates easy development is tied to CF API, are you meant to build your own cloudflared and wrangler just so you dont have to grant everyone in the org super admin?
This is where AWS wins, AWS gives you everything from $0/month, IAM in the above case, but you don't stay at $0 for long, where as CF gives you partial bits but you remain at $0-low/pm for longer
Its actually really ironic the same company that says SAML for all for free, can't even get basic RAC on their own admin controls
This is where AWS wins, AWS gives you everything from $0/month, IAM in the above case, but you don't stay at $0 for long, where as CF gives you partial bits but you remain at $0-low/pm for longer
Its actually really ironic the same company that says SAML for all for free, can't even get basic RAC on their own admin controls
What would you like to see?
Disclaimer: Engineer on the IAM team at Cloudflare
Disclaimer: Engineer on the IAM team at Cloudflare
Just provide an Aws IAM type interface and everyone would be happy. Also don't charge for it, because that just states your systems literally were not designed with authorization in mind.
IAM/RBAC for members of an organization to control who can change settings, deploy scripts, add domains, etc.
I like how Cloudflare is taking a serious run at AWS.
And you can't even subscribe a program to a SNS topic and Cloudflare is giving us pub/sub.
And you can't even subscribe a program to a SNS topic and Cloudflare is giving us pub/sub.
FWIW with AWS, despite the naming, the IoT Core MQTT broker is an excellent product for exactly this - it's just sold as IoT specific. It's really just an MQTT broker at it's core and could probably be split off into it's own product without the IoT branding.
AWS has several "pub/sub" like services: SNS, IoT MQTT, Managed Kafka, Kinesis, EventBridge, etc.
And they're all kinda fucky, like SNS handles a lot but you can't write a program to receive events.
What do you mean by "receive events"? SNS can deliver to any HTTP endpoint.
Then you have to go through the whole process of setting up, routing, and registering an HTTP endpoint, rather than you would as a normal pub/sub simply have the program connect to the publisher and start receiving messages/events.
Pub/sub is an architectural model so there's no real "normal" implementation, but SQS is the answer for that.
SNS is the router, SQS is the storage (for more advanced push/pull consuming).
SNS is the router, SQS is the storage (for more advanced push/pull consuming).
People are butt hurt that SNS doesn't allow the "subscribe" portion. A webhook is a push action.
They forget SQS exists...
They forget SQS exists...
Sounds like a SQS if you want to subscribe. SNS is for push notifications?
Not clear for me if I can `push` messages from my backend to client? At lest for me this is a killer feature.
In the MQTT world, in which Pub/Sub is based, every publisher is a client, and every subscriber is a client. You can be both, too!
Assuming your existing backend is in $SOME_CLOUD_PROVIDER, publishing messages to Pub/Sub is just a few lines of code in your favorite language - we have a couple of examples up in JS + Rust - e.g. https://developers.cloudflare.com/pub-sub/examples/connect-j...
Subscribers receiving those messages can be anywhere you can run an MQTT client, which turns out to be... a lot of places :-)
(I'm the PM for this, and we'll be adding more examples to our docs in the coming weeks)
Assuming your existing backend is in $SOME_CLOUD_PROVIDER, publishing messages to Pub/Sub is just a few lines of code in your favorite language - we have a couple of examples up in JS + Rust - e.g. https://developers.cloudflare.com/pub-sub/examples/connect-j...
Subscribers receiving those messages can be anywhere you can run an MQTT client, which turns out to be... a lot of places :-)
(I'm the PM for this, and we'll be adding more examples to our docs in the coming weeks)
I like that Cloudflare is steadily becoming a serious alternative to the existing cloud providers. If you folks can build a decent IAM tool for access control to Cloudflare resources, much more people will start using your services.
[deleted]
cloudflarer around here ?
Did you wrote the MQTT broker from scratch or are you using some existing broker (Mosquitto , Nanomq, etc). Just curious.
Did you wrote the MQTT broker from scratch or are you using some existing broker (Mosquitto , Nanomq, etc). Just curious.
(I'm the PM for this at Cloudflare & the blog author)
We looked at a number of existing options — trying to strike a balance between languages we run at scale (tooling, ops, security), memory safety, and long-term maintainability as a growing company.
We ended up writing an MQTT broker from scratch in Rust, with our own Durable Objects[1] for consistency and co-ordination across our hundreds of edge locations (and tens of thousands of machines). The MQTT spec itself[2] is very approachable, which made this undertaking easier still.
One of the increasingly cool parts of Cloudflare is that we use more and more of what our customers use: our Broker service sits behind Spectrum[3], and we use Workers + DO for all of the metadata/session handling. The next set of new products will invariably start to build on Pub/Sub itself, too :-)
[1]: https://developers.cloudflare.com/workers/learning/using-dur... [2]: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html [3]: https://developers.cloudflare.com/spectrum/
We looked at a number of existing options — trying to strike a balance between languages we run at scale (tooling, ops, security), memory safety, and long-term maintainability as a growing company.
We ended up writing an MQTT broker from scratch in Rust, with our own Durable Objects[1] for consistency and co-ordination across our hundreds of edge locations (and tens of thousands of machines). The MQTT spec itself[2] is very approachable, which made this undertaking easier still.
One of the increasingly cool parts of Cloudflare is that we use more and more of what our customers use: our Broker service sits behind Spectrum[3], and we use Workers + DO for all of the metadata/session handling. The next set of new products will invariably start to build on Pub/Sub itself, too :-)
[1]: https://developers.cloudflare.com/workers/learning/using-dur... [2]: https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html [3]: https://developers.cloudflare.com/spectrum/
The MQTT spec itself[2] is very approachable, which made this undertaking easier still.
Indeed. I've found writing an MQTT client is a good tutorial project when learning a new language.
Indeed. I've found writing an MQTT client is a good tutorial project when learning a new language.
Some time ago I was at a talk about verifying different MQTT brokers so they fulfill the spec.
They presented several inconsistencies they found (in OSS brokers) and submitted fixes for.
I looked up if there's some info online and found that paper (unfortunately paywalled): https://graz.pure.elsevier.com/en/publications/learning-base...
I looked up if there's some info online and found that paper (unfortunately paywalled): https://graz.pure.elsevier.com/en/publications/learning-base...
The F-Secure folks also have https://github.com/F-Secure/mqtt_fuzz - although it doesn't support authenticated brokers as of yet.
Thanks, make sense.
Actually, I asked because it surprised me that D1 - sqlite - was not written the same way.
[1] https://blog.cloudflare.com/introducing-d1/
Actually, I asked because it surprised me that D1 - sqlite - was not written the same way.
[1] https://blog.cloudflare.com/introducing-d1/
D1 _does_ use Durable Objects under the hood! Like Pub/Sub, it's ultimately an implementation detail, albeit probably interesting to this audience.
>>> And, since we're building on the redundant storage of Durable Objects, your database can physically move locations as needed, resulting in self-healing from even the most catastrophic problems in seconds.
True. I missed that part !
Thanks.
True. I missed that part !
Thanks.
Do you plan to opensource the implementation some time?
No concrete plans at this time, especially since there are many competing implementations.
If we open-sourced we’d need to do it right (proper governance, triage, support, etc) vs just putting into a public repo and calling it a day.
If we open-sourced we’d need to do it right (proper governance, triage, support, etc) vs just putting into a public repo and calling it a day.
huh, Fastly is doing pub/sub lately too with fanout now:
https://www.fastly.com/blog/fastly-fanout-why-real-time-mess...
https://www.fastly.com/blog/fastly-fanout-why-real-time-mess...
Since you guys seem to be using Rust, what client library do you recommend for MQTTv5 support?
(Pub/Sub Eng at Cloudflare)
We are testing the broker using the Paho C wrapper https://github.com/eclipse/paho.mqtt.rust
.. and have added a client example to our dev-docs https://developers.cloudflare.com/pub-sub/examples/connect-r...
We are testing the broker using the Paho C wrapper https://github.com/eclipse/paho.mqtt.rust
.. and have added a client example to our dev-docs https://developers.cloudflare.com/pub-sub/examples/connect-r...
This is such a wonderful news
I love it!! There needs to be more message brokers as a service providers.
How many of these commenters work at Cloudflare? :)
How does this compare to something like Ably?
[deleted]
One question that the example given raises for me, is how to manage that function I see there in a robust, "proper software engineering" sort of way. This has been a challenge for me on AWS when trying out Lambdas. The individual pieces are cool, but the system that connects them all together is a little opaque. How do you version control the code? How do you keep track of which pieces work with which other pieces?
We use terraform to manage the AWS resources, but it's kind of clunky, and not official. Is there some Cloudflare-supported way to maybe declaratively define your different resources?
Someone who has built a somewhat complex app with a lot of workers, and durable objects, and different pieces of the ecosystem: how do you manage that? How do people review changes?
Since it sounds like from another comment that Cloudflare dogfoods its own services internally, I'm curious if there's a write-up about how they actually do so? The little function code snippets give a nice taste of the code a worker might run, but that's still kind of a long way clearing up for me how you would actually use this in practice in a standard production environment.