StackStorm: Event-driven automation(github.com)
github.com
StackStorm: Event-driven automation
https://github.com/StackStorm/st2
34 comments
Every time I need to automate something and I look at these solutions it becomes clear that I can put together a Ruby script to handle it much faster than getting one of these automation solutions up and running.
I agree that writing a ruby script is tempting. Where I disagree with you is that, when you have an organisation of 100s of devs, these scripts become tech debt. You need to have standard operating procedures delivered via one platform, that's when rundeck et al plays an important role.
Never have been able to get management buy in for automation.
I even had one manager tell me that there was value in having someone do it manually instead of automatic, I suspect part of their pay or career progression was based on the size of their team.
I even had one manager tell me that there was value in having someone do it manually instead of automatic, I suspect part of their pay or career progression was based on the size of their team.
That is precisely what I mentioned in my first post. Activating the customer and getting them to value is the biggest challenge these frameworks have. It is hard to buy into a framework from get go.
So part of the problem from a sysadmin/developer perspective here is that these automation solutions are too difficult/time consuming to setup on the side and prove their value to management. They basically can’t start as a one or two person on the side initiative and then grow the way other technologies like introducing version control and CI can.
They basically can’t start as a one or two person on the side initiative and then grow the way other technologies like introducing version control and CI can.
They absolutely can. Find a spare host or create a VM. Write a shell script to run docker, or much better, use "docker compose" to create the setup. Have it run on the quiet for a while and get familiar with it, a little more every day, before using it in production. It's crucial that you are confident about using it, so that when you come to sell it to anyone else, your confidence transfers to them. (Sales, in large part, is about the transfer of feeling.) Once you have got the deployment and backup/restore, and tool-specific critical procedures clear in your mind, you're ready.
It's not a five-minute job; it's about demonstrating your strategic value to management by getting out of the toil loop. But, to be clear, if you feel you don't have the backing of anyone about such things, you certainly have my sympathy. The best way I have found out of the mire is to work on a workflow (create/destroy, backup/restore, LDAP integration) a little a day, until it's there, and then suddenly, your work put in over the days and weeks will be an overnight success.
Good luck!
They absolutely can. Find a spare host or create a VM. Write a shell script to run docker, or much better, use "docker compose" to create the setup. Have it run on the quiet for a while and get familiar with it, a little more every day, before using it in production. It's crucial that you are confident about using it, so that when you come to sell it to anyone else, your confidence transfers to them. (Sales, in large part, is about the transfer of feeling.) Once you have got the deployment and backup/restore, and tool-specific critical procedures clear in your mind, you're ready.
It's not a five-minute job; it's about demonstrating your strategic value to management by getting out of the toil loop. But, to be clear, if you feel you don't have the backing of anyone about such things, you certainly have my sympathy. The best way I have found out of the mire is to work on a workflow (create/destroy, backup/restore, LDAP integration) a little a day, until it's there, and then suddenly, your work put in over the days and weeks will be an overnight success.
Good luck!
The competition is cron and a folder of scrips which provides a productive result from day one.
I am not in an ops role at the moment, but in the past nothing has been able to beat cron. And the main reason is the time required to get automation tools up and running.
I am not in an ops role at the moment, but in the past nothing has been able to beat cron. And the main reason is the time required to get automation tools up and running.
You missed the point of my reply, and simply repeated your position.
Red Hat is working towards a similar feature for anisble automation platform.
https://www.ansible.com/blog/getting-started-with-event-driv...
https://www.ansible.com/blog/getting-started-with-event-driv...
I played around with StackStorm maybe 5 years ago, but in the end I decided to use RunDeck instead. https://www.rundeck.com/. Used that for a few years but ended up abandoning it, but am thinking about resurrecting it, it was pretty good.
Ex-Rundeck engineer here. Would be very interested to here why you initially dropped it and why you are thinking about picking it back up!
It was just that we weren't using Rundeck very effectively. I set it up so that others could run a few Ansible jobs, and so that a slack bot could fire off one of those jobs. The Slack API changed a few years later and our old bot completely broke and I couldn't get it working again without rewriting. As far as other people running them, nobody else seemed to use it. So when it came time to upgrade the EOL OS that was hosting Rundeck, I just shut it down.
But, now we have some Jenkins jobs that would benefit from being isolated from Ansible runs, so Rundeck might be a good way to run those.
But, now we have some Jenkins jobs that would benefit from being isolated from Ansible runs, so Rundeck might be a good way to run those.
This is interesting. It's like IFTTT for ops. The list of adopters is intriguing too. I poked around their blog for a few minutes and it seemed to be targeted more at people who have already adopted StackStorm. It would be great to be able to read some success stories from the people using it.
https://github.com/StackStorm/st2/blob/master/ADOPTERS.md includes a few more examples and use cases with the links.
Oh thanks, somehow I looked right at that and didn't process the fact that it was annotated with case studies and conference talks.
[deleted]
I tested StackStorm sometime ago to confirm you can have cycles in the workflow graph: https://kinoshita.eti.br/2021/10/01/cyclic-workflows-with-cy...
I work with climate and weather workflows, where cycles in workflows are common (ecFlow and Cylc are popular tools that support it).
Haven't checked in a while but I expect StackStorm's Orchesta engine to have even more interesting features now.
I work with climate and weather workflows, where cycles in workflows are common (ecFlow and Cylc are popular tools that support it).
Haven't checked in a while but I expect StackStorm's Orchesta engine to have even more interesting features now.
I wouldn’t trust any tool that claims to have anything to do with security that has a curl pipe to bash install advertised.
One of the maintainers here and I second that.
Other options available:
Other options available:
- deb/rpm packages
- Vagrant
- Docker-compose
- Ansible/Puppet
- Kubernetes/Helm
...probably better idea to adveritse docker container, not everyone wants to have half-rotten corpse of python app spreading shit around your system forever (there is never an uninstall...) just to test something.
...or at least one that fucking works
...or at least one that fucking works
zzz@hydra:~$ curl -sSL https://stackstorm.com/packages/install.sh | bash -s -- --user=st2admin --password=Ch@ngeMe
*** Detected Distro is Debian ***
*** Detected flavor bookworm ***
Unsupported ubuntu codename bookworm. Please use 16.04 (xenial) or Ubuntu 18.04 (bionic) or Ubuntu 20.04 (focal) as base system!
zzz@hydra:/tmp$ ./install.sh --user=st2admin --password=Ch@ngeMe
*** Detected Distro is Debian ***
*** Detected flavor bookworm ***
Downloading deployment script from: https://raw.githubusercontent.com/StackStorm/st2-packages/v3.8/scripts/st2bootstrap-deb.sh...
Running deployment script for st2 ...
OS specific script cmd: bash st2bootstrap-deb.sh --stable --user=st2admin --password=****
[sudo] password for zzz:
`curl | sh` "installs" are not a good look on any package, and should be dropped -- you are absolutely right -- there is never an "uninstall" process.
It's supporting a few OS platforms and require a clean server for installation: https://docs.stackstorm.com/install/system_requirements.html
Expect to get MongoDB, RabbitMQ and Redis installed as dependencies.
Expect to get MongoDB, RabbitMQ and Redis installed as dependencies.
I wouldn't recommend it as a security tool
Works well for DevOps and SRE operations and like K8s, needs a whole team to run it
Works well for DevOps and SRE operations and like K8s, needs a whole team to run it
I've tried on clean account
It is curl|sh that also requires sudo
It is curl|sh that also requires sudo
If you're looking for a similar, commercially supported product, checkout RackN Digital Rebar: https://www.rackn.com. Similar concept, vastly easier deployment/HA/GitOps/multi-tenancy story.
StackStorm was an open-core project with a paid enterprise offering on top of it, belonging to a single vendor.
In 2019 the project was donated to the Linux Foundation as a neutral umbrella, also open-sourcing the enterprise features like RBAC, LDAP, UI Workflow Designer.
Today StackStorm has an open-source ecosysem with several partner companies (5) providing consulting, commercial support, custom solutions or training for the clients that may need it: https://stackstorm.com/partners/
These partners also contribute back to the core and help supporting the project together with the other maintainers.
Today StackStorm has an open-source ecosysem with several partner companies (5) providing consulting, commercial support, custom solutions or training for the clients that may need it: https://stackstorm.com/partners/
These partners also contribute back to the core and help supporting the project together with the other maintainers.
How it compares to:
- Prefect.io
- Apache Airflow
- Ansible Tower
- Prefect.io
- Apache Airflow
- Ansible Tower
Prefect and Apache Airflow defines logic in a pure Python code, while StackStorm uses a YAML-structured Workflow called Orquesta [1] supporting different logic patterns (parallel, merge, concurrency, join, retries, pause, ask, nested subworkflows, etc) [2] with Jinja and YAQL [3] support. YAQL is especially interesting one where Jinja is not enough.
There's also a bi-directional ChatOps framework you can tie with the workflows to make the experience more interactive.
So you can listen on events (could be a chat command) from 3rd party services, do some rule filtering & matching (all in YAML), and trigger workflows with the logic that runs actions in other tool(s) and service(s).
StackStorm has Exchange [4], - a collection of 100+ plugin integrations with different tools and services contributed and supported by community to make that building blocks juggling easier.
[1] https://docs.stackstorm.com/orquesta/index.html
[2] https://github.com/StackStorm/st2/tree/master/contrib/exampl...
[3] https://yaql.readthedocs.io/en/latest/
[4] https://exchange.stackstorm.org/
There's also a bi-directional ChatOps framework you can tie with the workflows to make the experience more interactive.
So you can listen on events (could be a chat command) from 3rd party services, do some rule filtering & matching (all in YAML), and trigger workflows with the logic that runs actions in other tool(s) and service(s).
StackStorm has Exchange [4], - a collection of 100+ plugin integrations with different tools and services contributed and supported by community to make that building blocks juggling easier.
[1] https://docs.stackstorm.com/orquesta/index.html
[2] https://github.com/StackStorm/st2/tree/master/contrib/exampl...
[3] https://yaql.readthedocs.io/en/latest/
[4] https://exchange.stackstorm.org/
Nice to see this on here.
I founded a startup that does something similar for Kubernetes:
http://github.com/robusta-dev/robusta
We've seen a bit of StackStorm too with our users!
I founded a startup that does something similar for Kubernetes:
http://github.com/robusta-dev/robusta
We've seen a bit of StackStorm too with our users!
Interesting. I saw this tweet (https://twitter.com/RobustaDev/status/1595767655687913473) last night, and I'm not sure that the image of people falling out of an airplane is in particularly good taste. It put me off your company and product immediately.
I love the design, but it just seems mostly abandoned. Maybe it needs better marketing?
One of the maintainers during EU late night here.
* The release management team is preparing the v3.8.0 release right now, - coming out soon next week.
* There are 15 Maintainers (https://github.com/StackStorm/st2/blob/master/OWNERS.md) in the Technical Steering Committee (TSC), a very diverse group, but I'd say only 1/3 or half are really active.
* There are monthly TSC meetings and we invite curious to join us: https://github.com/StackStorm/community/issues/33 (next meeting is on 6th Dec).
* Past Meetings: https://github.com/StackStorm/community/issues?q=is%3Aissue+...
* The project is under the neutral Linux Foundation umbrella with no single commercial owner, so probably why less marketing. We have blog/twitter/linkedin with the mostly engineering updates for engineers.
But I agree with the sentiment overall, - the development velocity could be higher and we definitely welcome new contributors and interested folks. The tool is written in Python if someone is willing to join, play with it and start contributing.
* The release management team is preparing the v3.8.0 release right now, - coming out soon next week.
* There are 15 Maintainers (https://github.com/StackStorm/st2/blob/master/OWNERS.md) in the Technical Steering Committee (TSC), a very diverse group, but I'd say only 1/3 or half are really active.
* There are monthly TSC meetings and we invite curious to join us: https://github.com/StackStorm/community/issues/33 (next meeting is on 6th Dec).
* Past Meetings: https://github.com/StackStorm/community/issues?q=is%3Aissue+...
* The project is under the neutral Linux Foundation umbrella with no single commercial owner, so probably why less marketing. We have blog/twitter/linkedin with the mostly engineering updates for engineers.
But I agree with the sentiment overall, - the development velocity could be higher and we definitely welcome new contributors and interested folks. The tool is written in Python if someone is willing to join, play with it and start contributing.
In what sense?
The repo linked here appears to have contributions merged in that last month and it has a bunch of active pull requests.
The repo linked here appears to have contributions merged in that last month and it has a bunch of active pull requests.
Rundeck is another closest competitors that worth mentioning. Things have evolved a lot in the ops space since then.
With advent of container orchestration of the operational automation pieces have also moved to the orchestration layer. There have been quite some new tools in the space I have been tracking for instance Fylament [2] which got acquired by netapp, blinkops etc. Post rendezvous with this domain, some bigger problems most of the major framework tools like this have is that, it is very hard to get started. There are lot of pre-requisites given the nature of the ops automation, this leads to a problem where it is very hard to active a customer. Secondly, these frameworks are always competing with it is bash/python script deployed with some container orchestration or some cronjobs. I have also seen people write CRDs and reconciliation loops in k8s doing lot of the tasks these automation frameworks would do.
[1] https://en.wikipedia.org/wiki/StackStorm
[2] https://www.netapp.com/newsroom/press-releases/news-rel-2022...