I've often thought that micro-services were more of a software engineering management solution rather than a technical one.
From the perspective of a high-level architect it makes delegating certain types of responsibility a bit easier and potentially reduces the need for communication between teams while maintaining centralized control of the overall system.
However, it always seems to come with such a high overhead that IMHO it's almost never worth doing unless there really isn't another option.
Looking through the pricing page I'm not sure what sort of workload this would make sense for. Just looking at the examples from the pricing page I think I'm getting sticker shock.
Example 1
> For example, your service uses 1 ECS Task, running for 10 minutes (600 seconds) every day for a month (30 days) where each ECS Task uses 1 vCPU and 2GB memory.
> Total vCPU charges = 1 x 1 x 0.00084333 x 600 x 30 = $15.18
> Total memory charges = 1 x 2 x 0.00021167 x 600 x 30 = $7.62
> Monthly Fargate compute charges = $15.18 + $7.62 = $22.80
So the total cost for 5 hours of running time is $22.80? Am I even reading this correctly? If so, what would this be cost effective for?
For large code bases in dynamic languages I think the advantage grows larger over time. Even a modest amount of test coverage will allow you to build new features without breaking existing features.
It's a lot faster to catch bugs when the tests run in your local development environment or in the CI pipeline than to wait until Q/A (hopefully) catches them in end-to-end integration testing and sends it back to you for rework.
The other advantage I've noticed is that the tests can serve as an additional form of documentation. New team members can look through the tests to see how code is supposed to work from the examples there. For some web API projects I've even been able to generate the documentation from the test cases.
Yup, agree. Depending if you are working with a good static type system or with a dynamic language the value of unit tests can vary.
When working with dynamic languages I always end up writing a bunch of unit tests and a bunch of integration tests. I've experimented some with type hinting and static analysis in some dynamic languages but it's not the same as having the compiler make guarantees.
If you want to create a large number of Objects (like for rows in a data set) using __slots__ saves a lot of memory over just using the standard __dict__. I've always used named tuples for this in the past, but this is a nice way to do it.
I think being born white, male, and good at math at this particular time in history is the very definition of luck. I'm definitely playing the game of life on "easy" mode.
That being said I think there is a minimum level of effort, intelligence, and resilience required to take advantage of certain lucky opportunities if they present themselves. When I look at how much some of my peers had to endure and overcome to reach those minimums compared to what I had to do, it's easy to see how lucky I've been.
I really should write a blog post about it :) but right now there isn't one.
Debian packages are easy to work with and we keep them as our primary artifact for any given release.
The number of cluster nodes varies between 10 and 35 depending on the workload at the time.
Nodes join the cluster by contacting one of the three "static" nodes that are not spot instances. If they can talk to at least one of those they get knowledge of the entire cluster. When one of the spot instances is going to be killed there is a notice is the instance metadata. The nodes just watch for that and initiate a normal shutdown if they see it.
Yes we do use ELBs but just as TCP proxies and for TLS termination.
I wonder if there is any possibility of using a newer iteration of Firefox with a configurable number of threads rather than Chrome as the engine for Electron (or something like it). In the browser at least this helps a lot to bring down memory consumption.
Electron does more than just about any other tool to enable building high quality cross platform apps. As someone who runs Linux on the desktop it's quite nice to be able to use the same apps as folks running Mac/Win. I just don't have any computers with less than 16 GB of memory :)
I've been running an Elixir app in production on about 20 nodes for 18 months. We just build Debian packages and deploy using Salt.
Additionally, most of the cluster nodes run on Spot Instances in AWS so they are relatively inexpensive also. When a new instance comes up it connects to the cluster and starts serving requests. When an instance is killed, traffic is routed to the remaining nodes. Works great.
I've had a few sales jobs where I was employed full-time (w2 not 1099) but also had an employment agreement with a defined term. Most often the term was 1 to 3 years, and if you were going to be let go it would happen when the term was over. They just would not renew your contract.
You could still be fired "for cause" but that almost never happened. The employee would almost have to commit a crime to be terminated early.
Of course you could always quit during the term if you wanted, but that meant you would have to pay back the signing bonus received when the contract was signed :)
I really did not mind this structure, but it seems like it's confined to specific industries and outside of those almost no one does it. For example, I've never seen a software dev job with this sort of structure.
I'm not so sure they are separate skills. It's possible to write functions that are easy to test and functions that are hard to test.
I find that the two tasks of writing functions and tests for those functions are closely intertwined. I like for developers to write their own unit tests, and then for Q/A to develop the functional/integration tests from the perspective of the client (machine or human).
It's very difficult to come in after the fact and write unit tests for someone else's code, especially if they weren't thinking about writing testable code.
From the perspective of a high-level architect it makes delegating certain types of responsibility a bit easier and potentially reduces the need for communication between teams while maintaining centralized control of the overall system.
However, it always seems to come with such a high overhead that IMHO it's almost never worth doing unless there really isn't another option.