gshx · HackerTrans

gshx

no profile record

comments

10 lat temudiscuss

Thanks for the bullets. This is a good list but some of the points like "Easily scalable" and "DB support" are not cheaply available in any runtime and require careful attention to detail as well as domain-specific thinking. IMHO, the JVM already does a lot of heavy-lifting in this regard.

10 lat temudiscuss

This looks very good. At the moment, for the jvm, there are not a lot of features above simply building services using netty and the protobufdecoder/encoder. Similarly, http2 goodness is also already available in netty.

10 lat temudiscuss

This translates exceptionally well to fields other than basketball, as well. I wish this "magically gifted engineer just needing to work 3hrs a day making e'thing look so easy" profile goes away.

10 lat temudiscuss

One big challenge is the expense that is otherwise avoided via usage of connection pooling. Without the hack to keep the backing containers alive, tearing down and restarting them is expensive and wipes out all hot path optimizations made by JIT compilers besides having to reestablish connections and pay the handshake prices. Apart from the simplest of use-cases at this time, imho this is not an efficient model if sustained performance and low latency are of interest.

10 lat temudiscuss

Reminded me of all the hours spent debugging with dbx. Turns out, even after all the yrs, they are still quite similar.

10 lat temudiscuss

Does anyone have real prod scale numbers - even order of magnitude concurrency would be helpful. We're obviously very wary of trying out new and young data stores.

10 lat temudiscuss

Your point is well-taken. This qn comes up often enough for similar use cases and hence I asked if someone has data to support an arg. Considering the criticality of Zk in the larger stack and our success and operational expertise with it, it has to take significant convincing to switch to something like etcd.

The following two paragraphs from the etcd project site seem to hint that they're trying to target overlapping use cases: "Your applications can read and write data into etcd. A simple use-case is to store database connection details or feature flags in etcd as key value pairs. These values can be watched, allowing your app to reconfigure itself when they change.

Advanced uses take advantage of the consistency guarantees to implement database leader elections or do distributed locking across a cluster of workers."

10 lat temudiscuss

Does someone have any benchmarks or comparisons with Zk? We have run it for many yrs without a problem and are very happy with it. Would be interested in hearing from anyone who switched from zk->etcd for distributed locking, presence, leader election type idioms and ran it in prod for a few months/yrs and their takeaways.

10 lat temudiscuss

Care to give some examples how? For me, Eclipse has remained very stable. Maybe we are using them for different languages and/or things.

10 lat temudiscuss

Go-lang is good but not as mature. Other than that, there's C++ and Erlang/Elixir. Not a whole lot of mature choices out there. For most of the critical production grade systems that need to be secure, have to scale and perform predictably, you don't want to blaze the trail.

10 lat temudiscuss

Would also be interested to hear more about the "deployment nightmares with Zookeeper". For us, it has to be one of the most stable pieces of 3rd party server software we run.

10 lat temudiscuss

Very nice, now that we have some good competition in this space - kafka, ddl and wormhole. If only Fb can open src wormhole, there will be some good related systems software to choose from and contribute to.

10 lat temudiscuss

You're right. I was mostly commenting on the common idiomatic ways ppl differentiate streams vs queues. Indeed, it can be used in both scenarios.

10 lat temudiscuss

It can be used as a queue but the bigger benefit is for streaming use cases. One of the key differences, among others, is that streaming assumes somewhat faster consumers as opposed to queueing. There's also the pub-sub use-case which is generally considered separate from that of a queue (considered a point to point transport).

10 lat temudiscuss

I have had good results with Yourkit. And so far in my experience, that is the only profiler that has been able to survive profiling busy services running large heaps. ymmv.

10 lat temudiscuss

Can you cite some examples? Did these companies decide to run containers on vm's or just bare metal? In the latter case, they shouldn't need 3 to 4 times more hardware. Your point on capacity planning is well taken. Capacity will always be finite and especially critical to put an upper bound on if doing container based services with multi-tenancy.

10 lat temudiscuss

There are other concerns to keep in mind, eg. the upgradability of storage to use next gen cards that use 3d nand, etc. I'm sure they thought through a list of concerns before going this route.

10 lat temudiscuss

Not at all, cost/TB of S3 is much higher compared to what you can build provided you know how to build it yourself.

11 lat temudiscuss

The other problem with Lambda is that you cannot keep persistent connections in a connection pool. It is after all, designed for statelessness. This can be considerable cost for doing calls to other business services (http connection pools) or infra services like databases that all maintain persistent connections.

11 lat temudiscuss

Content and programming are still quite limited. There are a few good original shows. Also, they keep adding a lot of movies with poor ratings (rotten tomatoes). There's definitely potential for some disruption in this space. Also, some more competition from others would be nice.