@bwasti, really good points - this is something we look forward to evaluating! Our post does indeed outline optimizations from tensorflow/serving to tensorflow/serving:* -devel [1]. The next logical improvement (given intel architecture and docs linked) is start building on top of the * -devel-mkl image.
Hi bwasti, the host's CPU platform is Intel Broadwell. While the CPU architecture of our production hosts are the same, the resources allocated are much higher than 4 cores. This post details an overview of the relative improvements that can be made from a vanilla setup :)
Hey! That's super interesting - so far we went with Tensorflow's ubuntu based official Docker devel image, but a clearlinux base looks like it would definitely be worth looking into!
Hey! Author here, thanks for linking the paper. The article was from an infrastructure perspective, but we're definitely diving deeper into graph execution optimizations after this:)
-masroor(author)
[1] https://github.com/tensorflow/serving/tree/master/tensorflow...