You can absolutely serve with Keras if your inference server is in Python. For instance, if you're looking for a basic solution, you can just set up a Flask app that calls `predict()` on a Keras model.
If you're looking for a high-performance solution that is entirely Python-free, then you can simply export your Keras model as a TF SavedModel and serve it via TFServing. TFServing is C++ based and works on both CPU and GPU.
We made sure that TFLite workflows would run smoothly with Keras 3 models. We did not come up with any TFLite related improvements. The focus was on the multi-backend architecture, distribution, and training performance.
Otherwise, if you want to reuse an existing LLM (or just see how a large one would be implemented in practice) you can check out the models from KerasNLP. For instance, this is BERT, basically just a stack of TransformerEncoders. https://github.com/keras-team/keras-nlp/blob/master/keras_nl...
We don't have a separate `ops.linalg` package, but we do include `numpy.linalg` ops as part of `keras.ops`. For now only 2 ops are supported: `qr` and `solve`. We're open to adding any `numpy.linalg` op that turns out to be useful (or you could open a PR for any op you need).
So far the export story focuses on SavedModel and the services that consume that format, e.g. TFLite, TFjs and TFServing. You can just do `model.export(path)`, and you also have access to the `ExportArchive` class for fine-grained configuration.
We have not tried CoreML export yet.
PyTorch `.compile()` works with Keras models. It may not necessarily result in a speedup however.
Both Keras models/layers (with the PyTorch backend) and Lightning Modules are PyTorch Modules, so they should be able to interoperate with each other in a PyTorch workflow. We have not tried this with Lightning, but we've had a good experience with custom PyTorch Modules.
More broadly, it's feasible to use Keras components with any framework built on PyTorch or JAX in the sense that it's always possible to write "adapter layers" that wrap a Keras layer and make it usable by another framework, or the other way around. We have folks doing this to use Flax components (from JAX) as Keras layers, and inversely, to use Keras layers as Flax Modules.
According to PyPI downloads and user surveys (like the yearly StackOverflow survey) the two main frameworks are TensorFlow and PyTorch for Deep Learning, and Scikit-Learn for classical ML.
This means that the API, the abstractions, the workflows are battle-tested.
The codebase itself went through 2 months of private beta and 5 months of public beta. It is already used in production by several companies. It's not as battle tested as older frameworks, but it's fairly reliable.
To clarify, I have never attacked PyTorch, on Twitter or otherwise.
What happened is that I was a target of online harassment campaign from 2017 to January 2021 (when it stopped abruptly), which originated from a PyTorch developer and took the form of frequent anonymous emails (or messages on social networks) with insults and threats. The thread you link to below is me complaining about the harassment campaign and calling for civility.
Some frameworks fans can be extremely toxic, e.g. creating anonymous accounts on HN for the purpose of bashing one framework or one person.
That's right, if the model is backend-agnostic you can train it with a PyTorch training loop and then reload it and use it with TF ecosystem tools, like serve it with TF-Serving or export it to Coral TPU.
Yes, you can check out KerasCV and KerasNLP which host pretrained models like ResNet, BERT, and many more. They run on all backends as of the latest releases (today), and converting them to be backend-agnostic was pretty smooth! It took a couple of weeks to convert the whole packages.
For a Keras Core model to be usable with the TF Serving ecosystem, it must be implemented either via Keras APIs (Keras layers and Keras ops) or via TF APIs.
To use pretrained models, you can take a look at KerasCV and KerasNLP, they have all the classics, like BERT, T5, OPT, Whisper, StableDiffusion, EfficientNet, YOLOv8, etc. They're adding new models regularly.
If you're looking for a high-performance solution that is entirely Python-free, then you can simply export your Keras model as a TF SavedModel and serve it via TFServing. TFServing is C++ based and works on both CPU and GPU.