The model does have some limitations (e.g., need for QAT for 4-bit quantization), lack of a C++ runner to execute the model, but parts of the model are promising.
If interested in further discussion, join the conversation on the ExecuTorch discord channel: https://discord.gg/xHxqsD5b
The model does have some limitations (e.g., need for QAT for 4-bit quantization), lack of a C++ runner to execute the model, but parts of the model are promising.
If interested in further discussion, join the conversation on the ExecuTorch discord channel: https://discord.gg/xHxqsD5b