Practical Llama 3 inference in Java(github.com)
github.com
Practical Llama 3 inference in Java
https://github.com/mukel/llama3.java
2 comments
[deleted]
Llama3.java: featuring .GGUF file format support, Q8_0 and Q4_0 quantizations, fast matrix/vector multiplication routines using Java's Vector API; served by a simple CLI with a --chat mode to interact with the Llama 3 models.