Umm, customary, tongue-in-cheek reference to McCarthy's proposal for a 10 person research team to solve AI in 2 months (over the Summers)[1]. This was ~70 years ago :)
Not saying we're in necessarily the same situation. But it remains difficult to evaluate effort required for actual progress.
Yes, this seems to be a common concern. We're trying to see how best to crisply address it. But yes, Khoj can be used even with your internet turned off
We're still trying to figure out the right balance between configurability and ease of use/maintenance.
The 7B version was a decent enough starting point in terms of what it can answer (and way fewer folks can run a 13B on their machine).
If you really want you can just replace the 7B model file with the 13B one under the ~/.cache/gpt4all directory on your device and it should just work.
Buggy for setup? We've done some improvements and have desktop apps (in beta) too now to simplify this. Feel free to report any issues on the khoj github. I can have a look.
Yes, we don't do optimizations on the query encoding yet. So SBERT just re-encodes the whole query every time. It gets results in <100ms which is good enough for incremental search.
I did create a plugin system, so that a data plugin just has to convert the source data into a standardized intermeditate jsonl format. But this hasn't been documented or extensively tested yet.
Khoj uses Llama 2 7B, 4bit quantized. So it just needs 3.5Gb of RAM (GPU or System) [1].
Khoj and your other apps need more RAM themselves, so practically 8GB of System or GPU RAM should suffice.
Khoj has been tested with CUDA and Metal capable GPUs. So Nvidia and Mac M1+ GPUs should work. I'm think it'll work with AMD GPUs out of the box too but let me know if it doesn't for you? I can look into what needs to be done to get that to work.
[1]: The calculation is [params] * [bytes] GB RAM, so 7 * 0.5 = 3.5Gb
1. AI video watermarks that carry over even if a video of the AI video is taken
2. Cameras that can see AI video watermarks and put an AI video watermark on the videos of any AI videos they take