Show HN: Execute local LLM prompts in remote SSH shell sessions
2 comments
How does it handle large outputs? If I pipe a big config file through, does the full content travel back to local for LLM processing, or is there any streaming/chunking?
It does travel back to local entirely. It's streamed, although buffered at local before making the completion. I also imagine on the fly compression at remote and decompression at local should be possible
This is a tool I've worked on the past few months.
Instead of giving LLM tools SSH access or installing them on a server, the following command:
makes a set of locally defined prompts "magically" appear within the remote shell as executable command line programs.
For example, I have locally defined prompts for `llm-analyze-config` and `askai`. Then on (any) remote host I can:
the prompts behind `llm-analyze-config` and `askai` execute on my local computer (even though they're invoked remotely) via the llm of my choosing.
This way LLM tools are never granted SSH access to the server, and nothing needs to be installed to the server. In fact, the server does not even need outbound internet connections to be enabled.
Eager to get feedback!
Github: https://github.com/tgalal/promptcmd/