About how they scrape other search engines, it's really simple: HTTP calls and parsing of HTML (for most of them).
In MiniSearch, I don't need to save the results by myself. The scrape is done in real-time by SearXNG and passed to MiniSearch, which in turn runs a similarity search and filters out the textual results that don't seem that useful.
But I can say the real differential of MiniSearch is that it's mobile-first. Since the beginning, it was made to run on the browsers of Chrome/Safari/Firefox Mobile, and [Wllama](https://github.com/ngxson/wllama) together with [Web-LLM](https://github.com/mlc-ai/web-llm), along with LLMs of <1B parameters, allowed it!
Ah! It was first implemented writing only the summary of the link results, as you mentioned, but I noticed the responses were not as good as they are now.
Summarizing each link and then rewriting the response based on them made a noticeable difference for that small-sized model.
But I'll take note of that, as it can indeed be added as a configurable setting.
Regarding the search engine, currently, there's no way to use a different one. At that point, the only search engine I knew could work was DuckDuckGo, but after the suggestion from @hackideiomat in the comments, I already started looking into adding it as a customization.
I have previously been impressed by https://github.com/0hq/WebGPT. Now, your project has rekindled hopes for a pure-JS GPT-like model.
Keep it up!