HackerTrans
TopNewTrendsCommentsPastAskShowJobs

fbouvier

no profile record

Submissions

Lightpanda Agent and PandaScript – LLM at buildtime, not runtime

lightpanda.io
8 points·by fbouvier·vor 27 Tagen·0 comments

Show HN: Lightpanda, an open-source headless browser in Zig

github.com
319 points·by fbouvier·letztes Jahr·137 comments

Show HN: Lightpanda, an open-source headless browser in Zig

github.com
7 points·by fbouvier·vor 2 Jahren·0 comments

comments

fbouvier
·vor 27 Tagen·discuss
[flagged]
fbouvier
·vor 6 Monaten·discuss
Keeping some kind of style information is definitely important to understand the semantics of the webpage.
fbouvier
·vor 6 Monaten·discuss
Thanks Steeve!
fbouvier
·vor 6 Monaten·discuss
Yes HTML is too heavy and too expensive for LLM. We are working on a text-based format more suitable for AI.
fbouvier
·vor 6 Monaten·discuss
Thanks for the feedback, we will try to make this clearer on the website. Lightpanda works with Playwright, and we have some docs[1] and examples[2] available.

Web APIs and CDP specifications are huge, so this is still a work in progess. Many websites and scripts already work, while others do not, it really depends on the case. For example, on the CDP side, we are currently working on adding an Accessibility tree implentation.

[1] https://lightpanda.io/docs/quickstart/build-your-first-extra...

[2] https://github.com/lightpanda-io/demo/tree/main/playwright
fbouvier
·vor 6 Monaten·discuss
Hi, I am Francis, founder of Lightpanda. We wrote a full article explaining why we choose Zig over Rust or C++, if you are interested: https://lightpanda.io/blog/posts/why-we-built-lightpanda-in-...

Our goal is to build a headless browser, rather than a general purpose browser like Servo or Chrome. It's already available if you would like to try it: https://lightpanda.io/docs/open-source/installation
fbouvier
·letztes Jahr·discuss
And some lightweight alternatives like Bellard's QuickJS (https://bellard.org/quickjs/) in C and Kiesel (https://kiesel.dev/) in Zig.
fbouvier
·letztes Jahr·discuss
Yes, argentic workflows are one of our use cases for Lightpanda.

We skip the graphical rendering of the web page for instant startup, fast execution and low resources usage.
fbouvier
·letztes Jahr·discuss
It might works if you need to handle a few websites. But this retro engineering approach is not maintainable if you want to handle hundreds or thousands of websites.
fbouvier
·letztes Jahr·discuss
I don't expect it to change a lot. All the main components are there, it's mainly a question of coverage now.
fbouvier
·letztes Jahr·discuss
Not now but we might do it in the future. It's easy to export a Zig project as a C ABI library.
fbouvier
·letztes Jahr·discuss
Thank you! Happy if you use it for your e2e tests in your servers, it's an open-source project!

Of course it's quite easy to spin a local instance of a headless browser for occasional use. But having a production platform is another story (monitoring, maintenance, security and isolation, scalability), so there are business use cases for a managed version.
fbouvier
·letztes Jahr·discuss
It was my first idea. Forking Chromium has obvious advantages (compatibility). But it's not architectured for that. The renderer is everywhere. I'm not saying it's impossible, just that it did look more difficult to me than starting over.

And starting from scratch has other benefits. We own the codebase and thus it's easier for us to add new features like LLM integrations. Plus reducing binary size and startup time, mandatory for embedding it (as a WASM module or as C lib).
fbouvier
·letztes Jahr·discuss
For now we just support CDP. But Selenium is definitely in our roadmap.
fbouvier
·letztes Jahr·discuss
Yes but WebKit is not a browser per se, it's a rendering engine.

It's less resource-intensive than Chrome, but here we are talking orders of magnitude between Lightpanda and Chrome. If you are ~10x faster while using ~10x less RAM you are using ~100x less resources.
fbouvier
·letztes Jahr·discuss
There are a lot of uses cases:

- LLM training (RAG, fine tuning)

- AI agents

- scraping

- SERP

- testing

- any kind of web automation basically

Bot protection of course might be a problem but it depends also on the volume of requests, IP, and other parameters.

AI agents will do more and more actions on behalf of humans in the future and I believe the bot protection mechanism will evolve to include them as legit.
fbouvier
·letztes Jahr·discuss
Thanks! No we haven't worked on bot detection.
fbouvier
·letztes Jahr·discuss
Every tool can be used in a good or bad way, Chrome, Firefox, cURL, etc. It's not the browser who doesn't play nicely, it's the user.

It's the user's responsibility to behave well, like in life :)
fbouvier
·letztes Jahr·discuss
We did not run benchmarks with chrome-headless-shell (aka the old headless mode) but I guess that performance wise it's on the same scale as the new headless mode.
fbouvier
·letztes Jahr·discuss
I fully understand your concern and agree that scrapers shouldn't be hurting web servers.

I don't think they are using our browser :)

But in my opinion, blocking a browser as such is not the right solution. In this case, it's the user who should be blocked, not the browser.