hyperknot

There was no cache write before!

https://openrouter.ai/openai/gpt-5.5?endpoint=58e5b336-423e-...

vs

https://openrouter.ai/openai/gpt-5.6-sol?endpoint=a54c5de0-8...

hyperknot

dündiscuss

> GPT‑5.6 also introduces more predictable prompt caching, including support for explicit cache breakpoints (opens in a new window) and a 30-minute minimum cache life.

Great to read they are moving away from the 5 minute cache defaults. Hopefully other providers follow soon!

hyperknot

2 ay öncediscuss

Add this to fix the contrast in all your dark themes:

  "experimental.theme_overrides": {
    "background": "#111",
    "panel.background": "#111",
    "text": "#eee",
    "text.muted": "#ccc",
    "title_bar.background": "#111",
    "tab_bar.background": "#111",
    "toolbar.background": "#111",
    "status_bar.background": "#111",
    "element.selected": "#333",
    "editor.background": "#000",
    "editor.gutter.background": "#000"
  }

hyperknot

9 ay öncediscuss

Use a simple block rule, not a WAF rule, those are free.

hyperknot

11 ay öncediscuss

Yes, wplace could solve their whole need by a single, custom-built static pmtile. No need to serve 150 GB of OSM data for their use-case.

hyperknot

11 ay öncediscuss

You are correct. I was putting "a caching server on their side" in the context of their side being a single dev hobby project running on a VPS, exploding on the weekend. I agree that these servers do exist and some companies do pay for this bandwidth as part of their normal operations.

hyperknot

11 ay öncediscuss

I'm sure the server hardware is not a problem. The full dataset is 150 GB and the server has 64 GB RAM, most of which will be never requested. So I'm sure that the used tiles would actually get served from OS cache. If not, it's on a RAID 0 NVME SSD, connected locally.

What I've been referring to is the fact that even unlimited 1 Gbps connections can be quite expensive, now try to find a 2x40 gig connection for a reasonable money. That one user generated 200 TB in 24 hours! I have no idea about bandwidth pricing, but I bet it ain't cheap to serve that.

hyperknot

11 ay öncediscuss

OK, technically there might exist such server, I guess Netflix and friends are using those. But we are talking about a community supported, free service here. Hetzner servers are my only options, because of their unmetered bandwidth.

hyperknot

11 ay öncediscuss

They are actually static files. There is just too many of them, about 300 million. You cannot put that in Pages.

hyperknot

11 ay öncediscuss

Yes, that's a good idea. But we are talking about 90+% of the titles being empty (I might be wrong on that), that's a lot of hard links. I think the nginx config just need to be fixed, I hope I'll receive some help on their forum.

hyperknot

11 ay öncediscuss

I've just written my question to the nginx community forum, after a lengthy debugging session with multiple LLMs. Right now, I believe it was the combination of multi_accept + open_file_cache > worker_rlimit_nofile.

https://community.nginx.org/t/too-many-open-files-at-1000-re...

Also, the servers were doing 200 Mbps, so I couldn't have kept up _much_ longer, no matter the limits.

hyperknot

11 ay öncediscuss

We are talking about an insane amount of data here. It was 56 Gbit/s (or 56 x 1 Gbit servers 100% saturated!). This is not something a "caching server" could handle. We are talking on the order of CDN networks, like Cloudflare, to be able to handle this.

hyperknot

11 ay öncediscuss

Yes, I designed the whole path structure / location blocks with caching in mind. Here is the generated nginx.conf, if you are interested:

https://github.com/hyperknot/openfreemap/blob/main/docs/asse...

hyperknot

11 ay öncediscuss

What if one user really wants to browse around the world and explore the map. I remember spending half an hour in Google Earth desktop, just exploring around interesting places.

I think referer based limits are better, this way I can ask high users to please choose self-hosting instead of the public instance.

hyperknot

11 ay öncediscuss

Feel free to migrate. If you ever worry about High Availability, self-hosting is always an option. But I'm working hard on making the public instance as reliable as possible.

hyperknot

geçen yıldiscuss

I got 700+ tokens/sec on o3 after the announcement, I suspect it's very much a quantized version.

https://x.com/hyperknot/status/1932476190608036243

hyperknot

2 yıl öncediscuss

There was also a video where they are resoldering memory chips on gaming grade cards to make them usable for AI workloads.

Submissions

Hetzner Network Fault in NBG1

OpenFreeMap survived 100k requests per second

comments