blaz0 · HackerTrans

2 tháng trướcdiscuss

macOS actually has an excellent SMB client, so the options actually are: WebDAV, NFS (3.0 and 4.0), SMB, FSKit.

5 tháng trướcdiscuss

This will make it easier to run automated tests in the Android emulator in CI using regular runners. It was a pain dealing with bare-metal instances just for that.

blaz0

9 tháng trướcdiscuss

There is also an identically named VFS, this time from the Perforce company itself [1]

[1] https://help.perforce.com/helix-core/server-apps/p4vfs/curre...

blaz0

9 tháng trướcdiscuss

Yes, games are a common case where your repo can be very large but building your code only requires access to a small subset of it.

For example, the League of Legends source repo is millions of files and hundreds of GB in size, because it includes things like game assets, vendored compiler toolchains for all of our target platforms, etc. But to compile the game code, only about 15,000 files and 600MB of data are needed from the repo.

That means 99% of the repo is not needed at all for building the code, and that is why we are seeing a lot of success using VFS-based tech like the one described in this blog. In this case, we built our own virtual filesystem for source code based on our existing content-defined patching tech (which we wrote about a while ago [1]). It's similar to Meta's EdenFS in that we built it on top of the ProjFS API on Windows and NFSv3 on macOS and Linux. We can mount a view into the multimillion-file repo in 3 seconds, and file data (which is compressed and deduplicated and served through a CDN) is downloaded transparently when a process requests it. We use a normal caching build system to actually run the build, in our case FASTBuild.

I recently timed it, and I can go from having nothing at all on disk to having locally built versions of the League of Legends game client and server in 20 seconds on a 32-core machine. This is with 100% cache hits, similar to the build timings mentioned in the article.

[1] https://technology.riotgames.com/news/supercharging-data-del...

blaz0

10 tháng trướcdiscuss

I'm the author of the GitHub issue that the blog links to, and I'd like to thank Stefan for quickly acknowledging the problem and addressing the issue! I try to keep one of our internal applications up to date with the latest libcurl version within a day or two of a release, so we sometimes hit fresh problems while running our battery of tests.

Ironically, our application has also struggled with blocking DNS resolution in the past, so I appreciate the discussion here. In case anyone is interested, here is a quick reference of the different asynchronous DNS resolution methods that you can use in a native code application for some of the most popular platforms:

  - Windows/Xbox: GetAddrInfoExW / GetAddrInfoExCancel
  - macOS/iOS: CFHostStartInfoResolution / CFHostCancelInfoResolution
  - Linux (glibc): getaddrinfo_a / gai_cancel
  - Android: android.net.DnsResolver.query (requires the use of JNI)
  - PS5: proprietary DNS resolver API

comments