HackerTrans
TopNewTrendsCommentsPastAskShowJobs

bigodanktime

no profile record

comments

bigodanktime
·3 lata temu·discuss
> There is no copy with mmap

You are right, if you are directly modifying the mmaped region. I always internally model my data as staging my changes to be synchronized to the mmaped region, so thats my mistake there.

> the page is either unwritable or CoW.

This is not universally true, or maybe I'm confused on this statement. MAP_SHARED exists, but maybe you are referencing a specific kernels' implementation on how they achieve coherence between file backed shared memory regions in two processes? Im not sure.

> Darwin kernel does though.

Sure we can always point to a kernel that has has implemented some feature or another, which is why I said typically you don't see it.
bigodanktime
·3 lata temu·discuss
I may have parsed your statement incorrectly, but I'm assuming you are talking about the copy of data when using either mmap or File IO (memcpy versus write) Whether you do File IO versus mmap, there's going to be copy. With files, the copy occurs within kernel space with data being copied into the pages in the buffer cache, with mmap the copy occurs in userspace with data being copied into the address space. Swapping can occur in the buffer cache or mmap, this is why so many databases implement their own buffer cache to ensure specific data isn't flushed, leaving them in an inconsistent state.

An advantage of copying in userspace is the ability to use more performant instructions to perform the memcopy, which the kernel does not typically have access to (https://www.mongodb.com/blog/post/getting-storage-engines-re...)
bigodanktime
·3 lata temu·discuss
What do you mean by work. The underlying page cache will keep much of the data actual cached if it's recent. Even databases like PostGreSQL use this to their advantage (https://github.com/postgres/postgres/blob/master/src/backend...).
bigodanktime
·3 lata temu·discuss
I can understand these folks struggling with what mmap is actually doing. But this isn't a new discussion about the qualities of MMAP versus file based IO etc. Although, many of the comments stated are quite wrong.

Related Work on this problem: 1. https://www.mongodb.com/blog/post/getting-storage-engines-re... - talks about developments on MongoDB's backend to use mmap. 2. https://www.pdl.cmu.edu/PDL-FTP/Database/p13-crotty.pdf - Talks about some of the cons of mmap, some I think are not as prevalent due to the existence of low latency, high throughput storage devices. 3. https://www.cs.cit.tum.de/fileadmin/w00cfj/dis/_my_direct_up... - less relevant but related.