This is from today apr 5 saying the mmap change loads twice as big models with x100 speed up - is this not a blatant lie?
Wasn’t it discovered last week that loading larger models was an error in measurement and the speed up was from keeping things in memory after the first loading?
Was that implementation confirmed to work or was it a bug?
I saw people saying it only loads the weights necessary, but all the weights are necessary (unless the architecture is heavily modified). Does it stream or load only the currently necessary layers in a way?