pizzao·14일 전·discussThe required compute seems a bit high: "We trained all CIFAR-10 models on 1xB200 GPU, and all ImageNet 64×64 models on 8xB200 GPUs. The largest CIFAR-10 model uses 20 B200 hours to train, and the largest ImageNet 64×64 model uses 640 B200 hours"20 B200 hours for CIFAR-10 seems like a lot...
pizzao·28일 전·discussI wonder if there is way local small LLMs can complement each other in away that the sum-total yields a much more performant LLM
pizzao·2개월 전·discussCan someone explain to me what is their "prompting-scaffolding" to make it work ?
20 B200 hours for CIFAR-10 seems like a lot...