We do have reference optimizers implemented for use on our system and available in the `cerebras_pytorch` package, but this isn't because those are the only ones supported; instead, no vanilla pytorch optimizer is currently `torch.compile()` compatible. The main difference is that we pre-initialize the optimizer state instead of doing it lazily in the first `step()`
You can read more here https://docs.cerebras.net/en/latest/wsc/tutorials/custom-opt...