right now, I am scoping it to nearby people; within X miles distance. Saying strangers look fishy - might have to add more verifications.

How open would you be to trying go out?

essekar

·2 miesiące temu·discuss

anyone has any videos?

essekar

·3 miesiące temu·discuss

lol, that is what we all needed. AI Boss

essekar

·3 miesiące temu·discuss

anything tbh. as long as you have runbook - you can try to automate actions through nvsx; it sits on top of NVSentinel. restarting will work mostly for smaller jobs - distributed training, pretty common will need more fault tolerant methods to continue rather than just restarting.

essekar

·3 miesiące temu·discuss

[dead]

essekar

·4 miesiące temu·discuss

One of the feedbacks we got while testing was - we might even reduce the duration of checkpointing. Which was a huge insight.

We are testing out & would love more collaboration from the community - if your team is running training jobs, hit us up.