essekar

right now, I am scoping it to nearby people; within X miles distance. Saying strangers look fishy - might have to add more verifications.

How open would you be to trying go out?

essekar

2 เดือนที่ผ่านมาdiscuss

anyone has any videos?

essekar

3 เดือนที่ผ่านมาdiscuss

lol, that is what we all needed. AI Boss

essekar

3 เดือนที่ผ่านมาdiscuss

anything tbh. as long as you have runbook - you can try to automate actions through nvsx; it sits on top of NVSentinel. restarting will work mostly for smaller jobs - distributed training, pretty common will need more fault tolerant methods to continue rather than just restarting.

essekar

3 เดือนที่ผ่านมาdiscuss

[dead]

essekar

4 เดือนที่ผ่านมาdiscuss

One of the feedbacks we got while testing was - we might even reduce the duration of checkpointing. Which was a huge insight.

We are testing out & would love more collaboration from the community - if your team is running training jobs, hit us up.