I had the exact same experience and have recently given up. Tried to lift less, eat better, etc. No matter what, I would randomly get really exhausted for 3-4 days. It just wasn't sustainable.
My new years resolution is to quit lifting and find any exercise routine that's sustainable and doesn't drain my energy.