The L2 cache on the M1 is the last level cache (LLC), where the 5950X has a 64MB L3 cache for LLC. Also I'm not sure we know yet how much of the chip is using that L2, it might be more than just the four high perf cores.
A closer comparison is probably Intel's 10900K which has a 20MB L3.