This is the way I build cybersecurity opertions centers and ingest data for mining.
Every log structure is analyzed by hand.
Every log structure data behaviour is verified with statistics by hand.
Every log data is normalized by hand.
Every parser is done by hand.
Every log is documented and gets unit tests.
What has not gone through this process ends up in the "automatic extraction" bucket waiting to get human love.
April's fool or not, you may laugh at me.
My L3 security analysts don't.
Every log structure is analyzed by hand. Every log structure data behaviour is verified with statistics by hand. Every log data is normalized by hand. Every parser is done by hand. Every log is documented and gets unit tests.
What has not gone through this process ends up in the "automatic extraction" bucket waiting to get human love.
April's fool or not, you may laugh at me. My L3 security analysts don't.