Who said only rich people are worried about automation? And about the second point, yeah, of course, the Unabomber was not trying to throw the economy into disarray...
It's not pejorative for you if you are a liberal, for me it is pejorative, I see Luddites as people who damage important parts of production systems and threaten people's lives.
I understand your point of view if you're from a first-world country and you never experienced the terror of having people interfering with the means of production.