The idea that speech is not "real-world conduct" is, in my considered opinion, not just wrong, but actively corrosive to the fabric of our civilization. Ideas are powerful, speeches can incite revolutions, send millions to their deaths, or be used to fight tyranny and oppression.
Should we highly value the idea of free speech? I believe we should. But we should also recognize that there's no such thing as "just speech". What we say matters, particularly if, like Dr. Stallman in this case, we are public figures speaking in what (given the number and types of people involved) is a semi-public forum.
There are two well know aphorisms: "the pen is mightier than the sword" and "he who lives by the sword dies by the sword". But if one recalls the last is derived from a line in the Bible that says in whole: “Put your sword back in its place,” Jesus said to him, “for all who draw the sword will die by the sword."
Sometimes, we should put the pen back in its place, lest we kill or die by it in our hubris.