"Letting a person speak" isn't the only impact of inviting that person to your event, particularly an event that facilitates social gatherings and an ongoing dialogue between its attendees. If we just wanted to exchange information as efficiently as possible, we wouldn't bother with the time and expense of in-person conferences. The event provides a social context, and that's why we put so much monetary and cultural value on attendance in comparison to, say, sitting at home and reading an academic paper or a blog post.