Yeah, it's a different idea, but inspired by yours. Good point about scaling: perhaps your whitelist could include selected whitelists of others; you can aggregate whitelists.
You make me think of a way for every user to make their own members only club, by filtering, and they 100% control who is visible in it; others can view through another's filter:
1. every user can config a "whitelist" of users, to personalize HN to only show comments from those users (and access other users via parent links). This is the "filter", of who they "follow".
2. you can set your whitelist to be the whitelist of any other users. This is "subscribing".
Some few users would end up working to maintain detailed whitelists (like, who they "follow"), and most others would just "subscribe" to them. New users would be left out in the cold til one of these services picked them up (like new bands getting signed to music labels?)
No way would pg go for this, but it's interesting to me. It's sort of a variation of subreddits; and a bit like the various newswire services (like reuters etc) being picked up by local newspapers...
Already implemented, but "hidden":
you must be on the link to that comment to see the flag link - eg. to see it on pg's GP comment:
http://news.ycombinator.com/item?id=1852736
Two sets of scores and arrows: one for agree/disagree, the other for high/low quality. Separating the concepts would emphasize the issues as distinct, and also make them easier to vote on. The high "quality" score would act like the current "flag" button, but with a finer control. Although a much more complex UI model, it's maybe worth trying.
I've noticed that if you reply to a mean-spirited comment with a straightforward and dispassionate counter-argument, completely ignoring its rudeness and tone, you get massive upvotes. Transcending meanness also feels really clean; it is neutralized.
By framing the meanness, the site's collegiality is not undermined, but demonstrated.