I think it's just as likely that only founders who fit that behavioral profile tend to do well in YC. Assuming that YC is representive of the world would be a mistake. There are a great many startups created differently outside YC that are very successful.
I also find it hard to imagine a lot of the most successful people I've known fitting this profile. Some maybe, but not most. In fact it's a profile, that as you point out, pg himself certainly does not fit.
The idea that you would make intentionally make misleading suggestions certainly would explain a lot. I was actually expecting that to some degree. I think what confounded me so much is just how ridiculous the statements were and how many there were.
At one point I was reduced to essentially defending the very idea of startups and technology innovation. In the stress of the moment I also assumed like the comments were made sincerely (call me naive) so I did my best to pick my jaw up off the floor and explain why they were wrong -- only to be cutoff half way into a sentence. Rinse. Repeat.
When I say I'm "comfortable debating" I don't mean that I need to be comfortable while debating. I grew up with lawyers in my family and in a household where heated debate was part of our every day life. I've worked with some extremely smart and opinionated people. I love a vigorous debate about topics I'm passionate about (like startups). No one who knows me would say I'm a pushover, and most would agree I'm very open to sound logic and evidence.
I think if there's one thing that felt very different in my YC experience it was the intentionally misleading statements. I really am not accustomed to arguing with people who repeatdly make disengenous statements just to judge how I'll react. It's one thing to take an arbitrary position and defend it, it's another to just throw out random misleading statements. To me that would seem to only muddy the waters.
I'm sure I could learn to play that game. Maybe the best founders are great their first run. For all I know I was judged okay on that front. I don't really know.
I do know though that one would learn a lot more about my ability to think by restricitng the conversation to an honest debate of the relevant issues.
Again. I respect all of you guys and know you're good people and ridiculously smart. I'm not making any judgements on that account. One of the reasons I find YC so amazing is that you don't have to be "someone" to get in. It's not an old boy's club. It is disappointing to me to see that tinge of VC arrogance in the interview process, which is why I point it out.
I think a huge portion of the blame of this is on the YC partners.
I'm extremely comfortable debating and discussing ideas in an extremely logical and thoughtful way. My favorite question every day to myself is "If I was starting over today what would I do". And Joe Kraus's "Face reality".
I found it impossible to do that during my YC interview though. From minute one it was a series of arrogant and condescending statements after another. Their ideas were nuggets of golden insight and my (far more educated in many instances) responses were dismissed out of hand.
I've had countless discussions of the same type with many other smart people (successful founders, investors, regular smart people) before and since. Some who raised far more difficult questions. I have reason to suspect the biggest problem was the YC partners and not some problem I have being flexible or thoughtful.
I think the essay as well as this quote are very revealing:
With the good groups, you can tell that everything you say is being looked at with fresh eyes and even if it's dismissed, it's because of some logical reason e.g. "we already tried that" or "from speaking to our users that isn't what they'd like," etc. Those groups never have that glazed over look.
I think you're mistaking some founders ability to placate you with thoughtfulness in many, but certainly not all, cases.
One person who publicly talked about this phenomenon (in a joking way) was one of the founders of Heroku. He said that pg said "You're an Oracle killer" so he "smiled and nodded because I wanted to get in" (paraphrasing).
I think that's very indicative of the way many founders probably feel when talking to the YC partners. The ones who disregard that idea are likely marked as "difficult to talk to."
I think there's a very real danger you guys aren't hearing stuff like this because who in their right mind would tell you and who would you listen to? You would probably just say they're difficult to talk to.
The problem with being so damn smart (which all the YC partners truly are) is that you can start to think that any problem you run into (like founders who seem hard to talk to) you're assumption is that it's their fault instead of your own.
I do think there's some unbiased truth to the essay and maybe the founders who are good at placating will make better founders. But you have to consider: would Bill Gates or Steve Jobs really have smiled and nodded or would they have said "What the hell are you talking about, that's stupid." (We know what Jobs would have done anyway.)
I'm still a really big fan of YC and all the YC partners, but I do think you might have let some arrogance creep in to your process. You guys are doing alright though, so feel free to ignore little ol' me.
I also find it hard to imagine a lot of the most successful people I've known fitting this profile. Some maybe, but not most. In fact it's a profile, that as you point out, pg himself certainly does not fit.