I used to be like that when I was younger and in earlier stages of my career. As time passed, the pride of achieving the status worn off, as did the novelty of problems I was solving. Many problems turned out to be reoccurring, so I moved on to harder and less mundane problems. I keep doing that to this day - moving away from mundane work to something new and/or more difficult.
At the same time, as I grew older, my social life improved, and I learned to understand humans, so that part became easier.
From my experience as a programmer, while some level of communication is certainly important, most of my time and effort is spent concentrating alone at the task at hand. That's the central part of the job, communication only supports it. Communication is necessary to divide work across the team and exchange experience; it also provides some psychological relief and motivation. While all of these are important, I consider the concentrated mental effort a far more important and difficult part.
> There's no evidence supporting the supposition that there is an innate ability gap.
First, this is a straw man argument: I never argued for "innate ability gap". I argued for "innate affinity", which I understand as (quoting myself) "they do not like working in it".
Second, I never claimed there was evidence to support the correctness of "innate affinity" argument. I only claimed that it is a possibility, and OP should not have ignored it.
Third, there is no consistent evidence supporting "social explanations", and that's why people resist attempts to "change the field to be more welcoming" at the expense of hard-working, deserving white males.
> any argument for innate characteristics would have to explain why the rates started going down in the 1980s despite the field becoming increasingly popular
It's when you talk about young men in kinder gardens that you realize how ridiculous this whole discussion truly is. There will never be 50:50 young men to young women in kinder gardens. Not even 20:80. That much is obvious.
Most young men do not like to be with children. Most young women do not like working in car repair. While none of these claims are sufficiently substantiated in research, if the first can be true, then surely the second one can be, as well?
> why we have any reason to believe that programming is a “masculine” profession
By exclusion: we have checked everything else we could think of and found no other logical explanation for the disparity of sexes in STEM. That doesn't mean women's preference is the true underlying reason, but then, we don't have a better explanation, or even any other explanation consistent with facts. Still, AFAIK, Damore never claimed it was THE reason, he just raised it as a possible and the likeliest explanation - given no other explanation seems to work.
> But in India, the vast majority of teachers are men.
I don't think India is a valid example here, because there is still a lot of inequality in that society. Let's talk about countries on the higher end of the equality spectrum, like Finland or Sweden.
> those fields are cognitively more demanding than commercial software development or, for that matter, undergraduate computer science
... you arrive at
> No cognitive ability or innate affinity explains the degree of disparity in computer science as practiced in industry.
Even if software development is "cognitively less demanding" in every sense (though I'm not convinced there is just one universal kind of cognitive ability), it may still be that women do not possess the "innate affinity" for it - namely, they do not like working in it, preferring other fields instead. To my understanding, there is nothing to contradict this explanation, and it makes perfect sense.
How about the moral imperative "we should treat men and women the same", as opposed to "women are heavily oppressed, we must discriminate in their favor"? Would that imperative be justified by stating a difference?
Ah, you're right, word2vec assigns one vector to each word, as opposed to one vector to each meaning. Then the problem remains: we can't differentiate between homonyms.
> Finally, conceptual search can also help with precision - by matching across all concepts within a document, you can help disambiguate its meaning, when you have words that have multiple meanings
Actually, I have an idea, albeit not without some doubts.
Let x1 be the number of vectors matching A, x2 the number of vectors matching B, etc, till xn. Let c1..cn be a particular selection of vectors. Now my main assumption here is that in order to determine which of these vectors are most often encountered together in the same context [1], our goal is to find j that maximizes sum_{i from 1 to n, i!=j}[d_i], where d_i=(c_j dot c_i) if the dot product is nonnegative, otherwise d_i=0. I'm not sure it's true primarily because I don't know if by summing up these dot products we add apples to apples or apples to oranges.
Then in order to find the best selection of vectors c1..cn we can iterate on every vector v_k matching A and dot v_k with every vector matching B, then pick the maximum m2 (or 0 if it's negative); dot v_k with every vector matching C, then pick the maximum m3; etc. Thus, for k'th iteration we obtain the selection of vectors that maximizes M_1k=sum_i[m_i]. After we're done with all c1 iterations, we pick the best such selection M1=max_k[M_1k]. This is all done in O(x1(x2+x3+...xn)) time.
Next, we repeat the above process for all x2 vectors matching B and obtain M2, etc, etc. Ultimately, we pick the selection of vectors that produced the highest M_t across all choices of t. Overall, we get O((sum_i[xi])^2), which seems fast enough. What do you think?
[1] One obvious problem is this limits the number of contexts we match against to just one.
Also, this makes me wonder what other things you can do with vectors. If you compute dot product between a verb or a noun with vector "singular"-"plural", will it give a positive value for plurals and a negative for singulars (or vice versa)?
Thanks! That paper is extremely helpful. Still, there is one thing missing to complete the picture for me right now. At the input, I have a list of words that I want to index or query. When indexing, they usually form a sentence, when querying, they might just be keywords. But in both cases the words will usually be selected by the user/author in such a way that a human that reads all the words from the input together is able to disambiguate the meaning of every word. This is precisely what I'm missing.
Let's say the user entered three words: A B C. You look up each of them among the vectors and discover that there are three matching vectors for A, four for B and five for C (and for the sake of generality let's assume that there are more words than just 3 in the input, so it's impractical to test every subset of these words for co-occurrence). How do you jointly select the correct vector for each of the words?