They explicitely mention Question Answering. Could it be that they use something like BERT trained with Squad dataset, and fine tuned on additional content?
If so, Bert is very intense in terms of required GPU hardware...
I would not consider word embeddings to be state of the art anymore.
Word Embeddings are like TF-IDF when word embeddings came out. Have a look at BERT model that just recently got published and is outperforming all kind if NLP tasks with one main Architecture.
I would consider BERT language model two levels higher than word embeddings, as it considers full context sensitive embeddings, dependent on the text left and right of the word in parallel.