the problem is that on noisy data, that adjustment of the variables isn't really as statistically clean as maths would say.
it's the same with red meat causes cancer studies. they say they remove smoking and lack of exercise and still wave the statistical significance.
it's very questionable how they removed the influence of these variables, since meat eaters do smoke way more, and do not exercise, and care about their health less.
kudos for citing Searn, it's one of the greatest joint prediction and learning algorithms that started a whole range of useful learning to search approaches.
LOLS is even better. all approaches inspired by the reinforcement learning but use the information from already present data to imitate the constructed policy.
training neural networks with the approach of LOLS could make so many tasks reachable.
one could probably even imitate a komodo 9 algorithm with it.
I was a little bit disappointed with the content feed.
I was getting spammed by the same authors (I eventually blocked some authors which was a shame), or by the types of questions I didn't find interesting.
When I picked a category as general as Food, I would constantly get questions and answers that weren't interesting and I couldn't find a way to say that it wasn't interesting.
Eventually I left, although I've made hundreds of comments, questions and answers. Just didn't enjoy the time there anymore.
It would be nice if there was some effort on personalizing. Currently, everything mentioned seems to be about content and not the user.
the idea is based on the incredible incremental perceptron coupled with beam search for having different hypotheses at the same time trying to find a policy which will build them and select the best one
The approach to structured prediction that was the fastest at the time was that incremental perceptron and the implementation extremely simple.
Some other approaches includes conditional random fields, hidden markov models, maximum margin markov networks, maximum entropy markov model and the approach they cite called SEARN (learning to search method - DAgger (Ross et al., 2011) and AggreVaTe (Ross & Bagnell, 2014) and LOLS (Chang et al., 2015)).
Although, the syntaxnet team incorrectly dismisses SEARN (implying it suffers from label bias) but it isn't justified.
Their approach approximates 0/1 (hamming) loss with log-loss and works only on problems where the output decomposes over a sequence of decisions where each decision can have that incremental loss.
LOLS can work on arbitrary loss functions. For example, lets say you are translating a text from english to chinese, how can you say what the loss is when you've translated half a document? It's hard if not impossible to decompose the loss over decisions. (It's easy for part-of-speech tagging, the tag is either correct or not, or for dependency parsing, either the parent word is correct or not)
LOLS is a much superior method. It can also be combined with any binary classifier - SVM with any kernel, perceptron, logistic regression, to guide the decisions.
SyntaxNet uses a non-recurrent network to find the best parsing policy and the same network can be used on the LOLS approach.
Problem of structured prediction is incredible and extremely interesting. For example, doing POS tagging and dependency parsing at the same time can increase the performance on both task, same would be accomplished if one recognizes the named entities and at the same time tries to extract relationship between them.
It's very nice to see past insights applied with heavier machinery. Exciting times!
it's the same with red meat causes cancer studies. they say they remove smoking and lack of exercise and still wave the statistical significance.
it's very questionable how they removed the influence of these variables, since meat eaters do smoke way more, and do not exercise, and care about their health less.