I worked on one of the agents for the first ORTS competition, waaay back in 2006. We were interested in Starcraft (which ORTS was a knockoff of) as a testbed for human-level AI, since ostensibly it has a pretty strong need for covering the entire "stack" from low-level reactions to planning and strategy. The strategic game is also open-ended enough that it seemed it would be especially interesting for testing humanlike AI compared to something like Chess or Go that can be reduced to a state search and heavily optimized.
What ended up happening was that actually winning the game came down to unit pathfinding and micromanagement (as well as plain old "not crashing"), and the planning and strategy part didn't really come out. So we moved on to other things.
I admittedly have not paid attention to this in years, but it looks like the modern game is still very much like that except much more mature, the strategic parts seem to be selecting heavily-optimized strategies from a playbook, making up something new doesn't help win the competition.
Does anyone know of any ongoing AI competitions that push more on the creative/strategic side?
The above like appears to be based on using tenure of current employees to measure "retention". So that is going to heavily skew low for companies that are growing, which makes it unsurprising that Kodak has the best retention and growing tech companies are generally low.
So I wouldn't put much stock in that as a meaningful comparison.
> 2011: Daryl Bem publishes his article, “Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect,” in a top journal in psychology. Not too many people thought Bem had discovered ESP but there was a general impression that his work was basically solid, and thus this was presented as a concern for pscyhology research.
> In retrospect, Bem’s paper had huge, obvious multiple comparisons problems—the editor and his four reviewers just didn’t know what to look for—but back in 2011 we weren’t so good at noticing this sort of thing.
I was a postdoc in a Psychology department when this was going on, and "obvious multiple comparisons problems" isn't a good characterization. Any competent psychology researcher in 2011 (a) understood multiple comparisons and looked for them as a matter of course (b) knew there was something wrong with Bem's paper (see the editorial disclaimer).
That is some pretty advanced statistics, not just "correct for multiple comparisons".
What was ongoing then, and continues now, is that psychology and social science in general is coming around to the realization that the tools of the past 50 years are flawed, and to correct them, they need to become better statisticians. But it isn't a matter of "take stats 101 noobs", these are people who have been doing statistical analysis routinely for years. I think there is anxiety that to really do things right you need to _primarily be_ a statistician.
So there is some defensiveness in social sciences about this, certainly not helped by the fact that every jackass on the internet whose taken an undergrad math class thinks they know better.
In the end I quit my psych research job to be a software engineer since all the stats hurt my head and I needed something less quantitative to do.
I guess I thought this was common. Maybe this development is private universities only? Where I went to school (Michigan) many (most IIRC) of the grad students were in the union:
http://www.geo3550.org/
What ended up happening was that actually winning the game came down to unit pathfinding and micromanagement (as well as plain old "not crashing"), and the planning and strategy part didn't really come out. So we moved on to other things.
I admittedly have not paid attention to this in years, but it looks like the modern game is still very much like that except much more mature, the strategic parts seem to be selecting heavily-optimized strategies from a playbook, making up something new doesn't help win the competition.
Does anyone know of any ongoing AI competitions that push more on the creative/strategic side?