IMHO it is not as flawed as the way you present it.
>>> One, the measurement of novelty is bad: it's basically the measure of whether the game exhibits a novel mashup of mechanics according to the BoardGameGeek ontology of mechanics.
>>> This is a terrible definition of novelty, as it ignores all of the other aspects of game design, both in gameplay (interesting systems, challenges, loops) and non-game play (fiction, setting, presentation, etc). Who measures novelty as "mashup of mechanics"?
Using the mashup of mechanics to measure novelty seems logical for two reasons. First, in creativity research, it is quite common to measure novelty as uniqueness or as distinctiveness regarding already existing "solutions." One way to do so is using the combinations of the base elements of the creative output. There is also similar research doing the same with patents. Second, the authors refer to the MDA framework in their method section as an explanation of why they use mechanics. Following this framework, mechanics are not only the core elements of a game but also the only thing a game designer is able to influence - not the dynamics (i.e., run-time behavior, which seems to be "gameplay" in your wording) nor aesthetics (i.e., emotional responses of the player, which seems to be "non-game play" in your wording).
>>> Two, the measure of knowledge diversity is based on BGG reports of how many genres the designers have worked in. This is, again, not only questionable, but also leads to weird effects: imagine knocking Salman Rushdie, JRR Tolkien, or Umberto Eco for having low knowledge diversity because Amazon says they write books in only one or two genres!
Why do you think using genres to measure knowledge diversity is questionable? What is your concern about doing so? Please elaborate if you seem to have such a strong opinion on that.
Again, there is quite some research (esp. in creative industries) about how working in specific genres might influence how you think/understand things, i.e., how you come up with solutions, what solutions you come up with, how you talk about them, and so on.
Regarding the proposed weird effect: you (illogically) compare a single author's knowledge diversity with a team's diversity. However, the authors do not compare a fantasy author against a fantasy/horror author, but how a team with low diversity (3 Tolkiens) would compete against a more diverse team (Rushdie, Tolkien, and Eco - for the sake of the example).
>>> And finally, the main problem is: sure, if you compare hobbyists (who make what they want for fun) with professionals (who are making a product that needs to sell more than a small handful of copies), of course you're going to find that hobbyists are more free to experiment with wild permutations of mechanics. But in doing so, they conflate personal ability with situational constraints. What about professional designers if they were able to work without commercial constraints, like hobbyists?
Although the last question might be interesting to answer, what you describe before is the direct effect of being a hobbyist, resp. professional on creativity. However, being a hobbyist, resp. professional is the moderation effect, hence the paper looks at "how the effect of diversity on creativity changes because being a hobbyist or professional."
All this, however, is described or at least referred to in the paper.
Well, you could, but then you only would have generated something "new," probably without any sense - something bizarre. That's why the authors measure usefulness, too. Only if both - novelty and usefulness - come together, you create something really creative. There is a lot of research about this, especially in the field of psychology.
Nevertheless, it's seems that the more knowledge diversity a team has, the MORE novelty and LESS usefulness. It's all about communication, sharing ideas, interpretation of knowledge (mental models), and self-regulation.
Well, to be fair, the article doesn't say anything about that the chosen set of mechanics (novelty) have to provide fun. Actually, this seems to be the usefulness part of creativity, or as they describe it, how "playable and entertaining" "appropriate observers" perceive the game.
As you already refer to LeBlanc's "8 kinds of fun": he also is one of the authors of the MDA framework - a framework to analyze games - which states, that the mechanics of a game are the only thing game designers can influence.
>>> One, the measurement of novelty is bad: it's basically the measure of whether the game exhibits a novel mashup of mechanics according to the BoardGameGeek ontology of mechanics. >>> This is a terrible definition of novelty, as it ignores all of the other aspects of game design, both in gameplay (interesting systems, challenges, loops) and non-game play (fiction, setting, presentation, etc). Who measures novelty as "mashup of mechanics"?
Using the mashup of mechanics to measure novelty seems logical for two reasons. First, in creativity research, it is quite common to measure novelty as uniqueness or as distinctiveness regarding already existing "solutions." One way to do so is using the combinations of the base elements of the creative output. There is also similar research doing the same with patents. Second, the authors refer to the MDA framework in their method section as an explanation of why they use mechanics. Following this framework, mechanics are not only the core elements of a game but also the only thing a game designer is able to influence - not the dynamics (i.e., run-time behavior, which seems to be "gameplay" in your wording) nor aesthetics (i.e., emotional responses of the player, which seems to be "non-game play" in your wording).
>>> Two, the measure of knowledge diversity is based on BGG reports of how many genres the designers have worked in. This is, again, not only questionable, but also leads to weird effects: imagine knocking Salman Rushdie, JRR Tolkien, or Umberto Eco for having low knowledge diversity because Amazon says they write books in only one or two genres!
Why do you think using genres to measure knowledge diversity is questionable? What is your concern about doing so? Please elaborate if you seem to have such a strong opinion on that.
Again, there is quite some research (esp. in creative industries) about how working in specific genres might influence how you think/understand things, i.e., how you come up with solutions, what solutions you come up with, how you talk about them, and so on.
Regarding the proposed weird effect: you (illogically) compare a single author's knowledge diversity with a team's diversity. However, the authors do not compare a fantasy author against a fantasy/horror author, but how a team with low diversity (3 Tolkiens) would compete against a more diverse team (Rushdie, Tolkien, and Eco - for the sake of the example).
>>> And finally, the main problem is: sure, if you compare hobbyists (who make what they want for fun) with professionals (who are making a product that needs to sell more than a small handful of copies), of course you're going to find that hobbyists are more free to experiment with wild permutations of mechanics. But in doing so, they conflate personal ability with situational constraints. What about professional designers if they were able to work without commercial constraints, like hobbyists?
Although the last question might be interesting to answer, what you describe before is the direct effect of being a hobbyist, resp. professional on creativity. However, being a hobbyist, resp. professional is the moderation effect, hence the paper looks at "how the effect of diversity on creativity changes because being a hobbyist or professional."
All this, however, is described or at least referred to in the paper.