What a great lunch read! I've been weekend-warrioring a terminal-based CRPG for a bit myself. I was recently exploring ways to use agents to help with balance testing, which is a real scale problem for solo indie dev. So far, all I've created is a fight simulator: essentially, have the current player state (stats, effects, gear, companions, etc.) do this fight, simulated, X number of times using one of the currently-implemented GOAP personalities and report how often it wins, loses, average end turn, stuff like that.
I hadn't really thought about trying to create a harness for agents to play the full game interactively. I'd love to explore this. If you don't mind, here are a few questions:
1) Correct to assume that I probably need a text-only harness even though my game is text-based already because I do make use of menu selections made via arrow-key-and-enter interactions?
2) Do you have prompt recommendations for the type of feedback you have found to be useful? I would guess in your case, the objectives of the game are more clear than an open-world RPG. What dead ends have you run into? Maybe a variety of approaches would be good? One agent tries to fight everything. Another focuses on gaining and completing as many quests as possible?
3) How bad is the token burn doing this? Any optimization strategies you've employed?
I think it would be fantastic to have a reference site for significant, complex projects either developed or substantially extended primarily via agent(s). Every time I look at someone's incredible example of a workflow for handling big context projects, it ends up being a greenfield static microblog example with vague, arm-wavey assertions that it will definitely scale.
A friend convinced me to try AppFuse once. I have a burning, probably un-justified hatred of Eclipse, so I tried the AppFuse + NetBeans tutorial.
That may be the most unproductive I have felt in my entire life. I made a damn login form and it took like 12 pages of tutorial, tons of setup, generating a whole bunch of files that seemed like they could have been done algorithmically because they are always the same, editing like nine different XML files with little bits of glue, and eventually like one or maybe two lines of code.
Then, I got to wait for the WAR to compile each time I wanted to test a change, because the tutorial didn't seem to know about a developer mode which probably exists.
I hate to sound all ranty and partisan, but I could find no way in which that process could be efficient and couldn't imagine anyone doing it voluntarily unless they were ordered or just had no exposure to the rest of the world.
Also, I sort of object to having code stuffed in an eight-folder deep hierarchy where the preceding seven folders contain nothing at all. That's a lot of wasted navigation and clicking time and distraction.