I think most people don't actually want to make games.
Most people want to take an existing basic game, and tweek it a bit, incrementally, adding new art, ideas and levels, and removing the stuff they don't like.
They definitely do not want to write code.
They most certainly do not want to debug generated code and I think you've hit the nail on the head with 'generating actual code sucks, is hard and basically doesn't work' with current models.
What I see in the demos looks different though; it looks like you have a configuration driven set of basic game elements and let the AI tweek the structured and easily-to-validate-schema config.
Which I think is fantastic.
"Make this unit bigger", "Add new type of fighter", "Get rid of this unit", "make the terrain here smoother and add a lake"; I feel like this kind of structured tool use is currently possible, if you write a framework of constrained actions (which is what it appears you've done).
So, props to you. This feels like something that is both actually novel and interesting.
...but; I feel there is going to be a difficult path ahead.
The more generic you make the engine, the more different types of games you can make with it; however, in doing so, the tools you have (eg. "modify unit config", "generate 2d sprite for unit", etc.) will have to become more generic; and the more generic they become, the less effective the agents will be at using them.
Our current batch of GPT4-ish models seems like they're relatively well suited for using specific tools with specifically constrained inputs to plan and achieve goals; however, as the tools become generic, the solution space balloons out widely and you start getting random crap instead of actual real solutions to tasks.
Specifically, I'm skeptical about your generic `createOrUpdateRule` and that you can effectively scale it to complex interactive behaviors; it just feels like the agents will never express complex ideas with that kind of coding; it'll only ever be an array of trivial behaviors.
Maybe that's all you need in some cases; but I don't think you can build actual games that way.
So, practically speaking, this might actually only ever really work as like a 'super modding tool' that takes basically a fully working game and lets you mod it in a very specific set of ways; but that would mean creating a 'template game' for various different game types, and different 'modify game' tools for each one. Otherwise, the final product is never going to really be beyond 'unit moves back and forth randomly'.
...but, that would still be really fantastic.
I guess, I hope you don't get lost trying too hard to build a generic Unity clone using AI that is kind of so-so (ie. generates random code that doesn't work and you have to constantly debug), instead of an amazing thing that is slightly less generic, but lets you do amazing things within a specific set of restrictions.
Most people want to take an existing basic game, and tweek it a bit, incrementally, adding new art, ideas and levels, and removing the stuff they don't like.
They definitely do not want to write code.
They most certainly do not want to debug generated code and I think you've hit the nail on the head with 'generating actual code sucks, is hard and basically doesn't work' with current models.
What I see in the demos looks different though; it looks like you have a configuration driven set of basic game elements and let the AI tweek the structured and easily-to-validate-schema config.
Which I think is fantastic.
"Make this unit bigger", "Add new type of fighter", "Get rid of this unit", "make the terrain here smoother and add a lake"; I feel like this kind of structured tool use is currently possible, if you write a framework of constrained actions (which is what it appears you've done).
So, props to you. This feels like something that is both actually novel and interesting.
...but; I feel there is going to be a difficult path ahead.
The more generic you make the engine, the more different types of games you can make with it; however, in doing so, the tools you have (eg. "modify unit config", "generate 2d sprite for unit", etc.) will have to become more generic; and the more generic they become, the less effective the agents will be at using them.
Our current batch of GPT4-ish models seems like they're relatively well suited for using specific tools with specifically constrained inputs to plan and achieve goals; however, as the tools become generic, the solution space balloons out widely and you start getting random crap instead of actual real solutions to tasks.
Specifically, I'm skeptical about your generic `createOrUpdateRule` and that you can effectively scale it to complex interactive behaviors; it just feels like the agents will never express complex ideas with that kind of coding; it'll only ever be an array of trivial behaviors.
Maybe that's all you need in some cases; but I don't think you can build actual games that way.
So, practically speaking, this might actually only ever really work as like a 'super modding tool' that takes basically a fully working game and lets you mod it in a very specific set of ways; but that would mean creating a 'template game' for various different game types, and different 'modify game' tools for each one. Otherwise, the final product is never going to really be beyond 'unit moves back and forth randomly'.
...but, that would still be really fantastic.
I guess, I hope you don't get lost trying too hard to build a generic Unity clone using AI that is kind of so-so (ie. generates random code that doesn't work and you have to constantly debug), instead of an amazing thing that is slightly less generic, but lets you do amazing things within a specific set of restrictions.