I think the prompt does add value, but its about wrapping the LLM + Prompt into a journey that works well for the intended use case.
I think the most elegant implementation so far of this is:
- notionai
- blenderGPT
Both are embedded right where you need them, and they are tuned to provide great results.
Another ChatGPT with a better prompt probably isn't much of a moat.
I think the idea behind agents is to have them a bit less constrained.
If you can force input/output with better prompts that might help.
Or if you can reduce the temperature of the underlying LLM generation, that could make it a bit more deterministic.
Both are embedded right where you need them, and they are tuned to provide great results. Another ChatGPT with a better prompt probably isn't much of a moat.