Ask HK: Does it feel like the top models routinely dismiss your opinion/advice?
I find that the latest SOTA modals blatantly ignore some of my instructions or requests, sometimes even after mentioning or requesting it multiple times? I suspect its a consequence of the creators of these's drive for autonomous and completely agentic workflow without human in the loop? I prefer to pair-program and not leave everything to the agent, and this is why I noticed it (or at least feels like its happening).
3 comments
No.
lol... nice
To give a real answer, here are some thoughts.
When I am writing good prompts that make sense, I mostly get good results. When I don't understand what I'm doing and write prompts that don't really make sense often models flounder and act strange and inconsistently.
I don't think a lot of people do it, but I will write up a paragraph about a programming task, explaining what I want done, what concerns I have, and mentioning other examples in the code where something similar was done. I usually end with something like
"Let's talk this through before you start coding. Does this make sense? Do you have any questions for me?"
and go back and forth until we are all happy. I treat it like a precocious junior programmer.
Another thing is that agents tend to get confused as the conversation continues and once you get into the place where you're getting frustrated and the model is going in circles the right thing to do is regroup and start a new conversation, maybe cutting and pasting some highlights from the last conversation.
Like you might have 10 little coding tasks and think "it will go better if it remembers the previous context" and maybe that is true if you batch 2 or 3 related tasks, but if you try all 10 tasks it may be getting "tired" or "confused" halfway through. So it's not a bad plan to just start a new conversation for every little task unless it's the kind of thing like "You did a great job with that last patch, now let's make a similar patch to this other part of the system"
I could say doing that I rarely get into a situation where the model gets dazed and confused, except when I'm starting out confused!
When I am writing good prompts that make sense, I mostly get good results. When I don't understand what I'm doing and write prompts that don't really make sense often models flounder and act strange and inconsistently.
I don't think a lot of people do it, but I will write up a paragraph about a programming task, explaining what I want done, what concerns I have, and mentioning other examples in the code where something similar was done. I usually end with something like
"Let's talk this through before you start coding. Does this make sense? Do you have any questions for me?"
and go back and forth until we are all happy. I treat it like a precocious junior programmer.
Another thing is that agents tend to get confused as the conversation continues and once you get into the place where you're getting frustrated and the model is going in circles the right thing to do is regroup and start a new conversation, maybe cutting and pasting some highlights from the last conversation.
Like you might have 10 little coding tasks and think "it will go better if it remembers the previous context" and maybe that is true if you batch 2 or 3 related tasks, but if you try all 10 tasks it may be getting "tired" or "confused" halfway through. So it's not a bad plan to just start a new conversation for every little task unless it's the kind of thing like "You did a great job with that last patch, now let's make a similar patch to this other part of the system"
I could say doing that I rarely get into a situation where the model gets dazed and confused, except when I'm starting out confused!