Agent Implementing in Plan Mode

[logar16]

Describe the bug

AI was in Plan mode and was editing files in the repo, not just the plan file. It implemented the entire feature in Plan mode. I think I may have shifted it into plan mode right after I asked a question to prompt it but technically still after it got the request.

More detailed: I was having a conversation with the AI after it had implemented a feature. I wanted it to really think through the next steps to make the best thing and have a structured plan as it would be relatively involved. I posed the problem I wanted it to solve and hit enter and then switched it into plan mode. I came back to see it's plan after a minute and saw it deep in the implementation. It made 13 significant edits. Here was some of its reasoning:

Thinking...

● Read/Grep/Explore [lots of files]

● Good. Now I have the full picture. Let me implement the data store. I think function-based getData/setData is the most robust
  approach since it works seamlessly across the QuickJS bridge for both own entity and other in callbacks.

● Todo added: Add per-entity data store

● First, add to the Entity interface:

● Edit world-builder\src\types\Entity.ts (+9)
...

It doesn't seem to know that the mode has changed and it seems we don't enforce any editing limits either. Is this because I was in /yolo mode?

Funny thing is that I didn't change the mode after it implemented (still in plan) and I asked a follow up question about what it implemented and then it acted all in plan mode, made a big plan for this little detail I was curious about. So seems like it was just super delayed.

Affected version

No response

Steps to reproduce the behavior

1. Have agent do some editing in normal agent mode
2. Give it a task that takes some planning. Submit
3. Immediately switch to planning mode (shift + tab)
4. Watch if agent starts doing implementation or just does planning

Expected behavior

I expected that if I switched to planning mode the agent would be updated of the change at least after it finished it's initial first round of thinking and before it started reading tons of files. It even pauses and says it has the full picture. I'd expect at this point it would be notified it was in Plan mode. At the very least, when it tried to start editing non-markdown files it would have been rebuffed with some sort of warning like "You are in Plan mode, don't implement!" and also its instructions would be swapped out so it is in more planning mode.

Additional context

• VS Code Terminal

[laeubi]

Funny thing is that I didn't change the mode after it implemented (still in plan) and I asked a follow up question about what it implemented and then it acted all in plan mode, made a big plan for this little detail I was curious about.

What is sometimes NOT very funny, because this can happen the other way round if explicitly NOT in plan mode and then copilot is not implementing anything see this:

• Copilot is too lazy sometimes #1495

So I describe what needs to be implemented but copilot thing its better to make a plan on its own and then do nothing :-\

So it seems to be sometimes confused what is the best "mode" to operate, in your case maybe it things your task to simple for a plan and just started right away :-D

Stupid enough I today told it to create a plan and create a markdown file with explanation and the plan for further implementation -> resulting in it doing a plan to make the plan then asking me if it should now implement the plan by creating the markdown file with the plan! 🤦🤦