Enhance Agentic AI: Plan Editing & Approval Workflow

by Alex Johnson 53 views

Summary

This article discusses a critical feature request to enhance the agentic AI and CortexON platforms: implementing a plan approval workflow with a feedback loop. This improvement will allow users to review, modify (via feedback), approve, or cancel generated plans before execution begins. This workflow is crucial for ensuring user control, safety, efficiency, and a better overall user experience.

The Problem: Lack of User Control in Plan Execution

Currently, the process within these systems is linear and lacks essential user oversight. When a user submits a task, the planner agent generates a plan, and this plan is immediately executed without any opportunity for user review. This presents several significant issues. First and foremost, users have no control over reviewing the plan before execution. This means that potentially flawed or suboptimal plans are executed without the chance for correction. Secondly, users cannot request modifications to the plan. The system operates on a 'take it or leave it' basis, which is not ideal for complex tasks that require fine-tuning. Users also lack the ability to cancel execution before it starts, potentially leading to wasted computational resources and time on plans that may not align perfectly with their needs. Furthermore, the current system doesn't allow for approving individual sensitive steps, such as web automation or code execution, raising security concerns.

These limitations can lead to several detrimental outcomes. Unexpected agent behavior is a primary concern, as unchecked plans can lead to unintended actions. The system might also waste compute resources on incorrect or inefficient plans. Security risks are heightened, particularly when dealing with sensitive operations. Finally, the absence of a review process results in a poor user experience, especially when plans require refinement or adjustments. In essence, the current system lacks the necessary checks and balances to ensure optimal performance and user satisfaction. Implementing a robust plan approval workflow is essential to address these shortcomings and create a more reliable and user-friendly AI platform.

Proposed Solution: A User-Centric Plan Approval Workflow

To address the shortcomings of the current system, we propose implementing a user-centric plan approval workflow with a feedback loop. The user flow will incorporate several key stages. It starts with the [User Query], which is then processed by the system to generate a [Plan Generated]. Crucially, the next step is [USER REVIEW], where the user has the opportunity to interact with the generated plan. From this point, the user has three primary options: [Approve & Run], [Request Changes], or [Cancel Run]. If the user chooses to [Approve & Run], the plan is executed. If the user decides to [Cancel Run], the process is stopped, preventing any unwanted actions. The most innovative aspect of this workflow is the [Request Changes] option. When selected, the user can provide feedback, leading to a [Regenerate Plan] process. This creates a feedback loop, returning the user to the [USER REVIEW] stage with a revised plan. This loop continues until the user approves the plan or cancels the run, allowing for iterative refinement and optimization of the plan.

To support this workflow, several key features will be implemented. A Plan Approval Gate will display the generated plan to the user before execution, offering the three options: Approve & Run, Request Changes, and Cancel Run. The Feedback-Based Plan Modification feature will allow users to provide text feedback, which the AI will use to regenerate the plan, incorporating the suggested changes. To prevent manual errors and sync issues, a Read-Only Plan Display will be implemented, ensuring that users can only modify the plan through the AI-powered feedback loop. For sensitive workflows, an Optional Step-by-Step Approval feature will allow users to require approval before each web action or coding step, providing an extra layer of security. Finally, a Graceful Cancellation mechanism will ensure that users receive a friendly message upon cancellation, avoiding any technical jargon or error messages. This comprehensive solution will significantly enhance user control, safety, and efficiency in plan execution.

Key Features of the Proposed Solution

  1. Plan Approval Gate:
    • The Plan Approval Gate is a pivotal feature designed to empower users with control over their AI interactions. Before any generated plan is executed, it is presented to the user for thorough review. This gate serves as a critical checkpoint, ensuring that users are fully aware of the actions their AI agent intends to undertake. At this stage, users are provided with three distinct options, each catering to different scenarios and needs. The first option, "Approve & Run," allows the user to give the green light to the plan, signifying their confidence in its accuracy and alignment with their objectives. Upon approval, the plan proceeds to execution, putting the AI agent into action. The second option, "Request Changes," provides a pathway for users to fine-tune the plan based on their expertise or specific requirements. By selecting this option, users can provide feedback, prompting the AI to regenerate the plan with the requested modifications. This iterative process ensures that the final plan is perfectly tailored to the user's needs. The third and final option, "Cancel Run," serves as a safety net, enabling users to halt the plan's execution if they have concerns or if circumstances have changed. This option prevents any unintended actions and provides users with peace of mind. The Plan Approval Gate, therefore, is a cornerstone of user-centric AI design, fostering trust and transparency in AI interactions. Its implementation is crucial for ensuring that AI systems operate in harmony with user expectations and goals. By placing the user at the center of the decision-making process, the Plan Approval Gate promotes a collaborative relationship between humans and AI, maximizing the benefits of AI technology while mitigating potential risks. This feature underscores the commitment to user control and safety, making it an indispensable component of any responsible AI system.
  2. Feedback-Based Plan Modification:
    • The Feedback-Based Plan Modification system introduces a dynamic and interactive element to AI plan generation, moving away from a rigid, one-size-fits-all approach. At its core, this feature empowers users to actively shape the AI's plans by providing specific, actionable feedback. When a user selects the "Request Changes" option within the Plan Approval Gate, they are prompted to provide input in the form of text feedback. This feedback can range from broad strategic adjustments to fine-grained tactical modifications. For instance, a user might suggest, "Include budget considerations in the plan" or "Prioritize tasks with shorter deadlines." The AI then leverages this feedback to regenerate the plan, effectively incorporating the user's expertise and insights. This regeneration process is not a one-time event; rather, it's part of an iterative loop designed to progressively refine the plan until it meets the user's exact specifications. The AI presents the revised plan back to the user for review, and the cycle continues until the user is satisfied and approves the plan for execution. To prevent the feedback loop from becoming infinite, a maximum retry limit is implemented. This limit, typically set to a reasonable number like five, ensures that the process remains efficient and avoids prolonged iterations. If the plan cannot be refined to the user's satisfaction within the retry limit, the user has the option to cancel the plan or seek alternative solutions. The Feedback-Based Plan Modification system not only enhances the quality of AI-generated plans but also fosters a sense of collaboration and control for the user. By actively participating in the plan creation process, users gain a deeper understanding of the AI's reasoning and can ensure that the final plan aligns perfectly with their objectives. This feature represents a significant step towards more transparent, user-centric AI systems, where human expertise and AI capabilities work in synergy to achieve optimal outcomes.
  3. Read-Only Plan Display:
    • The Read-Only Plan Display is a critical design choice aimed at maintaining the integrity and coherence of AI-generated plans. This feature restricts users from directly editing the plan text, ensuring that all modifications are channeled through the AI-powered feedback loop. This approach may seem limiting at first glance, but it is essential for preventing a host of potential issues. Allowing manual editing of the plan text could lead to inconsistencies and synchronization problems, particularly in complex plans with multiple interconnected steps. Users might inadvertently introduce errors or create conflicts that are difficult to resolve, undermining the plan's overall effectiveness. Moreover, manual edits could bypass the AI's logical framework and reasoning, potentially resulting in a plan that is no longer coherent or optimal. By enforcing a Read-Only Plan Display, the system ensures that all changes are processed and validated by the AI, maintaining the plan's structural integrity and logical consistency. Users can still exert significant influence over the plan's evolution through the Feedback-Based Plan Modification system. By providing textual feedback, users can guide the AI to make specific changes, effectively shaping the plan to their needs. This approach ensures that the plan remains aligned with the user's objectives while preserving the AI's ability to optimize the plan based on its knowledge and algorithms. The Read-Only Plan Display, therefore, is a crucial element in balancing user control with system stability. It prevents unintended consequences while empowering users to refine plans through a structured and reliable feedback mechanism. This feature underscores the importance of thoughtful design in AI systems, where user empowerment and system integrity are carefully balanced to create a seamless and effective user experience.
  4. Optional Step-by-Step Approval:
    • The Optional Step-by-Step Approval feature introduces a heightened level of control and security for users, particularly in scenarios involving sensitive operations. This feature allows users to scrutinize and approve each individual step within a plan, providing a granular level of oversight that is invaluable for high-stakes tasks. The feature is implemented through checkboxes that users can select based on their needs. One checkbox, labeled "Require approval before each web action," allows users to review and approve every interaction with a web-based system, such as logging into an account or submitting a form. This is particularly useful for preventing unintended data exposure or unauthorized access. Another checkbox, labeled "Require approval before each coding step," provides similar control over code execution. By selecting this option, users can ensure that each line of code generated and executed by the AI is reviewed and approved, minimizing the risk of errors or malicious code. The Optional Step-by-Step Approval feature is especially beneficial in workflows involving sensitive operations, such as financial transactions, data processing, or system administration. By requiring explicit approval for each step, users can mitigate risks and ensure that the AI's actions are fully aligned with their intentions. This feature also promotes transparency and accountability, as each approved step is documented, providing a clear audit trail of the AI's activities. While the Step-by-Step Approval feature offers enhanced control and security, it is designed to be optional. Users can choose to enable it only when necessary, allowing them to balance security with efficiency. For less sensitive tasks, users can disable the feature, streamlining the process and reducing the need for manual intervention. The Optional Step-by-Step Approval, therefore, is a versatile and valuable tool that empowers users to tailor the level of control and security to the specific requirements of their tasks. It represents a significant step towards more responsible and trustworthy AI systems, where user control and safety are paramount.
  5. Graceful Cancellation:
    • The Graceful Cancellation mechanism is a user-centric design element that prioritizes a positive and reassuring user experience, even when things don't go as planned. In any complex system, the ability to gracefully handle unexpected situations is crucial, and AI systems are no exception. The Graceful Cancellation feature is specifically designed to ensure that when a user cancels a task or a plan, the process is handled smoothly and transparently, without causing confusion or frustration. Instead of displaying technical error messages or Python tracebacks, which are often incomprehensible to non-technical users, the system presents a user-friendly message: "Task cancelled by user. No changes were made." This message provides clear and concise feedback, assuring the user that their action has been successfully processed and that no unintended consequences have occurred. The absence of technical jargon is intentional, as it prevents users from feeling overwhelmed or intimidated by the system. The message is designed to be reassuring, emphasizing that the cancellation was successful and that no adverse effects have resulted. The Graceful Cancellation feature also ensures that the system cleanly terminates the task or plan, preventing any residual processes from running in the background. This is essential for maintaining system stability and preventing resource wastage. By implementing a Graceful Cancellation mechanism, AI systems demonstrate a commitment to user experience and transparency. This feature underscores the importance of designing AI systems that are not only powerful and efficient but also user-friendly and forgiving. The ability to gracefully handle cancellations is a hallmark of a well-designed system, fostering trust and confidence among users. It represents a crucial step towards making AI technology more accessible and approachable for a wider audience.

UI Mockup: Enhancing User Interaction

The user interface (UI) mockups provide a visual representation of how the proposed plan approval workflow would be integrated into the system. These mockups focus on clarity, ease of use, and providing users with the necessary information to make informed decisions. The primary components of the UI include the Plan Approval Card and the Feedback Input interface.

Plan Approval Card

The Plan Approval Card is the central element of the plan review process. It is designed to present the user with a clear and concise overview of the generated plan, along with the options to approve, request changes, or cancel the run. The card includes a brief message prompting the user to review the generated plan before execution. This message serves as a gentle reminder of the importance of user oversight. A prominent "Click to view plan" link or button provides easy access to the full plan details. This allows users to examine the plan in detail before making a decision. To accommodate sensitive workflows, the card includes optional checkboxes for step-by-step approval. The "Require approval before each web action" and "Require approval before each coding step" checkboxes allow users to enable granular control over specific types of actions. Finally, the card presents three clearly labeled buttons: "Cancel Run," "Request Changes," and "Approve & Run." These buttons provide the primary means of interacting with the plan, allowing users to cancel the execution, request modifications, or approve the plan for execution.

┌─────────────────────────────────────────────────────────┐
│ Review the generated plan before execution.             │
│                                                         │
│ 📋 Click to view plan                            [→]    │
│                                                         │
│ ☐ Require approval before each web action              │
│ ☐ Require approval before each coding step             │
│                                                         │
│     [Cancel Run]  [Request Changes]  [Approve & Run]   │
└─────────────────────────────────────────────────────────┘

Feedback Input

When the user clicks the "Request Changes" button, the Feedback Input interface is displayed. This interface provides a text input area where users can provide specific feedback on the plan. The interface includes a prompt asking the user, "What changes would you like to see in the plan?" This prompt is designed to encourage users to provide clear and actionable feedback. A text input field allows users to enter their feedback. Example prompts, such as "E.g., Include a visit to the Louvre, add budget...", are provided to guide users in formulating their feedback. These examples help users understand the types of changes they can request. "Cancel" and "Submit Feedback" buttons provide the means to cancel the feedback process or submit the feedback to the AI for plan regeneration.

┌─────────────────────────────────────────────────────────┐
│ What changes would you like to see in the plan?         │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ E.g., Include a visit to the Louvre, add budget... │ │
│ └─────────────────────────────────────────────────────┘ │
│                                                         │
│                        [Cancel]  [Submit Feedback]      │
└─────────────────────────────────────────────────────────┘

Acceptance Criteria: Ensuring Functionality and User Satisfaction

To ensure that the proposed plan editing and approval workflow meets the required standards of functionality and user satisfaction, a set of acceptance criteria has been defined. These criteria serve as a checklist to verify that each aspect of the feature is implemented correctly and performs as expected. The first criterion is that the user can review the generated plan before execution, which is fundamental to the entire workflow. The second criterion ensures that the user has the ability to approve the plan to start execution, providing them with the final say in the process. The user must also be able to cancel the plan to abort it without encountering any technical traceback, ensuring a smooth and user-friendly experience. A crucial aspect of the workflow is the ability for the user to request changes with text feedback, allowing them to communicate their specific requirements to the system. The AI should then be able to regenerate the plan, incorporating the user's feedback to refine the plan according to their needs. To prevent infinite loops, the feedback loop should be limited to a certain number of iterations. The plan should be read-only, preventing manual editing and ensuring consistency. Optional checkboxes for step-by-step approval should be available for users who require more granular control. When enabled, the step approval UI should be displayed for web and coding operations, allowing users to review each step individually. Finally, the cancellation process should be graceful, providing a user-friendly message to confirm that the task has been canceled without any technical errors.

  • [ ] User can review generated plan before execution
  • [ ] User can approve plan to start execution
  • [ ] User can cancel plan to abort without traceback
  • [ ] User can request changes with text feedback
  • [ ] AI regenerates plan incorporating user feedback
  • [ ] Feedback loop limited to prevent infinite iterations
  • [ ] Plan is read-only (no manual editing)
  • [ ] Optional checkboxes for step-by-step approval
  • [ ] Step approval UI for web/coding operations (when enabled)
  • [ ] Graceful cancellation with user-friendly message

Benefits: Enhancing User Control, Safety, and Efficiency

The implementation of a plan editing and approval workflow with a feedback loop offers numerous benefits, significantly enhancing the user experience, safety, and efficiency of the system. User control is paramount, as this feature empowers users to make informed decisions about what gets executed, ensuring that the AI's actions align with their intentions. Safety is also greatly improved, particularly in sensitive operations, as the explicit approval requirement minimizes the risk of unintended consequences. The ability to catch plan errors before execution also leads to increased efficiency, preventing wasted computational resources and time on flawed plans. Furthermore, the flexibility afforded by natural language feedback allows users to refine plans iteratively, ensuring optimal outcomes. Finally, the improved user experience, with no technical errors displayed to users, fosters trust and confidence in the system. By empowering users with greater control, ensuring safety, improving efficiency, and enhancing the user experience, this feature represents a significant step forward in the development of user-centric AI systems.

  • User Control: Users decide what gets executed
  • Safety: Sensitive operations require explicit approval
  • Efficiency: Catch plan errors before wasting compute
  • Flexibility: Refine plans through natural language feedback
  • Better UX: No technical errors shown to users

Labels

enhancement feature-request ux frontend backend

In conclusion, the proposed plan editing and approval workflow with a feedback loop is a crucial enhancement for the Agentic AI and CortexON platforms. By implementing this feature, we can ensure greater user control, safety, efficiency, and a more positive overall user experience. This feature is essential for building trustworthy and reliable AI systems that meet the needs of their users. For further reading on best practices in AI development, visit The AI Standards Hub.