Develop Multimodal Task Planning and Memory Integration

implementationChallengeNovember 25, 2025

Prompt Content

Implement the `Planning` and `Memory Management` modules. Given a natural language task instruction (e.g., 'Find the red key and go to the blue door') and multimodal input (visual scene + text), use Gemini 2.5 Pro to generate a sequence of high-level sub-goals and then decompose these into specific tool calls. Integrate the Postgres + pgvector memory system to store and retrieve past experiences and learned strategies, allowing the agent to adapt its plan based on relevant historical context.

Usage Tips

Copy the prompt and paste it into your preferred AI tool (Claude, ChatGPT, Gemini)

Customize placeholder values with your specific requirements and context

For best results, provide clear examples and test different variations