Beyond Chat

CoachMe+ — Agentic boxing coaching with live generative UI

CoachMe+ is a working full-stack app where the AI coach does not stop at chat bubbles. A LangGraph deep agent (orchestrated with Gemini) calls TwelveLabs Pegasus to analyze indexed fight footage and returns structured coaching data through Command updates to shared agent state (coaching_data, coaching_status, coaching_score_history). The CopilotKit stack—AG-UI-style agent–frontend transport via the LangGraph integration in the CopilotKit BFF—mirrors that state into the client so the /coach canvas updates as a live dashboard: overall score ring, per-technique breakdowns, drill cards, a correction timeline with scrubber and optional HLS/MP4 playback, and session score trends after multiple clips. Generative UI also appears inside the chat through useFrontendTool renderers (e.g. focus highlights, interactive drill approve/skip, before/after comparisons), so both the main surface and the copilot respond directly to tool calls and state—not to hand-parsed markdown. This moves beyond text-in/text-out to functional, interactive interfaces driven by the agent. The project uses CopilotKit v2, Next.js, LangGraph (langgraph dev), TwelveLabs (Pegasus + Marengo search), and CopilotKit Intelligence for threads; the starter MCP Apps wiring remains available in the BFF for optional live MCP-driven UI. A2UI is not required for our path: we deliver the same hackathon intent—agent-authored, updating UI—through typed state and CopilotKit’s generative UI patterns. Originality comes from the boxing + video understanding vertical and the single source of truth in graph state powering two synchronized surfaces (canvas + chat widgets).