Home / Technology / Robots Learn by Watching: AI Decodes Human Skills
Robots Learn by Watching: AI Decodes Human Skills
6 Oct
Summary
- New AI model Pegasus 1.6 decodes human task performance from video.
- It transforms raw footage into structured, time-based metadata for robots.
- The AI specializes in first-person recordings for robotics training.
TwelveLabs has introduced Pegasus 1.6, a significant advancement in video AI aimed at revolutionizing robot training. This new model specializes in analyzing footage captured from the performer's perspective, breaking down complex tasks into usable data.
Pegasus 1.6 transforms raw videos into structured, timestamped metadata. This allows companies to document and organize physical workflows more efficiently, moving beyond basic video description and segmentation capabilities. The AI's native temporal reasoning helps understand cause and effect in sequences.
The AI is particularly adept at processing first-person recordings. This capability addresses the challenge of converting raw footage of human demonstrations into actionable data for robotics developers. Key workflows include breaking down recordings into labeled actions, generating detailed captions, and screening footage for quality.
While Pegasus 1.6 provides crucial insights into movements and trajectories, it does not fully equip robots for precise control. Robotics developers must integrate this video-derived information with other data and control systems. Pricing models involve billing for video duration and text output, with enterprise contracts offering custom rates. Potential customers are advised to pilot with bounded workflows to assess detail sufficiency and label accuracy.
