NVIDIA introduces Flora data engine for robotics video annotation
Flora segments actions, writes instructions and scores task execution to prepare human and robot video for robot learning.
NVIDIA has introduced Flora, a suite of models that converts human and robot video into structured training labels. Its pipeline identifies the boundaries between actions, describes the operation attempted during each interval and evaluates execution quality. NVIDIA uses the system in its GR00T robot-model data pipeline and reports annotation throughput of roughly 200,000 video hours per day on 512 GB300 GPUs.
Flora separates the work into three components. TempSeg locates actions in a recording; Captioner produces an instruction for a selected interval, using neighbouring footage to interpret what is happening; Rewarder assesses task outcomes and intermediate progress. Together, the outputs connect visual demonstrations with language and learning signals that developers can use to organise datasets, select examples and evaluate policies.
The Captioner model card describes a 27-billion-parameter model adapted from Qwen3.5-27B through supervised fine-tuning and reinforcement learning. It accepts either a single camera stream or synchronised main-camera and wrist-camera views. Before-and-after context helps distinguish the intended operation within the selected segment, including when a robot repeats an attempt or fails.
NVIDIA's evaluations show different strengths across annotation tasks. On the human-video subset of DenseCaptionBench, Flora achieved a segmentation F1 score of 54.6 at a 0.5 overlap threshold, compared with 44.2 for GPT-6 Astra. On the robot subset, joint localisation-and-captioning F1 was 51.0 for Flora and 57.9 for Astra. These measurements evaluate the quality of generated labels; the separate daily-throughput figure comes from NVIDIA's internal GPU deployment.
The Captioner and Rewarder weights are available under NVIDIA's OneWay Noncommercial License for research and development. The model cards date the Hugging Face release to 15 October. Researchers can use the components to prototype data curation and evaluation workflows, subject to those licence terms.