@OfficialNathanY: What if your robot could under...

What if your robot could understand any object you describe, just from a phone camera?

RADIO-ViPE builds a 3D map from raw monocular video that you can query with natural language.
(1/4)

How it works:
A foundation model (RADIO) extracts dense "meaning vectors" per pixel, then they reuse those same vectors three ways: improving optical flow on blank surfaces, adding a semantic loss into the geometry optimizer, and connecting similar keyframes in the factor graph.

Instead of bolting semantics after SLAM (like ConceptGraphs/HOV-SG), the semantic similarity error lives inside the bundle adjustment loss function. When the optimizer adjusts a camera pose, it’s satisfying geometric AND semantic consistency in the same gradient step.

For dynamic scenes (people walking, furniture getting moved), they track how stable each pixel’s semantic embedding is over time. If it is consistently similar across views, then we trust it. Otherwise, we suppress it.

(Day 8 of reading interesting VLA/World related model papers)

arxiv.org/pdf/2604.26067

Could World Models actually help with Autonomous self-driving Cars?

WorldVLM bridges a VLM (the brain) and a physics-aware world model (the body) in order to try and solve this problem.