Key Contributions

• DAAAM: A novel real-time approach to create a 4D scene graph as explicit large-scale spatio-temporal memory with highly detailed annotations
• Optimization-based annotation: Efficiently annotate entities with large localized captioning models online in batch
• State-of-the-art results on spatio-temporal question answering and sequential task grounding, plus a new extended benchmark (data and code open-source)

Results

State-of-the-art on spatio-temporal question answering (OC-NaVQA) and sequential task grounding (SG3D):

• OC-NaVQA: +53.6% question accuracy, -21.9% position error, -21.6% temporal error
• SG3D: +27.8% task grounding accuracy
• Real-time at 10Hz, scalable to 35+ min sequences and 1.5km distance

Check out the project page for more details, including links to the paper and code.

Key Contributions

Our work addresses the challenge of predicting human trajectories over long time horizons (up to 60 seconds) in complex indoor environments. The main contributions include:

• Novel approach for long-term human trajectory prediction using 3D Dynamic Scene Graphs
• LLM-based reasoning about human-environment interactions to guide trajectory predictions
• Probabilistic framework based on continuous-time Markov Chains for multi-modal trajectory distributions
• New dataset of long-term human trajectories with human-object interaction annotations

Results

Our approach achieves:

• 54% lower average negative log-likelihood compared to best non-privileged baselines
• 26.5% lower Best-of-20 displacement error for 60s prediction horizon

Check out the project page for more details, including the video presentation and links to the paper and code.