Sketch : main

1. Loss function

• Collision awareness를 어떻게 강제할 수 있을까

2. Training scheme

• Trumans라는 작은 데이터셋을 대강 annotation해서 text-motion-scene pair를 활용하여 adaptation을 한다. (Baseline)
  • Controlnet
    • attention
• Text-human motion 모델을 다른 task로 학습하여 scene-awareness 주입하기 (될지모르겠음)

3. Scene Representation

• SDF - 쓰는 사람이 없음 optimization-base scene awareness models)
• BPS - 최근에 좀 씀
• Voxel - Trumans, LINGO, SCENEMI
• PC - Affordmotion, HUMANISE

4. Generating dynamic human scene interaction in real world scenes

1. Generating interactable photorealistic scenes
   • Photorealistic -> real world scenes
   • Interactive -> Decomposition
2. Generating human scene interactions in these scenes
   • arbitrary scene -> zero-shot
   • complex task -> locomotion&interaction
   • arbitrary text input -> LLM planning

Brainstorming

Brainstorming - 민수