physical intuition for scalable humanoid rig procedural animation
zanebeeai
1 month ago
Connected people:
Yi Zhou (https://scholar.google.com/citations?user=a8P5CVIAAAAJ&hl=en): MIT SGI research mentor, Roblox & Adobe researcher
Jiaxin Lu (https://jiaxin-lu.github.io/): Prev student of Yi, worked at Roblox & Stanford
Tingwu Wang (https://scholar.google.com/citations?user=HvJj-pEAAAAJ&hl=en): Not yet contacted (friend of Yi), at NVIDIA GEAR lab, FA on motionbricks
Jiaju Ma (https://majiaju.io/): Lecturer for this project w/ MIT SGI, Stanford PHD, prev work in 2D gen animations based on primitives, particularly on evals
zanebeeai
1 month ago
Things to improve:
Have a flywheel vlm-in-loop eval
Create 5 possibilities and pick the best one
Run a bunch of cases and pick up only the successful cases with RAG
Use RAG to give in-context examples of a pre-existing library
How to embed in a vector db for rag:
Include the prompt when storing the animation
Could also extract keyframes from an animation and allow a VLM to label what the animation is like
Could use CLIP model? Or some other like cosine similarity tool idk
Endgame, could also do a council-based approach
For sims
Isaac sims
https://arxiv.org/pdf/2509.26633
https://jiaxin-lu.github.io/humoto/
Potential idea: use COSMOS (or a more egocentric version) to fetch a video (or get a very specified prompt w/ world model to generate such a thing?), convert to sim for context and then animate with the in-context example (basically sim2real2sim)
e.g. dataset https://signavatars.github.io/
Log in to join the conversation if you are a friend or collaborator.