GPT for Robotics: Perception-Action Causal Transformer(microsoft.com)

2 pointsby madratman4 years ago1 comments

GPT for Robotics: Perception-Action Causal Transformer

https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/perception-action-causal-transformer-for-autoregressive-robotics-pretraining/

1 comments

madratman4 years ago

Generative self-supervised model for robotics data.

Highlights:

- Transformer model for autoregressive prediction of states and actions over time to implicitly encode dynamics and behaviors for a particular robot

- The representation learned can be fine-tuned to distinct tasks (navigation, mapping, localization) with minimal data

- The base transformer model serves as a generative robotics model, similarly to GPT-3, and can be prompted to result in different robot behaviors

Paper: https://arxiv.org/abs/2209.11133;

Video: https://youtu.be/mNQvQu_atuw;

Code and data: https://github.com/microsoft/PACT