Run Sample Code in Genesis

About

I used to have a strong interest in reinforcement learning. I thought it was an amazing technology that fused simulation with machine learning. At the time, I experimented with various algorithms using tools like OpenAI Gym and PyBullet. While the simulation environments were user-friendly, the learning techniques were extremely complex, and I struggled a lot.

A few weeks ago, a new project called Genesis was released. It seems to feature stunning graphics and simulation environments, so I tried running some sample code. Unlike back then, there are now more reinforcement learning frameworks and projects available, which might mean the field is moving closer to practical applications. It could be worthwhile to investigate where the costs lie in the future.

Anyway, below is a quick summary of the sample results. Since I only used the tutorial, the results aren’t particularly interesting, but the training was completed in just a few minutes, and the results were visualized after about a minute of compilation.

Run Sample

Train a model with rsl-rl

https://genesis-world.readthedocs.io/en/latest/user_guide/getting_started/locomotion.html

# Install rsl_rl.
git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl && git checkout v1.0.2 && pip install -e .

# Install tensorboard.
pip install tensorboard

python examples/locomotion/go2_train.py

Rewards in this sample

• tracking_lin_vel: Tracking of linear velocity commands (xy axes)
• tracking_ang_vel: Tracking of angular velocity commands (yaw)
• lin_vel_z: Penalize z axis base linear velocity
• action_rate: Penalize changes in actions
• base_height: Penalize base height away from target
• similar_to_default: Encourage the robot pose to be similar to the default pose

You can check these reward sequences thru tensorboard

python examples/locomotion/go2_eval.py

the results were visualized after about a minute of compilation.

Result

18 FPS in my environment

Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          39 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   8
  On-line CPU(s) list:    0-7
Vendor ID:                GenuineIntel
  Model name:             Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
    CPU family:           6
    Model:                158
    Thread(s) per core:   1
    Core(s) per socket:   8
    Socket(s):            1
    Stepping:             12
    CPU max MHz:          3600.0000
    CPU min MHz:          800.0000

NVIDIA GeForce GTX 1660