Clamped arm DOFs for Sports2D IK

Clamped arm DOFs for Sports2D IK

Full Changelog: v0.10.11...v0.10.12

Various improvements

Various small fixes

• Fixed model with muscles and contact spheres
• Handle approx_time_maxspeed specified once for all videos
• Fixed issue with extrinsic calibration with board
• Fixed frame_range display on progress bar
• Fixed body_25 markers
• Set larger RoMs for wrist
• Removed qt5Agg matplotlib backend

What's Changed

• Handle approx_time_maxspeed specified once by @AurelienCoppee in #160

Full Changelog: v0.10.9...v0.10.10

Kinematics with contact spheres and muscles by default

• Kinematics with contact spheres and muscles by default
• Separated markerset from model files
• fixed coco_133 skeleton

What's Changed

• remove "Mouth21" by @hunminkim98 in #159

Full Changelog: v0.10.8...v0.10.9

DeepSort tracking and better skeleton visualization

• Skeleton visualization: A bounding-box color per person. Left and right limbs in a different color to better spot swaps. Keypoint color based on confidence.
• Support DeepSort tracking across frames
• Video files processed in alphabetical order on Unix machines

Full Changelog: v0.10.7...v0.10.8

I can't believe how much I struggled to implement it. 😅 Long story short, it works on bounding boxes rather than on keypoints, so I first tried to subclass the rtmlib PoseTracker to extract the bounding boxes obtained from the person detector. But everything got messed up when I tested it when with RTMO, which is single-stage and does not use any bounding boxes. Moreover, the option "det_frequency" makes everything more complicated, because bounding boxes are then not searched for all frames. So I recomputed bounding boxes from keypoints, tricked the deepsort tracker into returning the original bounding box indices (rather than the ones after kalman filtering) as well as the person indices, and tried to make it correctly find the right correspondence.

And then my brain was fried and I could not manage to make sense of the logic between so many indices. This took me way more time than it should have. Anyway, it is now properly implemented, this is one thing less to worry about!

One word of cautions: In the vast majority of cases (non crowded scenes), I would use the automatic Sports2D tracking algorithm. DeepSort can be tricky to parametrize, and you need to understand what each of the parameters mean to if you want best results. Otherwise, you may very well end up with worse results. I tried to set the best possible default parameters, but it won't work in every case. Results are faster and better with {'embedder_gpu': True, embedder':'torchreid'}, which uses the GPU and runs osnet_ain_x1_0 by default. Note that it requires pip install torch torchvision torchreid gdown tensorboard. If you feel like there default parameters could be improved, feel absolutely free to let me know!

Any detection and pose model can be used + more

• install custom rtmlib version (see Tau-J/rtmlib#45) with:
  • a new more general Custom class
  • PoseTracker working with single-stage models (without detection)
  • working NMS for RTMO
  • fixed wholebody solution in balanced mode (see #143 (comment))
• More versatile "mode" argument in pose2sim: Any person detection and/or pose estimation model can now be used
• Natively supporting Hand, Face, Animal, whole body with face and hand models (up to filtering 3D results, no inverse kinematics implemented yet)
• Optional manual selection of backend and device
• Unified most sports2d and pose2sim functions
• Default names for Halpe_26, coco_133, coco_17 are now: body_with_feet, whole_body, body
• Automatic person height estimation
• Better scaling with best_coords_for_measurements function
• Kinematics works on Coco17 model
• Released pronation (-90° instead of 0°)
• Removed Lai-Uhlrich shoulder definition in LSTM model
• Handled case when center hip keypoint is missing
• Automatic FPS detection rounded to integer
• custom logging possible
• Improved the trc_gait_events script

What's Changed

• Calculate height for marker augmenatation by @hunminkim98 in #152

Full Changelog: v0.10.6...v0.10.7

Better kinematics + various fixes

Scaling and IK:

• Support wrist motion on LSTM model
• Wrist flexion clamped to 110° instead of 70°
• Scaling on straight postures in priority
• IK and scaling IKCoordinateTask weights of 0.1 instead of 0.5 for L5_S1_Flex_Ext

Calibration:

• fixed calibration if board and show=false
• removed CALIB_FIX_PRINCIPAL_POINT + convert objp to mm to avoid scale mismatch with intrinsics)

Pose estimation:

• ROCM (AMD GPU) support

What's Changed

• LSTM with wrist motion by @hunminkim98 in #148
• Optimized Setup Files(scaling+IK) by @hunminkim98 in #151

Full Changelog: v0.10.4...v0.10.6

multi_person synchro + minor fixes

• Synchronization in multi-person mode @hunminkim1998
• Supports running on image sequences instead of videos
• Added missing Geometry file for LSTM model
• Fixed frame_range when different frames for each camera

What's Changed

• Modify the frame range from max to min. by @hunminkim98 in #138
• Synchronization for multi-person by @hunminkim98 in #135

Full Changelog: v0.10.3...v0.10.4

Attempt to fix caliscope support

Full Changelog: v0.10.2...v0.10.3

Fixed CalisCope calib support + Fixed kinematics() when no marker augmentation

• Fixed CalisCope calib support
• Fixed kinematics() when no marker augmentation

What's Changed

• Site test back to main branch by @davidpagnon in #134

Full Changelog: v0.10.1...v0.10.2

Support caliscope calibration + 2D sorting for future multi-person synchro

1. Native support of Caliscope calibration
2. Sort people's IDs at the 2D stage, to prepare for future multi-person synchronization

Full Changelog: v0.10.0...v0.10.1