guides/workouts-monitoring/

[giscus[bot]]

guides/workouts-monitoring/

Workouts Monitoring Using Ultralytics YOLOv8

https://docs.ultralytics.com/guides/workouts-monitoring/

[wejdross]

Hello, I'm playing with this project for fun and learning and I have few questions:

• how can I train my own dataset, I saw this video: https://www.youtube.com/watch?v=m9fH9OWn8YM and it's pretty simple to create dataset, I'd love to see tutorial how to do similar thing for videos etc.
• I'd like to instruct algorithm for learning purposes how to make squats
• is it possible and how to detect person on the video, for example I'd like to create something that would tell me: "lukas is doing 1,2,3,4(...) squats", I'd like to introduce my wife then and see "unknown is doing pushups" because why not

I'm really new to the topic, but it seems to be quite powerful and interesting

[pderrenger]

Hey there! 🚀 It's great to see your enthusiasm for learning and experimenting with YOLOv8 for workout monitoring. Let's dive into your questions:

1. Training on Your Own Dataset: For training on videos, you can extract frames from your videos and annotate them to create a dataset. Tools like CVAT are great for this. Once you have your dataset, training is similar to the process shown in the video. Just use your dataset's path in the data argument.

2. Teaching the Algorithm Squats: To teach the algorithm to recognize squats, you'll need to collect and annotate images or video frames of squats. Ensure your dataset includes various squat positions and angles. Then, train a pose estimation model like yolov8n-pose.pt with this dataset.

3. Detecting and Counting Squats: For detecting a person and counting squats, you can use the pose estimation model (yolov8n-pose.pt). The code snippet from the docs page you found is a great starting point. You can modify the pose_type argument to "squats" (you might need to customize the AI Gym module for new exercises).

To distinguish between different people (e.g., "Lukas" vs. "unknown"), you'd need to integrate a facial recognition system alongside the pose estimation. This would involve training a separate model on faces or using a pre-trained facial recognition model.

Here's a simplified example to get you started with squat detection and counting:

from ultralytics import YOLO
from ultralytics.solutions import ai_gym
import cv2

model = YOLO("yolov8n-pose.pt")  # Load pose estimation model
cap = cv2.VideoCapture("path/to/your/video.mp4")  # Load your video
gym_object = ai_gym.AIGym()  # Initialize AI Gym for workout monitoring
gym_object.set_args(pose_type="squats")  # Set to monitor squats

while True:
    success, frame = cap.read()
    if not success:
        break
    results = model.predict(frame, verbose=False)
    frame = gym_object.start_counting(frame, results)
    cv2.imshow("Squat Detection", frame)
    if cv2.waitKey(1) == ord('q'):
        break

cv2.destroyAllWindows()

Remember, you'll need to customize the AI Gym module for squat detection, as it primarily supports pushups, pullups, and ab workouts out of the box.

Feel free to explore and experiment, and don't hesitate to reach out if you have more questions. Happy coding! 🎉

[felipeca2]

hello, this is possible whit CLI?
thanks

[pderrenger]

Hello,

Yes, you can monitor workouts using the CLI with Ultralytics YOLOv8. You can use the track mode to process video files and track exercises like pushups, pullups, and ab workouts. Here's a quick example:

yolo track model=yolov8n-pose.pt source=path/to/video/file.mp4 tracker=botsort.yaml

This command will use the YOLOv8 pose estimation model to track exercises in your video. For more detailed information, please refer to the documentation.

Happy training! 💪

[duncanwaugh]

Hello, it seems like it has to be setup for a specific type of workout eg. pushups and then it would track pushups, but if you started squatting it wouldn't track your squats. I'm curious if there is a way where U can have a live video feed where lets say you do 10 pushups, then 10 pullups, it will identify how its a different exercise and count it. Thanks.

[glenn-jocher]

Hello,

Thank you for your insightful question! You're correct that the current setup of the AIGym solution in Ultralytics YOLOv8 is designed to track specific types of workouts like pushups, pullups, and ab workouts individually. However, your idea of dynamically identifying and counting different exercises in a single session is quite intriguing and can be approached with some modifications.

Dynamic Workout Detection and Counting

To achieve dynamic detection and counting of different exercises, you would need to implement a mechanism that can:

1. Identify the type of exercise being performed: This could be done by analyzing the keypoint movements and angles to classify the exercise type.
2. Switch the counting logic based on the identified exercise: Once the exercise type is identified, the corresponding counting logic can be applied.

Here's a conceptual approach to get you started:

1. Exercise Classification: Use a simple heuristic or a machine learning model to classify the exercise based on keypoint movements.
2. Dynamic Counting: Switch between different AIGym configurations based on the classified exercise.

Example Code

Below is a basic example to illustrate this concept. Note that this is a starting point and may require further refinement based on your specific requirements.

import cv2
from ultralytics import YOLO, solutions

# Load the pose estimation model
model = YOLO("yolov8n-pose.pt")
cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

# Initialize video writer if needed
video_writer = cv2.VideoWriter("workouts.avi", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))

# Define a function to classify the exercise type
def classify_exercise(keypoints):
    # Placeholder logic for exercise classification
    # Replace with actual logic based on keypoint analysis
    if some_condition_based_on_keypoints:
        return "pushup"
    elif some_other_condition_based_on_keypoints:
        return "pullup"
    else:
        return "unknown"

# Initialize the AIGym object
gym_objects = {
    "pushup": solutions.AIGym(line_thickness=2, view_img=True, pose_type="pushup", kpts_to_check=[6, 8, 10]),
    "pullup": solutions.AIGym(line_thickness=2, view_img=True, pose_type="pullup", kpts_to_check=[6, 8, 10]),
    # Add other exercises as needed
}

frame_count = 0
current_exercise = "unknown"
while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    frame_count += 1
    results = model.track(im0, verbose=False)  # Tracking recommended

    # Classify the exercise based on keypoints
    keypoints = results[0].keypoints
    exercise_type = classify_exercise(keypoints)

    if exercise_type in gym_objects:
        gym_object = gym_objects[exercise_type]
        im0 = gym_object.start_counting(im0, results, frame_count)
    
    video_writer.write(im0)

cv2.destroyAllWindows()
video_writer.release()

Next Steps

1. Refine Exercise Classification: Develop a robust method to classify exercises based on keypoint data. This could involve machine learning models trained on labeled exercise data.
2. Expand Exercise Types: Add more exercises to the gym_objects dictionary and their corresponding keypoint checks.
3. Optimize Performance: Ensure the solution runs efficiently in real-time, especially if processing live video feeds.

For more detailed guidance and advanced configurations, you can refer to our Workouts Monitoring Guide.

Feel free to experiment with this approach and let us know how it works for you! If you encounter any issues or have further questions, don't hesitate to reach out.

[glenn-jocher]

@duncanwaugh hello,

Thank you for your insightful question! You're correct that the current setup of Ultralytics YOLOv8 for workout monitoring is designed to track specific exercises like pushups, pullups, and ab workouts individually. However, your idea of having a live video feed that can dynamically identify and count different exercises in sequence is fascinating and certainly feasible with some modifications.

Dynamic Exercise Recognition

To achieve dynamic exercise recognition, you would need to implement a mechanism that can classify the type of exercise being performed in real-time and then switch the pose estimation model accordingly. Here's a high-level approach to how you might achieve this:

1. Exercise Classification: Use a classification model to identify the type of exercise being performed. This model can be trained to recognize different exercises based on the input video frames.

2. Pose Estimation Switching: Once the exercise type is identified, switch to the appropriate pose estimation model for that exercise. This can be done dynamically within the video processing loop.

Example Implementation

Below is a simplified example to illustrate this concept. This example assumes you have separate models for exercise classification and pose estimation.

import cv2
from ultralytics import YOLO, solutions

# Load models
classification_model = YOLO("exercise_classification_model.pt")
pose_models = {
    "pushup": YOLO("yolov8n-pose-pushup.pt"),
    "pullup": YOLO("yolov8n-pose-pullup.pt"),
    "squat": YOLO("yolov8n-pose-squat.pt")
}

cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

gym_objects = {
    "pushup": solutions.AIGym(line_thickness=2, view_img=True, pose_type="pushup", kpts_to_check=[6, 8, 10]),
    "pullup": solutions.AIGym(line_thickness=2, view_img=True, pose_type="pullup", kpts_to_check=[6, 8, 10]),
    "squat": solutions.AIGym(line_thickness=2, view_img=True, pose_type="squat", kpts_to_check=[6, 8, 10])
}

current_exercise = None
frame_count = 0

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    frame_count += 1

    # Classify exercise type every few frames
    if frame_count % 30 == 0:  # Adjust frequency as needed
        classification_results = classification_model.predict(im0)
        current_exercise = classification_results[0].probs.top1  # Assuming top1 gives the exercise type

    if current_exercise:
        pose_model = pose_models[current_exercise]
        gym_object = gym_objects[current_exercise]
        results = pose_model.track(im0, verbose=False)
        im0 = gym_object.start_counting(im0, results, frame_count)

    cv2.imshow("Workout Monitoring", im0)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

Key Points:

• Exercise Classification: The classification_model predicts the type of exercise.
• Pose Estimation Switching: Based on the predicted exercise, the corresponding pose estimation model and AIGym object are used.
• Dynamic Processing: This setup allows for dynamic switching between different exercises during a workout session.

This approach provides a flexible and scalable solution for monitoring multiple types of exercises in a single session. For more detailed guidance and advanced configurations, you can refer to our Workouts Monitoring Guide.

Feel free to experiment with this setup and adjust it to better fit your specific use case. If you encounter any issues or have further questions, don't hesitate to reach out!

[duncanwaugh]

awesome thanks

[glenn-jocher]

@duncanwaugh hello,

Thank you for your enthusiasm! 😊

To address your query about dynamically identifying and counting different exercises within a single workout session, such as transitioning from pushups to pullups, you would need to implement a mechanism that can classify the type of exercise being performed in real-time and then switch the pose estimation model accordingly.

Dynamic Exercise Recognition

Here's a high-level approach to achieve this:

1. Exercise Classification: Use a classification model to identify the type of exercise being performed. This model can be trained to recognize different exercises based on the input video frames.
2. Pose Estimation Switching: Once the exercise type is identified, switch to the appropriate pose estimation model for that exercise. This can be done dynamically within the video processing loop.

Example Implementation

Below is a simplified example to illustrate this concept. This example assumes you have separate models for exercise classification and pose estimation.

import cv2
from ultralytics import YOLO, solutions

# Load models
classification_model = YOLO("exercise_classification_model.pt")
pose_models = {
    "pushup": YOLO("yolov8n-pose-pushup.pt"),
    "pullup": YOLO("yolov8n-pose-pullup.pt"),
    "squat": YOLO("yolov8n-pose-squat.pt")
}

cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

gym_objects = {
    "pushup": solutions.AIGym(line_thickness=2, view_img=True, pose_type="pushup", kpts_to_check=[6, 8, 10]),
    "pullup": solutions.AIGym(line_thickness=2, view_img=True, pose_type="pullup", kpts_to_check=[6, 8, 10]),
    "squat": solutions.AIGym(line_thickness=2, view_img=True, pose_type="squat", kpts_to_check=[6, 8, 10])
}

current_exercise = None
frame_count = 0

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    frame_count += 1

    # Classify exercise type every few frames
    if frame_count % 30 == 0:  # Adjust frequency as needed
        classification_results = classification_model.predict(im0)
        current_exercise = classification_results[0].probs.top1  # Assuming top1 gives the exercise type

    if current_exercise:
        pose_model = pose_models[current_exercise]
        gym_object = gym_objects[current_exercise]
        results = pose_model.track(im0, verbose=False)
        im0 = gym_object.start_counting(im0, results, frame_count)

    cv2.imshow("Workout Monitoring", im0)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

Key Points:

• Exercise Classification: The classification_model predicts the type of exercise.
• Pose Estimation Switching: Based on the predicted exercise, the corresponding pose estimation model and AIGym object are used.
• Dynamic Processing: This setup allows for dynamic switching between different exercises during a workout session.

This approach provides a flexible and scalable solution for monitoring multiple types of exercises in a single session. For more detailed guidance and advanced configurations, you can refer to our Workouts Monitoring Guide.

Feel free to experiment with this setup and adjust it to better fit your specific use case. If you encounter any issues or have further questions, don't hesitate to reach out!

[MarserHK]

Hello, I would like to ask if there are any examples of using YOLOv8 for behavior recognition in electrical power field scenarios?

[glenn-jocher]

@MarserHK currently, we don't have specific examples for behavior recognition in electrical power field scenarios. However, you can adapt the pose estimation capabilities of YOLOv8 for such tasks by training on relevant datasets. For more details on pose estimation, you can refer to the workouts monitoring guide here.

[Shengguang-Zhou]

Can we train extra classes for pose prediction?

[glenn-jocher]

@Shengguang-Zhou yes, you can train extra classes for pose prediction by customizing the AIGym class and specifying the keypoints and angles for the exercises you want to monitor. For more details, please refer to the Workouts Monitoring Guide.

[AdadAlShabab]

Hi. The model is working pretty fine. But when I'm running the python file the detected angle data are shown in box. But I want to have the data stored in variable or maybe a list so that I can use them use outside or print in console. Please help.

[glenn-jocher]

@AdadAlShabab to store the detected angle data in a variable or list, you can modify the start_counting method to return the angles. Here's a basic example:

import cv2
from ultralytics import YOLO, solutions

model = YOLO("yolov8n-pose.pt")
cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"

gym_object = solutions.AIGym(
    line_thickness=2,
    view_img=True,
    pose_type="pushup",
    kpts_to_check=[6, 8, 10],
)

angles = []

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        break
    results = model.track(im0, verbose=False)
    im0, angle_data = gym_object.start_counting(im0, results)
    angles.append(angle_data)

cap.release()
cv2.destroyAllWindows()
print(angles)

This will store the angle data in the angles list for further use.

[AdadAlShabab]

Hi. I want detect angle of two hands simultaneously. So I wrote the code :
gym_object = solutions.AIGym(
line_thickness=2,
view_img=True,
pose_type="pushup",
kpts_to_check={[8, 6, 12,],[5, 7, 9]},
)

But it giving me error:
TypeError: unhashable type: 'list'

How to solve? Please let me know. TIA

[glenn-jocher]

To solve the error, replace the curly braces {} with square brackets [] for kpts_to_check. Here's the corrected line:

gym_object = solutions.AIGym(
    line_thickness=2,
    view_img=True,
    pose_type="pushup",
    kpts_to_check=[[8, 6, 12], [5, 7, 9]],
)

This should resolve the "unhashable type" error.

[AdadAlShabab]

Hi Glenn! Still it throwing error:
im0 = gym_object.start_counting(im0, results)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\Bass fish streamlit.venv\Lib\site-packages\ultralytics\solutions\ai_gym.py", line 85, in start_counting
k[int(self.kpts_to_check[0])].cpu(),
^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: int() argument must be a string, a bytes-like object or a real number, not 'list'

[glenn-jocher]

The error occurs because kpts_to_check should be a list of integers, not lists. To track both hands, you need to handle each set of keypoints separately. You might need to modify the start_counting method to handle multiple sets of keypoints. If you need further assistance, please check the latest documentation or update your package to the latest version.

[actionTurtle]

If I wanted to count how many steps you took running. Would you just set the pose type to "running"? Or do I need to make my own model for running? Thank you

[glenn-jocher]

@actionTurtle to count steps for running, you'll need to customize the model since "running" isn't a predefined pose type like "pushup" or "pullup." You can adapt the existing model by training it with running-specific keypoints. For more details, please refer to the workouts monitoring guide.

[actionTurtle]

"You can adapt the existing model by training it with running-specific keypoints."

Sorry, I still don't understand. Is there a walk though of how I would do that?

[glenn-jocher]

@actionTurtle to create a custom model for running, you'll need to train it with running-specific keypoints. Unfortunately, there's no built-in "running" pose type, so you'll need to gather a dataset of running poses and train the model accordingly. For a detailed guide on training custom models, please refer to the Ultralytics documentation.

[AdadAlShabab]

Hi. Suppose I'm detecting the angle between [8, 6, 12] and I wanna show text something like: "The angle between Shoulder and Wrist : {detected angle}" using cv2.putText. How3 can I do that?

[glenn-jocher]

@AdadAlShabab to display the angle using cv2.putText, you can calculate the angle between the keypoints and then overlay the text on the image. Here's a minimal example:

import cv2
import numpy as np

# Assuming you have the coordinates of the keypoints
shoulder = np.array([x1, y1])
elbow = np.array([x2, y2])
wrist = np.array([x3, y3])

# Calculate the angle
angle = np.degrees(np.arctan2(wrist[1] - elbow[1], wrist[0] - elbow[0]) - 
                   np.arctan2(shoulder[1] - elbow[1], shoulder[0] - elbow[0]))

# Display the angle
cv2.putText(im0, f"Angle: {angle:.2f}", (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)

This will show the angle between the shoulder, elbow, and wrist on your video frame. Adjust the coordinates and text position as needed.

[AdadAlShabab]

Hi Glenn. Thanks for your wonderful reply. Can you please tell the about the keypoints [x1, y1]?
Like :
"shoulder = np.array([x1, y1])
elbow = np.array([x2, y2])
wrist = np.array([x3, y3]" What values should I put here?

[glenn-jocher]

@AdadAlShabab to determine the values for keypoints like [x1, y1], you need to extract them from the model's output. When you run the model, it provides coordinates for each keypoint. For example, results[0].keypoints will give you the keypoints, and you can access specific ones using their indices. Adjust these based on your specific use case.

[AdadAlShabab]

This code:
" if result[0].keypoints is not None:
keypoints = result[0].keypoints.data.cpu().numpy() # Access the actual keypoints data

                # Debugging: Print keypoints shape to understand the issue
                print(f"Keypoints shape: {keypoints.shape}")"

Providing me results for each frame:
0: 384x640 1 person, 45.3ms
Speed: 2.0ms preprocess, 45.3ms inference, 0.0ms postprocess per image at shape (1, 3, 384, 640)
Keypoints shape: (1, 17, 3)
Person keypoints shape: (17, 3)
Keypoint 0: (x=1086.50, y=456.84), Confidence=0.97
Keypoint 1: (x=1098.32, y=446.95), Confidence=0.95
Keypoint 2: (x=1077.20, y=450.47), Confidence=0.90
Keypoint 3: (x=1121.03, y=453.73), Confidence=0.87
Keypoint 4: (x=1068.29, y=461.00), Confidence=0.64
Keypoint 5: (x=1147.66, y=499.11), Confidence=0.99
Keypoint 6: (x=1054.84, y=529.72), Confidence=0.98
Keypoint 7: (x=1192.78, y=521.58), Confidence=0.98
Keypoint 8: (x=1061.91, y=577.40), Confidence=0.90
Keypoint 9: (x=1161.27, y=528.25), Confidence=0.95
Keypoint 10: (x=1100.56, y=558.10), Confidence=0.86
Keypoint 11: (x=1189.49, y=611.80), Confidence=0.99
Keypoint 12: (x=1134.91, y=632.54), Confidence=0.98
Keypoint 13: (x=1234.99, y=644.81), Confidence=0.97
Keypoint 14: (x=1194.94, y=665.07), Confidence=0.94
Keypoint 15: (x=1295.68, y=737.43), Confidence=0.88
Keypoint 16: (x=1300.56, y=750.85), Confidence=0.85

Are these the values you were talking about?

[glenn-jocher]

Yes, those are the keypoint values. You can use them to calculate angles between specific joints. For example, use the coordinates of keypoints 8 (wrist), 6 (shoulder), and 12 (hip) to compute the angle and display it with cv2.putText. If you need further assistance, feel free to ask!

[wynshiter]

May I ask how to obtain the value of step and perform some logical operations, such as counting to 10 times for pop-up reminders，many thanks

[glenn-jocher]

@wynshiter to track repetitions and perform logical operations like pop-up reminders, you can use the AIGym class to monitor keypoints and implement custom logic for counting. For further details, please refer to the documentation on workouts monitoring.

[nel-mv]

Thanks for your answer, Glenn. And the documentation on workouts monitoring is just this same page (https://docs.ultralytics.com/guides/workouts-monitoring) we are commenting on?

And from the comments you gave above, we know that we can get the angles from the gym object:
im0, angle_data = gym_object.start_counting(im0, results)

I was expecting we could get the 'step' value similarly via a return variable.

Do you know where we could see a list of return variables that the gym object gives?

[glenn-jocher]

The AIGym class currently provides angles via angle_data, but it does not directly return a 'step' count. To track steps or repetitions, you might need to implement custom logic within your processing loop to count specific movements based on the angles or keypoint positions. For further details, please refer to the Workouts Monitoring documentation.