Challenges Of Human Pose Estimation in AI-Powered Fitness Apps

AI-Powered Fitness Apps have successfully permeated almost all aspects of our everyday life – from opening our phone with face ID to helping compose error-free mail! If you thought artificial intelligence is too ‘sci-fi’ to have anything to do with your life, you are mistaken.

AI is the latest ubiquitous technology trend overtaking every conceivable human field – from fashion to fitness. The fitness industry revenue is growing by 8.7% every year. Even before the pandemic hit us, the fitness app development sector was already high on the popularity index, but its need is now more pronounced than ever.

However, whether artificial intelligence technology can develop solutions to replace fitness coaches remains to be seen?  

The technology hasn’t been able to bellow out orders or scream, “Even my grandma can do better than you!” Yet, it still can do wonders to analyze the accuracy of exercise performance.

What is Pose Estimation?

Powered by computer-vision technology and machine learning algorithms, human pose estimation software identifies and analyses the posture of human beings. It bases pose estimation on human body modeling, much like facial recognition. The most common types of human body models used to develop a fitness app are Contour-based, 

Skeleton-based and Volume-based model types.

Skeleton-Based Model: 

This model is used for pose estimation by detecting the human body’s skeletal structure. It identifies the key points such as knee joints, ankles, shoulders and limb orientations of the body. Since this model offers a lot of flexibility, it is used in 2D and 3D pose estimation techniques.

Contour-Based Model: 

This model considers the basic contours of the human body. It identifies the width of the torso and limbs for designing the system. And the parts of the body are represented within rectangular boundaries of the person’s silhouette.

Volume-Based Model: 

This model develops a fully-fleshed out 3D human body. Since the human structure estimation is usually done using 3D scans, the result is a more volume-based body shape. To develop a fitness app, you should always choose to use 3D estimations instead of 2D as it is more supportive of human pose estimation during physical activities.

How Does Pose Estimation work in Fitness Apps

To better understand the challenge faced by AI-powered fitness apps in detecting human pose, we should first know how pose estimation works in fitness app development.

The human pose estimation algorithm takes the captured image of the person as an input and outputs coordinates related to specific key points on the body. These landmarks on the body correspond to particular joints or heat spots. The pose estimation techniques currently used are based primarily on the convolutional neural networks technique or variants.

Generally, artificial intelligence-based fitness apps require smartphones or devices to have a camera. The camera should have the capability of recording videos up to 720p and at 60 frames per second so that the application can accurately capture each frame during exercises. The typical process flow the human pose estimation algorithm follows is:

• When starting to use the fitness application, the camera records the user’s movements and actions during their exercise.
• This video is then carefully split into separate frames by the algorithm. It is then processed based on the human pose estimation model by detecting key points or landmarks on the user’s body. These keypoints are developed to form a virtual ‘skeletal’ structure of the user in either 2D or 3D formats.
• The mistakes or errors in the exercise are easily identified by analyzing the virtual ‘skeletal’ structure using geometry-based techniques.
• These errors are communicated to the user, who can then make corrections based on the recommendations.

Challenges in Pose Estimation

Let’s assume for a minute that an AI-based training application replaces your trainer. Would the application be able to deliver the same capabilities as your human trainer? Many companies that try to develop a fitness app intend it to be the perfect single solution covering all aspects of fitness and training. 

However, since the technology used is still in its infancy, most apps are still being ‘perfected.’ Having said that, it is a good idea to know what pitfalls and challenges app developers could face during the app development stage.

1. Men and Women Body Types

When developing human pose estimation models for fitness applications, it is essential to know that male and female body types are uniquely and physiologically different. The models should be trained for both male and female body types. Otherwise, the chances of it throwing up inaccurate results are significant.

Women use fitness applications twice more than men, and 54% of all surveyed were more likely to buy a body analyzing device for their health.

2. Physiology Differences

There is no ‘perfectly proportionate’ human body. We are disproportionate in one way or the other – either the length of our upper body, our legs, or even our arms. The model used for pose estimation is based on the number of images people have used for training. So, the pose estimation model will analyze the user’s body found on images used in training. If the training image dataset doesn’t have a model type that corresponds with the user’s body type, then output discrepancies could occur.

Moreover, comparing the exercise performance of a user with unusual body proportions with the standard model could also throw up less than accurate results.

3. Detection of the Beginning of the Exercise Session

AI in fitness apps should also take into consideration the start and end timings of exercise sessions. The app should also know from which point it is expected to start analyzing the exercise.

4. Limited Frontal View Error

When estimating the correctness of an exercise, two videos – a reference video and a user’s input video are analyzed. The data from both videos are taken out as frames, 3D keypoints analyzed and spotted on both, aligned to match and distances between user’s keypoints and reference model’s keypoints are analyzed.

Due to the limitations in the current datasets that do not have sufficient images of postures, movements, poses, and perspectives, it is tough to get a good analysis with a strictly frontal view.

5. Limited Lower Body Movements

AI in fitness apps are also proving insufficient in catching quick movements of the lower body -especially during squats or kicking in martial arts. The deep learning mechanism is unable to detect the fast movement of the leg during the kick.

6. Horizontal Position

Pushups are another exercise that the AI in fitness app is unable to detect correctly. Due to the insufficiency of visual data, the pose estimation model cannot detect 2D key points on a model doing horizontal push ups. However, when the user’s movements are analyzed vertically, they seem to work better.

Wrapping Up

These are some of the challenges that fitness app development company teams might have to overcome when developing apps based on pose estimation. However, it is possible to resolve these issues if there is a clear understanding of the AI-based fitness app’s requirements and outcome.