How an AI Form Check Actually Works, Rep by Rep

Upload a video of your squat and thirty seconds later an app tells you rep 3 cut depth and your knees caved on rep 5. To decide how much to trust that, it helps to know exactly what happens in between. This is a walkthrough of how AI video form analysis works, written by the team that builds one, including the parts where the technology has real limits.

Step 1: finding your body in every frame

The foundation is pose estimation: a neural network that locates your body's landmarks, frame by frame. HyperBody's form check uses pose detection of the MediaPipe family, which tracks 33 landmarks: eyes, ears, shoulders, elbows, wrists, hips, knees, ankles, and feet¹.

Two things matter about this step. First, the model estimates positions from what it can see, so a joint hidden behind a plate or another person gets a guess with a low confidence score, and good pipelines treat low-confidence joints with suspicion instead of trusting them. Second, the output is a skeleton over time: for a 30-second set at 24 frames per second, roughly 720 skeletons that together describe how your body moved.

Step 2: cutting the movement into reps

Raw skeletons are not feedback. The next stage finds the repetitions: the pipeline tracks a signal that rises and falls with the movement, like hip height for a squat or wrist height for an overhead press, smooths it to remove jitter, and cuts it into cycles. Each cycle is one rep, with a start, a bottom (or top), and an end.

This is also where honest engineering shows. Filming angle, partial reps, and re-racking the bar all produce ambiguous cycles, and a form checker has to decide what counts as a rep before it can judge one. When an app tells you it saw 6 reps and you did 8, this stage is where the disagreement happened.

Step 3: measuring what a coach watches

With reps isolated, the pipeline computes the numbers a good coach eyeballs:

• Depth: for a squat, the hip crease's position relative to the knee at the bottom of each rep.
• Joint angles: hip and knee angles at key moments, such as the hip angle at lockout in a deadlift.
• Bar or hand path: horizontal drift across the rep, which exposes the bar looping forward.
• Symmetry: left-right differences in angles or heights that suggest one side is doing more work.
• Tempo: how long the lowering and lifting phases took, and whether control degraded across the set.

Each lift has its own checklist. A bench press analysis cares about elbow flare and bar drift; a deadlift analysis cares about back rounding and lockout; a pull-up analysis cares about range of motion and kipping. That is why serious form checkers support a specific list of exercises rather than claiming to analyze anything: the measurements are lift-specific. HyperBody's covers 15 lifts, each with its own checks, described on the per-exercise pages like the squat form check and deadlift form check.

Step 4: turning numbers into a verdict

A measurement becomes feedback through thresholds: a hip crease that finishes above the knee by more than a tolerance is flagged as high, a knee angle collapsing inward past a bound is flagged as valgus. Findings carry a severity, so the feedback can distinguish a rep that missed depth by a centimeter from one that missed it by ten, and the worst, most consistent fault across the set leads the report rather than a wall of nitpicks.

What AI form checks honestly cannot do

The same pipeline that produces the feedback has real limits, and you should distrust any product that hides them:

• It cannot see what the camera cannot see. In a direct side view, the far side of your body is estimated, not observed. Filming angle is the single biggest factor in analysis quality.
• It does not know your load. Video shows positions, not effort: a grinder at 95% and an easy warm-up rep with identical form read the same.
• Bad lighting degrades everything. Pose models trained on visible bodies lose precision in dark footage and heavy occlusion.
• It measures movement, not pain or history. No video analysis knows about your old knee injury. It flags patterns; a clinician diagnoses.

These limits are why a form check is a feedback tool, not a medical device, and why the best use of one is trend-level: film a set each week, fix the leading fault, and watch whether it stops appearing.

How to film for the most accurate analysis

• One set per video, whole body in frame for the entire set, including the bar at the top.
• Camera at hip height, roughly perpendicular to the movement for squats, deadlifts, and presses.
• Even lighting, no backlight; avoid filming into a window.
• Keep other people out of frame so the tracker locks onto you.

Done that way, uploading a set takes under a minute and the analysis comes back with per-rep verdicts you can act on the same session. If you want to see what that looks like on your own lifting, the AI form check is free to try, and articles like weak point training and how to build a program that progresses cover what to do with the findings. All six demo clips live on the demo videos page if you want to see what well-filmed footage looks like.

---

References

Footnotes

1. Bazarevsky V et al. (2020). BlazePose: On-device Real-time Body Pose Tracking. arXiv:2006.10204. https://arxiv.org/abs/2006.10204