Gait Analysis Tool

Important

This application was created with extensive use of agent-based artificial intelligence.

An original data analysis algorithm and graphical interface design were implemented. Approaches to video analysis were implemented using AI under careful human supervision, and elements of the graphical interface were created by agent-based AI in accordance with the original design.

Overview

Gait Analysis Tool is specialized software for automated and manual gait analysis of animals (e.g., rodents) based on video recordings (CatWalk-type systems). The program uses computer vision methods for body and paw detection, as well as tracking algorithms (Kalman Filter) to monitor movement trajectories.

Key Features

1. Video Loading and Viewing

Drag-and-Drop: Support for dragging video files (.mp4, .avi) directly into the program window.
Zoom & Pan: Ability to zoom (mouse wheel) and pan video (left mouse drag) for detailed examination.
Navigation: Frame-by-frame viewing (left/right arrows), playback speed control (Play/Pause).
Range Selection: Time range selection for processing (Start Time / End Time).
Reset Zoom/Pan: Quick return to initial zoom and position.

2. Detection

Static Corridor Detection: Automatic detection of red corridor (movement zone) based on static background.
Body Detection: Body silhouette extraction based on brightness threshold (Dark Threshold) and movement (Movement Threshold).
Paw Detection: Paw detection by color (Green HSV Threshold) within the corridor.
Paw Grouping: Automatic merging of individual paw pads into a single paw based on distance (Max Paw Grouping).
Spatial Filtering: Artifact filtering based on distance from body silhouette (Max Paw-Sil Dist).
Live Color Preview: Dynamic display of detected color in settings (Color Preview widget).

3. Tracking

Simple Mode: Basic detection without preserving identifiers between frames.
Global Tracking Mode: Using Kalman Filter and Hungarian Algorithm to build stable paw tracks.
ID Persistence: Preserving paw identifiers even during temporary disappearance (occlusion).
Color Coding: Each track has a unique color for visualization.
Kalman Filter Settings: Flexible adjustment of process/measurement noise for balance between smoothness and reactivity.

4. Annotation

Manual Labeling: Assigning labels to paws: F_L (Front Left), F_R (Front Right), R_L (Rear Left), R_R (Rear Right).
Smart Propagation: Automatic label propagation to all frames where this paw is present (forward and backward in time), based on bounding box overlap (IoU > 0).
Area Graph: Visualization of paw contact area over time with color coding for each paw.
Interactive Cursor: Clicking on the graph moves video to corresponding frame.
Visual Indication: Labeled paws are displayed with colored bounding boxes and labels.

5. Export

Video Export: Saving video with overlaid detection masks, tracks, and labels in MP4 format.
TIFF Stack Export: Exporting 3D mask stack (Time, Y, X) in TIFF format, where each paw has a unique pixel value (for further analysis in ImageJ/Fiji or Python).
- 0: Background
- 1: Unlabeled paw
- 2: F_L (Front Left)
- 3: F_R (Front Right)
- 4: R_L (Rear Left)
- 5: R_R (Rear Right)

Installation and Launch

Requirements

The program is written in Python 3.7+ and requires the following libraries:

opencv-python (cv2) — video and image processing
numpy — mathematical operations with arrays
scipy — Hungarian algorithm for assignment problem
qtpy — wrapper for PyQt5/PySide2
PyQt5 (or PySide2) — graphical interface
tifffile — TIFF stack export

Installing Dependencies

pip install opencv-python numpy scipy qtpy PyQt5 tifffile

Launch

python app_run.py

Note: The program has been refactored into a modular architecture:

analysis.py — mathematical apparatus and computer vision algorithms

run.py — user interface

The old file application.py is left for compatibility, but it is recommended to use app_run.py.

Module Structure

The program is composed of two main modules:

analysis.py (Analytical Engine)

Purpose: Mathematical apparatus and computer vision algorithms

Main classes:

ProcessorConfig — configuration of processing parameters
GaitVideoProcessor — main video processor
TrackerManager — tracker manager (Kalman + Hungarian Algorithm)
KalmanTracker — Kalman filter based tracker

Usage:

from analysis import GaitVideoProcessor, ProcessorConfig
import cv2

# Configuration creation
config = ProcessorConfig()
config.green_lower_h = 40
config.paw_grouping_distance = 120

# Processor initialization
processor = GaitVideoProcessor(config)
processor.calculate_static_corridor('video.mp4')

# Frame analysis
frame = cv2.imread('frame.jpg')
results = processor.analyze_frame_data(frame)
print(f"Found {len(results['paws'])} paws")

run.py (UI Layer)

Purpose: Graphical interface on Qt

Main components:

MainWindow — main window of the program
AreaGraphWidget — contact area graph
ZoomableVideoWidget — video display widget
VideoProcessingThread — background video processing

Advantages of modular architecture:

Testing: analysis.py can be tested independently
Reusability: Analytical engine can be used in CLI, Jupyter, web services
Support: Clear separation of logic and UI

User Guide

Basic Workflow

Loading Video:
- Drag file into window or click "Load Video" button.
- Program will automatically detect static corridor and background.
- Video will load, but processing will not start automatically.
Parameter Settings (Detection/Tracking Tab):
- Body Detection:
  - Adjust Dark Threshold (recommended 60-100) for correct body extraction.
  - Adjust Movement Threshold (recommended 15-35) for noise filtering.
- Paws Detection:
  - Adjust Green Min Hue/Sat/Val, guided by Color Preview window, so color matches paw color in video.
  - Adjust Max Paw Grouping (recommended 80-120) for pad merging.
  - Adjust Max Paw-Sil Dist (recommended 30-70) for artifact filtering.
- Click "Process Video" to apply settings and see preview result.
Using Global Tracking (optional):
- Enable checkbox "Use Global Tracking (IDs & Colors)".
- Adjust tracking parameters:
  - Max Missing Frames: 20-40 for stable tracks.
  - Assignment Distance: 100-200 depending on movement speed.
  - Process Noise: 0.02-0.05 (lower = smoother, higher = more reactive).
  - Measurement Noise: 0.3-0.7 (higher = smoother).
- Click "Process Video" to recalculate with tracking.
Annotation (Annotation Tab):
- Switch to Annotation tab.
- Select radio button with desired label (F_L, F_R, R_L, R_R).
- Click on paw bounding box in video.
- Label will automatically apply to entire step sequence of this paw (forward and backward).
- Use Left/Right arrows to navigate between frames.
- Use Up/Down arrows to change selected label.
- Monitor area graph at bottom for labeling quality control.
Export:
- Return to Detection/Tracking tab.
- Click "Export TIFFs" to get masks (for quantitative analysis).
- Click "Export Video" for result visualization.

Keyboard Shortcuts

General

Space: Start/Stop video playback (in both tabs).
Left/Right Arrows: Previous/Next frame.
Mouse Wheel: Video zoom (in viewing widget).
Mouse Drag (Left Button): Pan zoomed video.

Annotation Mode

Up/Down Arrows: Change selected label (F_L → F_R → R_L → R_R).
Mouse Click: Assign label to paw (click on bounding box).

Parameters Description

Body Detection

Dark Threshold

Range: 0-255
Recommended: 60-100
Description: Brightness threshold for extracting dark body silhouette on light background. Lower values make detection stricter (only very dark pixels), higher values make it softer.
When to change: If body is poorly extracted or too much background is captured.

Movement Threshold

Range: 0-100
Recommended: 15-35
Description: Threshold for movement detection relative to static background. Higher values ignore small movements and noise.
When to change: If static objects are detected or slow movements are missed.

Paws Detection

Max Paw Grouping

Range: 20-300 pixels
Recommended: 80-120
Description: Maximum distance between individual contact spots (pads) to consider them one paw. Larger values merge more spots.
When to change: If individual pads of one paw don't merge or paws standing close together merge.

Max Paw-Sil Dist

Range: 0-200 pixels
Recommended: 30-70
Description: Maximum distance a paw can be from body silhouette to be considered valid. Filters artifacts outside body.
When to change: If artifacts are detected far from body or paws are missed in extended pose.

Green Min Hue

Range: 0-180 (OpenCV HSV)
Recommended: 30-50
Description: Minimum hue value for green color detection of paws.
When to change: Guided by Color Preview — color should match paw color in video.

Green Min Sat

Range: 0-255
Recommended: 30-60
Description: Minimum saturation for green color detection. Lower values allow detecting paler shades.
When to change: If paws have low color saturation or too much noise is detected.

Green Min Val

Range: 0-255
Recommended: 30-60
Description: Minimum brightness (Value/Brightness) for green color detection. Filters dark areas.
When to change: If paws are in shadow or dark artifacts are detected.

Paws Tracking — Global Tracking Mode

Max Missing Frames

Range: 5-60 frames
Recommended: 20-40
Description: Maximum number of consecutive frames a track can exist without new detections (occlusion). After this, track is deleted.
When to change: Increase for frequent occlusions, decrease for faster removal of stale tracks.

Assignment Distance

Range: 50-300 pixels
Recommended: 100-200
Description: Maximum distance between predicted track position (Kalman prediction) and new detection for linking them.
When to change: Increase for fast movements, decrease to avoid false associations.

Process Noise

Range: 0.01-0.1
Recommended: 0.02-0.05
Description: Kalman filter parameter determining motion model uncertainty. Higher values make filter more reactive to trajectory changes but may add jitter.
When to change: Increase for sharp direction changes, decrease for smoother tracks.

Measurement Noise

Range: 0.1-2.0
Recommended: 0.3-0.7
Description: Kalman filter parameter determining trust in measurements. Higher values make filter trust model more (smoother tracks), lower values trust detections more (more accurate but jittery).
When to change: Increase for noisy detections to smooth, decrease for more precise detection following.

Technical Details

Detection Algorithm

Static Background: Average frame values computed to extract static elements (corridor, background).
Corridor Extraction: Red area detection (HSV thresholding) to determine movement zone.
Body Extraction: Combination of darkness threshold and movement detection (frame differencing).
Paw Extraction: HSV thresholding for green color within corridor.
Grouping: Merging nearby contours into single paws based on Euclidean distance between centroids.
Filtering: Removing paws located too far from body silhouette.

Tracking Algorithm (Global Mode)

Initialization: Creating new track for each detection in first frame.
Prediction: Kalman filter predicts next position of each track.
Association: Hungarian algorithm finds optimal matching between predictions and detections (minimizing total distance).
Update: Tracks updated with corresponding detections, new detections create new tracks.
Deletion: Tracks without detections for Max Missing Frames are deleted.

TIFF Export Format

Dimensions: (T, H, W) — Time, Height, Width
Data type: uint8
Pixel values: 0 (background), 1 (unlabeled paw), 2-5 (labeled paws)
Compatibility: ImageJ/Fiji, Python (tifffile, scikit-image), MATLAB

Troubleshooting

Paws Not Detected

Check Green Min Hue/Sat/Val settings using Color Preview.
Decrease Green Min Sat or Green Min Val to detect paler shades.
Verify paws are within corridor boundaries.

Artifacts Detected

Increase Green Min Sat to filter unsaturated colors.
Decrease Max Paw-Sil Dist to filter objects far from body.
Increase Movement Threshold to ignore static artifacts.

Unstable Tracks (Frequently Switch)

Increase Max Missing Frames to preserve tracks during occlusions.
Increase Measurement Noise for smoother tracks.
Decrease Process Noise for less reactive filter.

Tracks Don't Link After Occlusion

Increase Assignment Distance for larger search radius.
Increase Max Missing Frames for longer track preservation.

Exported TIFF is Empty

Ensure video is processed (clicked "Process Video").
Check if detections are visible in video preview.
Verify labeling in Annotation mode before export.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
pic		pic
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
README_ua.md		README_ua.md
analysis.py		analysis.py
app_run.py		app_run.py
application.py		application.py

License

Belan-Lab/SFtIr

Folders and files

Latest commit

History

Repository files navigation