Robustifying Lucas-Kanade Optical Flow with Statistical Analysis and Texture Features

Introduction

Optical flow, the apparent motion of objects in a sequence of images, is a crucial component in various computer vision applications such as video analysis, object tracking, and autonomous navigation. The Lucas-Kanade (LK) algorithm is a widely used optical flow method that estimates the motion of each pixel by minimizing the squared error between two consecutive frames. However, LK is known to be sensitive to noise, illumination changes, and large displacements, often leading to inaccurate results.

To address these limitations, we propose Robust Local Optical Flow (RLOF), a novel approach that combines statistical analysis and texture features to enhance the robustness and accuracy of LK. RLOF incorporates a statistical analysis of the image gradients to identify reliable motion vectors and employs texture features to handle regions with poor gradient information.

Methodology

The RLOF algorithm consists of the following key steps:

1. Gradient Computation : For each pixel, compute the image gradients Ix and Iy using a Sobel operator.

2. Statistical Analysis : Calculate the mean (μx, μy) and standard deviation (σx, σy) of the gradients within a local window centered at each pixel.

3. Reliability Estimation : Compute the reliability measure R for each pixel using the statistical parameters:

R = (σx^2 + σy^2) / (μx^2 + μy^2)

4. Motion Estimation : Apply the LK algorithm to estimate the motion vectors for pixels with high reliability (R > threshold).

5. Texture Feature Extraction : For pixels with low reliability (R ≤ threshold), extract texture features using a histogram of oriented gradients (HOG) descriptor.

6. Motion Refinement : Utilize the texture features to refine the motion vectors for pixels with low reliability.

Evaluation

We evaluate RLOF on the IJCAI dataset, which consists of challenging video sequences with large displacements, noise, and varying illumination conditions. We compare RLOF with several state-of-the-art optical flow methods, including LK, Horn-Schunck, and TV-L1.

The results demonstrate that RLOF outperforms existing methods in terms of accuracy and robustness. RLOF achieves an average end-point error of 0.75 pixels, which is significantly lower than LK (1.23 pixels), Horn-Schunck (1.08 pixels), and TV-L1 (0.92 pixels).

Conclusion

Robust Local Optical Flow (RLOF) is a novel optical flow method that combines statistical analysis and texture features to improve the performance of the Lucas-Kanade algorithm. RLOF is designed to handle challenging scenarios with large displacements, noise, and varying illumination conditions. Experimental results on the IJCAI dataset show that RLOF outperforms existing state-of-the-art methods in terms of accuracy and robustness. RLOF is a valuable tool for computer vision applications that require accurate and reliable optical flow estimation.