Low overlap stereo cameras

[mateosss]

Hello, I have a pair of stereo cameras from a VR headset that point at different angles. I've been able to get HybVIO to track stereo features only with -useRectification, but even then it doesn't seem possible for HybVIO to track features on the non-overlapping areas of the images.

1. Is there a way to avoid having to prerectify the images in this case?
2. Is it possible to track non-overlapping features in HybVIO?

Below is a stereo frame example obtained by running this command:

./main -i=../data/benchmark/ody-easy -p -c -s -windowResolution=640 -useSlam -useStereo -displayStereoMatching -useRectification

Over this HybVIO-formatted dataset. The original EuRoC-formatted dataset can be found here.

[oseiskar]

Hello! Your findings are consistent with our understanding of the capabilities of the current HybVIO code

1. Sometimes rectification is needed in practice or the left-to-right LK tracking fails (even though it "should not"). The current rectification implementation is very slow, which could be worked around by rectifying manually with, e.g, OpenCV before feeding the data to HybVIO (this also needs to be reflected in the calibration in parameters.txt). There are currently no work-arounds for the need for rectification in this case, except fixing/modifying the stereo LK-tracking implementation
2. The current code does not have the capability of tracking non-overlapping features in the case of -useStereo. It should be possible to extend the algorithm to support them

[mateosss]

I see, thank you!