about dataset

[1106280506Hx]

I want to collect some data to perform point tracking in specific scenarios. Does this require the cameras to be continuous (without cuts)?

[yangyi02]

Hi @1106280506Hx ,

No, the video does not need to be strictly continuous. For example, the TAPVid-Kinetics dataset contains quite a few cuts and still functions well for point tracking benchmarks.

However, note that cuts can make the annotation process slightly more challenging, particularly if you are using our optical flow-based Track Assist Algorithm, as optical flow relies on temporal consistency between nearby frames. That said, in practice, we didn't encounter any fundamental issues when annotating TAPVid-Kinetics despite the cuts.

Hope this helps!

[1106280506Hx]

So, if I track something in my video, but there is a part where it transitions from one scene to another while the tracked object remains the same, is that okay? Also, are there any other requirements for the video?

[yangyi02]

@1106280506Hx

Now that I review your question, I realize I misunderstood it earlier. You are likely asking whether our TAPIR/TAPNext algorithm can perform well on videos that contain cuts.

The short answer is that we don’t know for sure. You are welcome to give it a try—there is nothing in the algorithm that explicitly prevents tracking across cuts. That said, during training, the model only sees continuous videos, so performance on videos with cuts is not something we have specifically focused on.

In terms of input requirements, there are no strict constraints on the video. However, it is preferable that the video is not overly long, has a roughly square resolution, and does not contain extremely fast motion or significant motion blur.