Abstract:

Real-time perception and continuous tracking of unknown objects are critical for autonomous intelligent systems. However， the absence of prior category knowledge and limited training samples make the perception and tracking of unknown targets highly challenging. To address this issue. we propose a category-agnostic object tracking method based on the Segment Anything Model （SAM） and sparse feature point matching. The approach first guides SAM to segment unknown objects using prompt points， then extracts sparse keypoints via a network-based feature extraction model， and matches them across frames using an attention-based network to propagate object information. An Iterative SAM with Point Consensus （ISPC） is introduced to maintain segmentation and achieve stable tracking over time. The lightweight target descriptors based on sparse feature points can be efficiently shared among multiple agents， enabling the construction of a collaborative target tracking system. Experiments on the DAVIS 2017 dateset and a self-constructed near-infrared video dataset demonstrate strong robustness and accuracy in collaborative perception and tracking of unknown-category objects.

Key words: object tracking, object segmentation, feature extraction, feature matching, collaborative perception