other · Underline Science Inc. · 2024

Weakly Supervised Open-Vocabulary Object Detection

Association for Artificial Intelligence 2024, Cao, Liujuan, Li, Ke, Lin, Shaohui, Lin, Jianghang Lin, Shen, Yunhang, Wang, Bingquan

Cited by 0Open access

Abstract

Despite weakly supervised object detection (WSOD) being a promising step toward evading strong instance-level annotations, its capability is confined to closed-set categories within a single training dataset. In this paper, we propose a novel weakly supervised open-vocabulary object detection framework, namely WSOVOD, to extend traditional WSOD to detect novel concepts and utilize diverse datasets with only image-level annotations. To achieve this, we explore three vital strategies, including dataset-level feature adaptation, image-level salient object localization, and region-level vision-language alignment. First, we perform data-aware feature extraction to produce an input-conditional coefficient, which is leveraged into dataset attribute prototypes to identify dataset bias and help achieve cross-dataset generalization. Second, a customized location-oriented weakly supervised region proposal network is proposed to utilize high-level semantic layouts from the category-agnostic segment anything model to distinguish object boundaries. Lastly, we introduce a proposal-concept synchronized multiple-instance network, i.e., object mining and refinement with visual-semantic alignment, to discover objects matched to the text embeddings of concepts. Extensive experiments on Pascal VOC and MS COCO demonstrate that the proposed WSOVOD achieves new state-of-the-art compared with previous WSOD methods in both close-set object localization and detection tasks. Meanwhile, WSOVOD enables cross-dataset and open-vocabulary learning to achieve on-par or even better performance than well-established fully-supervised open-vocabulary object detection (FSOVOD).

Cite this paper

APA 7

2024, A. F. A. I., Cao, L., Li, K., Lin, S., Lin, J. L., Shen, Y., & Wang, B. (2024). Weakly Supervised Open-Vocabulary Object Detection. Underline Science Inc.. https://doi.org/10.48448/17gg-qe74

MLA 9

2024, Association for Artificial Intelligence, et al. “Weakly Supervised Open-Vocabulary Object Detection.” Underline Science Inc., 2024, https://doi.org/10.48448/17gg-qe74.

Chicago

2024, Association for Artificial Intelligence, Liujuan Cao, Ke Li, Shaohui Lin, Jianghang Lin Lin, Yunhang Shen, and Bingquan Wang. 2024. “Weakly Supervised Open-Vocabulary Object Detection.” Underline Science Inc.. https://doi.org/10.48448/17gg-qe74.

Harvard

2024, A.F.A.I. et al. (2024) ‘Weakly Supervised Open-Vocabulary Object Detection’, Underline Science Inc.. Available at: https://doi.org/10.48448/17gg-qe74.

BibTeX

@misc{20242024weakly,
  title = {{Weakly Supervised Open-Vocabulary Object Detection}},
  author = {2024, Association for Artificial Intelligence and Cao, Liujuan and Li, Ke and Lin, Shaohui and Lin, Jianghang Lin and Shen, Yunhang and Wang, Bingquan},
  journal = {Underline Science Inc.},
  year = {2024},
  doi = {10.48448/17gg-qe74}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.