preprint · arXiv (Cornell University) · 2023

Curriculum Learning for Compositional Visual Reasoning

Aissa, Wafa, Ferecatu, Marin, Crucianu, Michel

Cited by 0Open access

Abstract

Visual Question Answering (VQA) is a complex task requiring large datasets and expensive training. Neural Module Networks (NMN) first translate the question to a reasoning path, then follow that path to analyze the image and provide an answer. We propose an NMN method that relies on predefined cross-modal embeddings to ``warm start'' learning on the GQA dataset, then focus on Curriculum Learning (CL) as a way to improve training and make a better use of the data. Several difficulty criteria are employed for defining CL methods. We show that by an appropriate selection of the CL method the cost of training and the amount of training data can be greatly reduced, with a limited impact on the final VQA accuracy. Furthermore, we introduce intermediate losses during training and find that this allows to simplify the CL strategy.

Cite this paper

APA 7

Aissa, W., Ferecatu, M., & Crucianu, M. (2023). Curriculum Learning for Compositional Visual Reasoning. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2303.15006

MLA 9

Aissa, Wafa, et al. “Curriculum Learning for Compositional Visual Reasoning.” arXiv (Cornell University), 2023, https://doi.org/10.48550/arxiv.2303.15006.

Chicago

Aissa, Wafa, Marin Ferecatu, and Michel Crucianu. 2023. “Curriculum Learning for Compositional Visual Reasoning.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2303.15006.

Harvard

Aissa, W., Ferecatu, M. and Crucianu, M. (2023) ‘Curriculum Learning for Compositional Visual Reasoning’, arXiv (Cornell University). Available at: https://doi.org/10.48550/arxiv.2303.15006.

BibTeX

@misc{aissa2023curriculum,
  title = {{Curriculum Learning for Compositional Visual Reasoning}},
  author = {Aissa, Wafa and Ferecatu, Marin and Crucianu, Michel},
  journal = {arXiv (Cornell University)},
  year = {2023},
  doi = {10.48550/arxiv.2303.15006}
}

Made from the index record. Check names, capitalisation and page numbers against the paper. Need another style? AI for Research has over 10,000.