BabyVLM: Democratizing Research on the Pretraining of Vision Large Language Models

Abstract

"What I cannot create, I do not understand." — Richard Feynman

Pretraining vision foundation models (VFMs) is prohibitively expensive, making it a privilege for institutions with abundant resources and leaving independent researchers to downstream tasks such as benchmarking, interpreting, and aligning VFMs. This situation is a crisis for computer vision research: independent researchers and the public cannot gain a true understanding, trust, or safe use of VFMs passively from open weights or APIs.

We propose democratizing VFM pretraining by scaling it down to a developmentally plausible framework that is scientifically reasonable and computationally friendly to university budgets. Our goal is to promote exploration rather than exploitation of pretraining, enabling independent researchers to build general-purpose VFMs that approach "baby intelligence" — benefiting efforts toward "grown-up" AI.

This framework closely mimics the minimal yet highly informative sensory experiences of human infants, encompassing three pillars:

Tutorial Schedule

Presenters

Resources

All materials will be made available before the tutorial date. Links will be updated here.

Citation

If you find this work useful, please cite:

      TBD