A PyTorch Lightning solution to training CLIP from scratch
train-CLIP
A PyTorch Lightning solution to training CLIP from scratch.
Usage 🚂
This training setup is easily usable right outside the box! Simply provide a training directory or your own dataset and we’ve got the rest covered. To train a model just specify a name from the paper name and tell us your training folder and batch size. All possible models can be seen in the yaml files in models/config
python train.py --model_name RN50 --folder data_dir --batchsize 512
Training with our DataModule 📉
As long as each of the image pairs have the same stem name (i.e. coco_img1.png and coco_img1.txt) all that you need to do is specify the folder on runtime. Any subfolder structure will be ignored, meaning foo/bar/image1.jpg will always find