NParks Camera Trapping Video Animal Recognition, Using Foundational Models' Embeddings for Zero-shot Classification

As part of Singapore National Park’s Youth Stewards for Nature 2024 volunteer program, we built a system to recognize animals in camera trapping videos. In addition to fine-tuning a supervised trained model, we were able to use Foundational Models’ zero-shot learning capability to classify frames. This method is competitive with supervised finetuning on a large amount of in-domain data, while requiring no training, and re-formulating the model training task into a data curation task, making it more accessible to the laymen.