Description

Contrastive Language-Audio Pretraining (CLAP) is an opensoruce library that enable to train a neural network on both Audio and Text description enabling the possibility to search Audio from a Text input. On huggingface are also already present some pre-trained model to search on Songs.

Goals

Check how CLAP can be used for searching on song and which kind of query can give a good result by developing a Minimum Viable Product (MVP) in Python. Based on the result of this MVP, the future next step could be:

• Music Tagging;
• Free text search;
• Integration with an LLM model (for example with MCP or OpenAPI api) for music suggestions on your own library

Resources

• CLAP: the model object of the research;
• huggingface: pre-trained model for CLAP;
• Free Music Archive: creative common songs that can be used for testing;
• Colab: to be used as development envirorment.