The explosion in unstructured data, such as images, videos, sound records, and text, requires an effective solution for computer vision, voice recognition, and natural language processing. How to extract value from unstructured data poses as a big challenge for many enterprises.
AI, especially deep learning, has been proved as an effective solution. Vectorization of data features enables people to perform content-based search on unstructured data. For example, you can perform content-based image retrieval, including facial recognition and object detection, etc.
The AI search engine
Now the challenge turns into how to execute effectively search among billions of vectors. That’s what Milvus is designed for.
It is an open source distributed vector search engine that provides state-of-the-art similarity search and analysis of feature vectors and unstructured data. Some of its key features are:
You may follow this link for step-by-step procedures to carry out performance test on 100 million vector search (SIFT1B). If you want, you can also try testing 1 billion. Here is the hardware requirements. Milvus has been open sourced lately. We greatly welcome contributors to join us in reinventing data science. See us on GitHub.