Subscribe to DSC Newsletter

Smarter Housing Search and Recommendation Powered by Milvus

Image for post

Project Background

In recent years, Proptech has seen huge momentum and Artificial Intelligence especially is now playing a major role in people’s buying/selling decisions on real estate transactions.

For housing platforms and their agents, selecting the houses that can be sold quickly from a massive number of houses is often crucial and challenging. So, how can we search for a series of unstructured data such as house plans and house information more efficiently? This is a question many tech companies are now trying to answer.

In this case study, we work with Beike.com, which is a leading online real estate brokerage platform similar to Zillow or Realtor.com to show you what is their approach to provide better housing search and recommendation.

It is understood that traditional databases cannot process unstructured data such as image, voice, and natural language. However, everything can be embedded with the help of Artificial Intelligence. By mapping unstructured data into multidimensional vectors (i.e., Embedding), we can easily analyze these data using vector search tools.

To improve house analysis, such as recommending houses that best match users’ interests, Beike is moving towards vector search. 

Import Milvus

Milvus supports vectors converted by multiple AI models and provides corresponding search and analysis services. Its services include image processing, computer vision, natural language processing, speech recognition, recommender system, new drug discovery, etc. The general implementation process is as follows:

  1. The user uses deep learning models to convert unstructured data to feature vectors and import them into the Milvus library.
  2. Milvus stores and indexes the feature vectors.
  3. After receiving a vector query from the user, Milvus outputs a result similar to the input vector.

Image for post

The figure below illustrates a house search scenario, where Milvus searches for candidates similar to a given house through the following steps:

  1. According to features such as floor plan, area, outline, and orientation, use a machine learning model to extract 4 collections of feature vectors from the input house information.
  2. Use the 4 collections to perform similarity searches in the Milvus library. The search result of each collection of vectors indicates the similarity between the similar house and the input house in terms of the corresponding feature.
  3. Compare the search results of the 4 collections of vectors, and output the recommended house.

Image for post

In the figure above, the system adopts A/B table switching to update data on the T+1 day. In other words, Milvus stores the data of T days in Table A, and imports the updated data of the previous T days in Table B on the T+1 day. In this way, the system is not affected when the data is updated.

With the help of Miluvs, Beike Vector Search Platform has implemented high-performance analysis of massive feature vectors. The average response time for Milvus to search for 3 million data in a standalone environment is 113 ms. Also, Milvus supports distributed cluster solutions. Beike Vector Search Platform uses Kubernetes to deploy a distributed Milvus cluster, which has a performance comparable to that of a standalone solution and is ready for horizontal scalability and high availability.

Introduction to the house recommendation system

The house recommendation system employs a similarity-based recommendation strategy. When a user shows apparent interest in a certain house, similar houses are recommended for the user. The core of most recommendation algorithms is to calculate similarity, such as user similarity and item similarity. Next, we will briefly introduce the similarity recommendation algorithm:

  • Basic idea

The system finds similar houses based on the user’s favorite houses, and then recommends houses with higher similarity.

  • Algorithms

Milvus supports many similarity calculation methods, such as Euclidean distance, cosine distance, and Tanimoto coefficient. In the house recommendation scenario, we use cosine distance.

  • Steps
  1. Collect user preferences: Convert information of a user’s favorite houses to feature vectors for future processing.
  2. Search for similar houses: Calculate the cosine distance between the user’s favorite houses and other houses. Based on the calculated distances, recommend a sorted list of similar houses.
  • Implementation

The house recommendation system is similar to the abovementioned house search system. It can quickly retrieve a list of similarity results and recommend houses based on user preferences and with high accuracy.

More scenarios

There is no doubt that the vector search service of Beike Search Platform will go beyond the similarity analysis of houses. It plans to use Milvus to analyze the feature vectors of a series of unstructured data (such as images, videos, voices, and texts) to provide users with a more intelligent, more comprehensive, and more personalized house recommendation system. Milvus is committed to being a dominant general AI data processing platform in the AI industry. By combining mature AI models, Milvus can quickly apply AI technologies to various scenarios. We believe we can create a fast, comprehensive, and smart search platform based on Milvus.

Conclusion

Beike Search Platform mainly deals with house search and recommendation service. At present, Beike has a file search platform based on ElasticSearch, a graph database based on Dgraph, and a vector search platform based on Milvus. Beike began to explore vector search at the end of last year. In the field of AI, feature vector is a common data format, and feature vector similarity search engine is an important component for processing the fast-growing unstructured data. In the future, the demand for vector search will definitely explode, and vector search engines will also become a trend.

To get to know more about Milvus project, you can visit: https://github.com/milvus-io/milvus

Written and authorized by:

Yaofei Sun, Senior Engineer of Beike.com

ShiYu Chen, Zilliz Data Engineer

Views: 67

Tags: AI, cosine distance, embedding, feature vector, recommender system, similarity search

Comment

You need to be a member of Data Science Central to add comments!

Join Data Science Central

Videos

  • Add Videos
  • View All

© 2020   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service

console.log("HostName");