What is cosine similarity, and how is it used with embeddings?

Cosine similarity is a measure of how similar two vectors are, based on the cosine of the angle between them. It ranges from -1 (completely opposite) to 1 (identical direction), with 0 indicating orthogonality (no correlation). In machine learning, cosine similarity is widely used to compare embeddings—numeric representations of data like text, images, or user behavior—in high-dimensional spaces. Unlike raw distance metrics, cosine similarity focuses on directional alignment, making it effective for tasks where the relative orientation of vectors matters more than their magnitudes. For example, in natural language processing (NLP), word embeddings (like those from Word2Vec or BERT) encode semantic meaning into vectors, and cosine similarity helps quantify how semantically related two words or sentences are.

To compute cosine similarity, you take the dot product of two vectors and divide it by the product of their magnitudes (L2 norms). If embeddings are normalized to unit length (common in practice), the formula simplifies to just the dot product, which is computationally efficient. For instance, when using pre-trained embeddings in Python, you might normalize them with libraries like NumPy and then compute similarity via matrix multiplication. This efficiency makes cosine similarity scalable for large datasets. Embeddings often capture abstract features (e.g., semantic meaning in text), and their high-dimensional nature makes cosine similarity a natural fit—it handles sparse or dense vectors effectively, focusing on patterns rather than absolute differences.

Developers use cosine similarity with embeddings in applications like search engines, recommendation systems, and clustering. For example, in a search engine, query and document embeddings are compared using cosine similarity to rank results by relevance. In recommendation systems, user and item embeddings are compared to suggest products or content. A concrete example is matching user-generated text (e.g., a search for “cozy coffee shops”) to pre-embedded database entries (e.g., café descriptions) to find semantic matches. Similarly, in customer feedback analysis, embeddings of support tickets can be clustered via cosine similarity to identify recurring themes. By leveraging directional similarity, cosine similarity provides a robust way to measure relationships in embedding spaces, balancing accuracy and computational practicality.