语义搜索

语义搜索理解用户查询背后的意思，而不是只看精确的关键词。它利用机器学习来捕捉查询的意图和上下文，应对同义词、表达方式的不同以及词语之间的关系等语言细微差别。

🌐 Semantic search interprets the meaning behind user queries rather than exact keywords. It uses machine learning to capture the intent and context behind the query, handling language nuances like synonyms, phrasing variations, and word relationships.

什么时候使用语义搜索 #

🌐 When to use semantic search

语义搜索在那些需要深入理解和上下文来提供相关结果的应用中非常有用。一个很好的例子是在客户支持或知识库搜索引擎里。用户常常会用各种方式来表达他们的问题或疑问，而传统的基于关键词的搜索可能无法总是找到最有用的文档。借助语义搜索，系统可以理解查询背后的含义，并将其与相关的解决方案或文章匹配，即使表述方式不同也没问题。

🌐 Semantic search is useful in applications where the depth of understanding and context is important for delivering relevant results. A good example is in customer support or knowledge base search engines. Users often phrase their problems or questions in various ways, and a traditional keyword-based search might not always retrieve the most helpful documents. With semantic search, the system can understand the meaning behind the queries and match them with relevant solutions or articles, even if the exact wording differs.

例如，一个用户搜索“在显示器上增大文字大小”时，可能会遗漏标题为“如何在设置中调整字体大小”的文章，尤其是在基于关键词的搜索系统中。然而，语义搜索引擎可以理解查询背后的意图，并正确地将其匹配到相关的文章上，无论使用的具体术语如何。

🌐 For instance, a user searching for "increase text size on display" might miss articles titled "How to adjust font size in settings" in a keyword-based search system. However, a semantic search engine would understand the intent behind the query and correctly match it to relevant articles, regardless of the specific terminology used.

也可以将语义搜索与关键词搜索结合起来，从而获取两者的优势。更多详情请参见混合搜索。

🌐 It's also possible to combine semantic search with keyword search to get the best of both worlds. See Hybrid search for more details.

语义搜索是如何工作的 #

🌐 How semantic search works

语义搜索使用一种叫做“嵌入向量”的中间表示来将数据库记录与搜索查询关联起来。在语义搜索的上下文下，向量是一组数值，它们表示文本的各种特性，并允许对不同文本之间进行语义比较。

🌐 Semantic search uses an intermediate representation called an “embedding vector” to link database records with search queries. A vector, in the context of semantic search, is a list of numerical values. They represent various features of the text and allow for the semantic comparison between different pieces of text.

理解嵌入向量的最好方式是把它们画在图上，每个嵌入就是一个点，它的坐标就是向量里的数值。重要的是，嵌入向量会被画在图上，使得相似的概念彼此靠近，而不相似的概念则远离。更多详情，请参见 什么是嵌入向量？

🌐 The best way to think of embeddings is by plotting them on a graph, where each embedding is a single point whose coordinates are the numerical values within its vector. Importantly, embeddings are plotted such that similar concepts are positioned close together while dissimilar concepts are far apart. For more details, see What are embeddings?

嵌入向量是通过语言模型生成的，然后用相似度度量来比较这些嵌入向量。语言模型被训练来理解语言的语义，包括语法、上下文以及单词之间的关系。它会为数据库中的内容和搜索查询都生成嵌入向量。接着，通常使用余弦相似度或点积这样的函数来比较查询的嵌入向量和文档的嵌入向量（换句话说，就是看它们在图上的距离有多近）。与查询嵌入向量最相似的文档就被认为是最相关的，并作为搜索结果返回。

🌐 Embeddings are generated using a language model, and embeddings are compared to each other using a similarity metric. The language model is trained to understand the semantics of language, including syntax, context, and the relationships between words. It generates embeddings for both the content in the database and the search queries. Then the similarity metric, often a function like cosine similarity or dot product, is used to compare the query embeddings with the document embeddings (in other words, to measure how close they are to each other on the graph). The documents with embeddings most similar to the query's are deemed the most relevant and are returned as search results.

嵌入模型 #

🌐 Embedding models

现在有许多嵌入模型可用。Supabase Edge Functions 对 gte-small 模型有内置支持。其他模型可以通过第三方 API 访问，比如 OpenAI，你只需在请求中发送你的文本，就能在响应中收到一个嵌入向量。还有一些模型可以在你自己的计算机上本地运行，比如使用 JavaScript 的 Transformers.js。想了解更多本地实现的信息，请参见生成嵌入。

🌐 There are many embedding models available today. Supabase Edge Functions has built-in support for the gte-small model. Others can be accessed through third-party APIs like OpenAI, where you send your text in the request and receive an embedding vector in the response. Others can run locally on your own compute, such as through Transformers.js for JavaScript implementations. For more information on local implementation, see Generate embeddings.

记住一点很重要：在使用嵌入模型进行语义搜索时，你必须对所有嵌入比较使用同一个模型。用不同模型创建的嵌入进行比较是没有意义的。

🌐 It's crucial to remember that when using embedding models with semantic search, you must use the same model for all embedding comparisons. Comparing embeddings created by different models will yield meaningless results.

Postgres 中的语义搜索 #

🌐 Semantic search in Postgres

要在 Postgres 中实现语义搜索，我们使用 pgvector——一个允许高效存储和检索高维向量的扩展。这些向量是由嵌入模型生成的文本（或其他类型数据）的数值表示。

🌐 To implement semantic search in Postgres we use pgvector - an extension that allows for efficient storage and retrieval of high-dimensional vectors. These vectors are numerical representations of text (or other types of data) generated by embedding models.

1. 通过运行以下命令启用 pgvector 扩展：

   1
   create extension vector
   2
   with
   3
     schema extensions;

2. 创建一个表来存储嵌入：

   1
   create table documents (
   2
     id bigint primary key generated always as identity,
   3
     content text,
   4
     embedding extensions.vector(512)
   5
   );

   或者如果你已经有一个表格，你可以像这样添加一个向量列：

   1
   alter table documents
   2
   add column embedding extensions.vector(512);

   在这个例子中，我们创建了一个名为 embedding 的列，它使用了新启用的 vector 数据类型。向量的大小（括号中表示）表示嵌入的维度数量。这里我们使用 512，但请根据你的嵌入模型生成的维度数量进行调整。

关于向量列的更多详情，包括如何生成嵌入和存储它们，请参见 Vector columns。

🌐 For more details on vector columns, including how to generate embeddings and store them, see Vector columns.

相似度指标 #

🌐 Similarity metric

pgvector支持3种操作符来计算嵌入向量之间的距离：

这些运算符可以直接在你的 SQL 查询中使用，用来检索与用户搜索查询最相似的记录。选择合适的运算符取决于你的需求。如果你的向量已经归一化，内积（也叫点积）通常是最快的。

🌐 These operators are used directly in your SQL query to retrieve records that are most similar to the user's search query. Choosing the right operator depends on your needs. Inner product (also known as dot product) tends to be the fastest if your vectors are normalized.

在 Postgres 中执行语义搜索最简单的方法是创建一个函数：

🌐 The easiest way to perform semantic search in Postgres is by creating a function:

1
-- Match documents using cosine distance (<=>)
2
create or replace function match_documents (
3
  query_embedding extensions.vector(512),
4
  match_threshold float,
5
  match_count int
6
)
7
returns setof documents
8
language sql
9
as $$
10
  select *
11
  from documents
12
  where documents.embedding <=> query_embedding < 1 - match_threshold
13
  order by documents.embedding <=> query_embedding asc
14
  limit least(match_count, 200);
15
$$;

这里我们创建一个函数 match_documents，它接受三个参数：

🌐 Here we create a function match_documents that accepts three parameters:

1. query_embedding：为用户的搜索查询生成的一次性嵌入。在这里我们设置大小为512，但可以根据你的嵌入模型生成的维度数量进行调整。
2. match_threshold：嵌入之间的最小相似度。这个值在 1 到 -1 之间，1 表示最相似，-1 表示最不相似。
3. match_count：要返回的最大结果数量。请注意，如果 match_threshold 只生成了一个小的候选列表，查询返回的结果可能会少于这个数量。为了避免意外过载你的数据库，限制最多返回 200 条记录。

在这个例子中，我们返回一个 setof documents，并在整个查询中引用 documents。根据你的应用使用相关的表进行调整。

🌐 In this example, we return a setof documents and refer to documents throughout the query. Adjust this to use the relevant tables in your application.

你会注意到我们在查询中使用了余弦距离（<=>）运算符。当你不确定你的嵌入是否已经归一化时，余弦距离是一个安全的默认选择。如果你确定它们已经归一化（例如，你的嵌入是从 OpenAI 返回的），你可以使用负内积（<#>）来获得更好的性能：

🌐 You'll notice we are using the cosine distance (<=>) operator in our query. Cosine distance is a safe default when you don't know whether or not your embeddings are normalized. If you know for a fact that they are normalized (for example, your embedding is returned from OpenAI), you can use negative inner product (<#>) for better performance:

1
-- Match documents using negative inner product (<#>)
2
create or replace function match_documents (
3
  query_embedding extensions.vector(512),
4
  match_threshold float,
5
  match_count int
6
)
7
returns setof documents
8
language sql
9
as $$
10
  select *
11
  from documents
12
  where documents.embedding <#> query_embedding < -match_threshold
13
  order by documents.embedding <#> query_embedding asc
14
  limit least(match_count, 200);
15
$$;

注意，由于 <#> 是负的，我们在 where 条款中相应地取 match_threshold 的负值。有关不同操作符的更多信息，请参见 pgvector 文档。

🌐 Note that since <#> is negative, we negate match_threshold accordingly in the where clause. For more information on the different operators, see the pgvector docs.

从你的应用调用 #

🌐 Calling from your application

最后你可以从你的应用中执行这个函数。如果你使用的是像 supabase-js 这样的 Supabase 客户端库，你可以使用 rpc() 方法来调用它：

🌐 Finally you can execute this function from your application. If you are using a Supabase client library such as supabase-js, you can invoke it using the rpc() method:

1
const { data: documents } = await supabase.rpc('match_documents', {
2
  query_embedding: embedding, // pass the query embedding
3
  match_threshold: 0.78, // choose an appropriate threshold for your data
4
  match_count: 10, // choose the number of matches
5
})

你也可以直接从 SQL 调用这个方法：

🌐 You can also call this method directly from SQL:

1
select *
2
from match_documents(
3
  '[...]'::extensions.vector(512), -- pass the query embedding
4
  0.78, -- choose an appropriate threshold for your data
5
  10 -- choose the number of matches
6
);

在这种情况下，你很可能会使用 Postgres 客户端库，从你的应用直接连接到数据库。最好在执行查询之前对参数进行参数化。

🌐 In this scenario, you'll likely use a Postgres client library to establish a direct connection from your application to the database. It's best practice to parameterize your arguments before executing the query.

通过元数据过滤向量搜索 #

🌐 Filtering vector search by metadata

在实际应用中，你通常希望把相似性搜索和另一列的过滤条件结合起来，比如只匹配某个类别的文档、特定用户拥有的文档，或者在 jsonb 列中具有匹配元数据的文档。推荐的做法是把过滤条件放到 SQL 函数里，这样规划器就能把它和向量谓词结合起来。在 PostgREST 中，rpc() 应用后再链式调用 .eq() 会作为函数结果的外部过滤执行，也就是在函数已经完成相似性排名和 limit 之后——所以向量规划器无法利用它，而选择性过滤可能导致返回的行数少于 match_count。

🌐 In real applications you usually want to combine the similarity search with a filter on another column, for example only matching documents in a given category, owned by a specific user, or carrying matching metadata in a jsonb column. The recommended pattern is to push the filter into the SQL function so the planner can combine it with the vector predicate. Chaining .eq() after rpc() is applied by PostgREST as an outer filter on the function's result, after the function has already executed its similarity ranking and limit — so the vector planner can't use it, and selective filters can leave you with fewer than match_count rows.

假设你已经向 documents 添加了一个 category text 列，你可以用一个类型化的过滤参数扩展 match_documents 的余弦变体：

🌐 Assuming you've added a category text column to documents, you can extend the cosine variant of match_documents with a typed filter parameter:

1
-- Match documents in a given category using cosine distance (<=>)
2
create or replace function match_documents (
3
  query_embedding extensions.vector(512),
4
  match_threshold float,
5
  match_count int,
6
  filter_category text
7
)
8
returns setof documents
9
language sql
10
as $$
11
  select *
12
  from documents
13
  where documents.category = filter_category
14
    and documents.embedding <=> query_embedding < 1 - match_threshold
15
  order by documents.embedding <=> query_embedding asc
16
  limit least(match_count, 200);
17
$$;

如果你把辅助数据存储在 jsonb metadata 列而不是专用列，同样的模式也可以用 @> 包含操作符来实现：

🌐 If you store side data in a jsonb metadata column instead of dedicated columns, the same pattern works with the @> containment operator:

1
where documents.metadata @> filter_metadata
2
  and documents.embedding <=> query_embedding < 1 - match_threshold

通过传入额外参数从你的应用调用过滤后的函数：

🌐 Call the filtered function from your application by passing the extra parameter:

1
const { data: documents } = await supabase.rpc('match_documents', {
2
  query_embedding: embedding,
3
  match_threshold: 0.78,
4
  match_count: 10,
5
  filter_category: 'blog',
6
})

下一步 #

🌐 Next steps

随着你的数据库规模扩大，你需要在向量列上建立索引，以保持快速的查询速度。查看 向量索引 了解各种索引类型及其工作原理的详细指南。

🌐 As your database scales, you will need an index on your vector columns to maintain fast query speeds. See Vector indexes for an in-depth guide on the different types of indexes and how they work.

对于更大的数据集，选择和调整合适的索引对于保持快速且准确的语义搜索至关重要。

🌐 For larger datasets, choosing and tuning the right index is critical for maintaining fast and accurate semantic search.

pgvector 索引调优 #

🌐 pgvector index tuning

在处理大规模嵌入数据集（10万行以上）时，索引的选择和调优会显著影响查询的延迟和准确性。

🌐 When working with embedding datasets at scale (100k+ rows), index selection and tuning can significantly impact query latency and accuracy.

Supabase 使用带有 pgvector 扩展的 Postgres，它支持两种主要的向量相似度搜索索引类型：IVFFlat 和 HNSW。

🌐 Supabase uses Postgres with the pgvector extension, which supports two primary index types for vector similarity search: IVFFlat and HNSW.

IVF平面索引 #

🌐 IVFFlat index

最佳适用对象：

• 大型数据集（10万到1000万行）
• 快速近似搜索
• 降低内存使用

1
create index on documents
2
using ivfflat (embedding vector_cosine_ops)
3
with (lists = 100);

HNSW 索引 #

🌐 HNSW index

最佳适用对象：

• 高精度要求
• 以读为主的工作负载
• 低延迟语义搜索
• 在召回比内存使用更重要的情况下

1
create index on documents
2
using hnsw (embedding vector_cosine_ops);

另请参阅 #

🌐 See also

• 嵌入概念
• 向量列
• 向量索引
• 混合搜索
• 关键词搜索