向量列
Supabase 提供了多种在 Postgres 中存储和查询向量的方法。本指南中包含的 SQL 对所有编程语言的客户端都适用。如果你是 Python 用户,阅读 Learn 部分后,可以查看你的 Python 客户端选项 。
🌐 Supabase offers a number of different ways to store and query vectors within Postgres. The SQL included in this guide is applicable for clients in all programming languages. If you are a Python user, see your Python client options after reading the Learn section.
Supabase 中的向量是通过 pgvector 启用的,这是一个用于在 Postgres 中存储和查询向量的 Postgres 扩展。它可以用来存储 嵌入。
🌐 Vectors in Supabase are enabled via pgvector, a Postgres extension for storing and querying vectors in Postgres. It can be used to store embeddings.
用法 #
🌐 Usage
启用扩展 #
🌐 Enable the extension
- 在仪表板中转到数据库页面。
- 点击侧边栏的 扩展。
- 搜索“vector”并启用扩展。
创建一个用于存储向量的表 #
🌐 Create a table to store vectors
启用 vector 扩展后,你将可以访问一个名为 vector 的新数据类型。向量的大小(括号中表示)代表该向量存储的维度数量。
🌐 After enabling the vector extension, you will get access to a new data type called vector. The size of the vector (indicated in parentheses) represents the number of dimensions stored in that vector.
1create table documents (2 id serial primary key,3 title text not null,4 body text not null,5 embedding extensions.vector(384)6);在上面的 SQL 代码片段中,我们创建了一个 documents 表,并包含一个 embedding 列。这是一个标准的 Postgres 列,所以你可以随便命名。embedding 列使用了 vector 数据类型,维度为 384。将这个数字改成与你的嵌入模型生成的维度相匹配。例如,如果你正在使用开源的 gte-small 模型生成嵌入,就把它设置为 384。
🌐 In the SQL snippet above, we create a documents table with an embedding column. This is a standard Postgres column, so you can name it anything you like. The embedding column uses the vector data type with 384 dimensions. Change this number to match the dimensions your embedding model produces. For example, if you're generating embeddings using the open source gte-small model, set this to 384.
一般来说,维度更少的嵌入表现最好。看看我们关于 pgvector 中更少维度的分析吧(/blog/fewer-dimensions-are-better-pgvector)。
🌐 In general, embeddings with fewer dimensions perform best. See our analysis on fewer dimensions in pgvector.
存储向量/嵌入 #
🌐 Storing a vector / embedding
在这个例子中,我们将使用 Transformers.js 生成一个向量,然后使用 Supabase JavaScript 客户端将其存储到数据库中。
🌐 In this example we'll generate a vector using Transformers.js, then store it in the database using the Supabase JavaScript client.
1import { pipeline } from '@huggingface/transformers'23const generateEmbedding = await pipeline('feature-extraction', 'Supabase/gte-small')45const title = 'First post!'6const body = 'Hello world!'78// Generate a vector using Transformers.js9const output = await generateEmbedding(body, {10 pooling: 'mean',11 normalize: true,12})1314// Extract the embedding output15const embedding = Array.from(output.data)1617// Store the vector in Postgres18const { data, error } = await supabase.from('documents').insert({19 title,20 body,21 embedding,22})这个例子使用了 JavaScript 的 Supabase 客户端,但你可以修改它以适用于任何 支持的语言库。
🌐 This example uses the JavaScript Supabase client, but you can modify it to work with any supported language library.
查询向量/嵌入 #
🌐 Querying a vector / embedding
相似度搜索是向量最常见的使用场景。pgvector 支持 3 个用于计算距离的新操作符:
🌐 Similarity search is the most common use case for vectors. pgvector supports 3 new operators for computing distance:
| 操作符 | 描述 |
|---|---|
<-> | 欧几里得距离 |
<#> | 负内积 |
<=> | 余弦距离 |
选择合适的运算符取决于你的需求。如果你的向量已经归一化,点积通常是最快的。想了解更多关于嵌入是如何工作的以及它们之间的关系,请参见 什么是嵌入?。
🌐 Choosing the right operator depends on your needs. Dot product tends to be the fastest if your vectors are normalized. For more information on how embeddings work and how they relate to each other, see What are Embeddings?.
像 supabase-js 这样的 Supabase 客户端库通过 PostgREST 连接到你的 Postgres 实例。PostgREST 目前不支持 pgvector 相似度操作符,所以我们需要把查询封装在一个 Postgres 函数中,然后通过 rpc() 方法调用它:
🌐 Supabase client libraries like supabase-js connect to your Postgres instance via PostgREST. PostgREST does not currently support pgvector similarity operators, so we'll need to wrap our query in a Postgres function and call it via the rpc() method:
1create or replace function match_documents (2 query_embedding extensions.vector(384),3 match_threshold float,4 match_count int5)6returns table (7 id bigint,8 title text,9 body text,10 similarity float11)12language sql stable13as $$14 select15 documents.id,16 documents.title,17 documents.body,18 1 - (documents.embedding <=> query_embedding) as similarity19 from documents20 where 1 - (documents.embedding <=> query_embedding) > match_threshold21 order by (documents.embedding <=> query_embedding) asc22 limit match_count;23$$;这个函数接收一个 query_embedding 参数,并将其与 documents 表中的所有其他嵌入进行比较。每次比较都会返回一个相似度分数。如果相似度大于 match_threshold 参数,它将被返回。返回的行数由 match_count 参数限制。
🌐 This function takes a query_embedding argument and compares it to all other embeddings in the documents table. Each comparison returns a similarity score. If the similarity is greater than the match_threshold argument, it is returned. The number of rows returned is limited by the match_count argument.
可以随意修改这个方法以适应你应用的需求。match_threshold 确保只有与 query_embedding 具有最低相似度的文档才会被返回。没有这个,你可能会返回主观上不匹配的文档。这个值会因应用而异——你需要自己测试来确定对你的应用来说合适的阈值。
🌐 Feel free to modify this method to fit the needs of your application. The match_threshold ensures that only documents that have a minimum similarity to the query_embedding are returned. Without this, you may end up returning documents that subjectively don't match. This value will vary for each application - you will need to perform your own testing to determine the threshold that makes sense for your app.
如果你为向量列建立索引,确保 order by 是直接按距离函数排序的(而不是按计算出的 similarity 列排序,否则可能会导致索引被忽略,性能下降)。
🌐 If you index your vector column, ensure that the order by sorts by the distance function directly (rather than sorting by the calculated similarity column, which may lead to the index being ignored and poor performance).
要从你的客户端库执行函数,调用 rpc() 并传入你的 Postgres 函数名:
🌐 To execute the function from your client library, call rpc() with the name of your Postgres function:
1const { data: documents } = await supabaseClient.rpc('match_documents', {2 query_embedding: embedding, // Pass the embedding you want to compare3 match_threshold: 0.78, // Choose an appropriate threshold for your data4 match_count: 10, // Choose the number of matches5})在这个例子中,embedding 会是你想要与预先生成的嵌入文档表进行比较的另一个嵌入。例如,如果你在构建一个搜索引擎,每次用户提交查询时,你首先会在搜索查询本身上生成一个嵌入,然后将其传入上面的 rpc() 函数进行匹配。
🌐 In this example embedding would be another embedding you wish to compare against your table of pre-generated embedding documents. For example if you were building a search engine, every time the user submits their query you would first generate an embedding on the search query itself, then pass it into the above rpc() function to match.
要从 JS 客户端按另一列过滤向量搜索,可以在上面的函数中添加一个额外的参数和 where 子句。请参见 通过元数据过滤向量搜索 获取示例。
🌐 To filter your vector search by another column from the JS client, extend the function above with an extra parameter and where clause. See Filtering vector search by metadata for a worked example.
在计算距离时,一定要使用来自同一个嵌入模型的嵌入。比较两个不同模型的嵌入是没有任何意义的。
🌐 Be sure to use embeddings produced from the same embedding model when calculating distance. Comparing embeddings from two different models will produce no meaningful result.
向量和嵌入不仅仅可以用于搜索。想了解更多关于嵌入的信息,请访问 什么是嵌入?。
🌐 Vectors and embeddings can be used for much more than search. Learn more about embeddings at What are Embeddings?.
索引 #
🌐 Indexes
一旦你的向量表开始增大,你可能会想添加一个索引来加快查询速度。查看 向量索引 了解向量索引的工作原理以及如何创建它们。
🌐 Once your vector table starts to grow, you will likely want to add an index to speed up queries. See Vector indexes to learn how vector indexes work and how to create them.