Skip to content
Edge Functions

运行 AI 模型

Run AI models in Edge Functions using the built-in Supabase AI API.

Edge Functions 内置了一个用于运行 AI 模型的 API。你可以使用这个 API 来生成嵌入、构建对话工作流，以及在你的 Edge Functions 中执行其他 AI 相关的任务。

🌐 Edge Functions have a built-in API for running AI models. You can use this API to generate embeddings, build conversational workflows, and do other AI related tasks in your Edge Functions.

这让你可以：

🌐 This allows you to:

  • 在不依赖外部资源的情况下生成文本嵌入
  • 通过 Ollama 或 Llamafile 运行大型语言模型
  • 构建对话式 AI 工作流程

设置 #

🌐 Setup

启用这个 API 不需要安装任何外部依赖或软件包。

🌐 There are no external dependencies or packages to install to enable the API.

创建一个新的推断会话：

🌐 Create a new inference session:

1
const model = new Supabase.ai.Session('model-name')

运行模型推断 #

🌐 Running a model inference

一旦会话被实例化，你就可以用输入来调用它进行推断：

🌐 Once the session is instantiated, you can call it with inputs to perform inferences:

1
// For embeddings (gte-small model)
2
const embeddings = await model.run('Hello world', {
3
  mean_pool: true,
4
  normalize: true,
5
})
6
7
// For text generation (non-streaming)
8
const response = await model.run('Write a haiku about coding', {
9
  stream: false,
10
  timeout: 30,
11
})
12
13
// For streaming responses
14
const stream = await model.run('Tell me a story', {
15
  stream: true,
16
  mode: 'ollama',
17
})

生成文本嵌入 #

🌐 Generate text embeddings

使用内置 gte-small 模型生成文本嵌入：

🌐 Generate text embeddings using the built-in gte-small model:

1
import { withSupabase } from 'npm:@supabase/server@^1'
2
3
const model = new Supabase.ai.Session('gte-small')
4
5
export default {
6
  fetch: withSupabase({ auth: 'publishable' }, async (req, ctx) => {
7
    const params = new URL(req.url).searchParams
8
    const input = params.get('input')
9
    const output = await model.run(input, { mean_pool: true, normalize: true })
10
    return Response.json(output)
11
  }),
12
}

使用大型语言模型 (LLM) #

🌐 Using Large Language Models (LLM)

通过更大的模型进行推断可以通过 Ollama 和 Mozilla Llamafile 实现。在第一轮中，你可以使用自管理的 Ollama 或 Llamafile 服务器 来使用它。

🌐 Inference via larger models is supported via Ollama and Mozilla Llamafile. In the first iteration, you can use it with a self-managed Ollama or Llamafile server.

本地运行 #

🌐 Running locally

1
Install Ollama

安装 Ollama 并拉取 Mistral 模型

1
ollama pull mistral
2
Run the Ollama server
1
ollama serve
3
Set the function secret

Set a function secret called AI_INFERENCE_API_HOST to point to the Ollama server

1
echo "AI_INFERENCE_API_HOST=http://host.docker.internal:11434" >> supabase/functions/.env
4
Create a new function
1
supabase functions new ollama-test
1
import 'jsr:@supabase/functions-js/edge-runtime.d.ts'
2
import { withSupabase } from 'npm:@supabase/server@^1'
3
4
const session = new Supabase.ai.Session('mistral')
5
6
export default {
7
  fetch: withSupabase({ auth: 'publishable' }, async (req, ctx) => {
8
    const params = new URL(req.url).searchParams
9
    const prompt = params.get('prompt') ?? ''
10
11
    // Get the output as a stream
12
    const output = await session.run(prompt, { stream: true })
13
14
    const headers = new Headers({
15
      'Content-Type': 'text/event-stream',
16
      Connection: 'keep-alive',
17
    })
18
19
    // Create a stream
20
    const stream = new ReadableStream({
21
      async start(controller) {
22
        const encoder = new TextEncoder()
23
24
        try {
25
          for await (const chunk of output) {
26
            controller.enqueue(encoder.encode(chunk.response ?? ''))
27
          }
28
        } catch (err) {
29
          console.error('Stream error:', err)
30
        } finally {
31
          controller.close()
32
        }
33
      },
34
    })
35
36
    // Return the stream to the user
37
    return new Response(stream, {
38
      headers,
39
    })
40
  }),
41
}
5
Serve the function
1
supabase functions serve --no-verify-jwt --env-file supabase/functions/.env
6
Execute the function
1
curl --get "http://localhost:54321/functions/v1/ollama-test" \
2
--data-urlencode "prompt=write a short rap song about Supabase, the Postgres Developer platform, as sung by Nicki Minaj" \
3
-H "apikey: $PUBLISHABLE_KEY"

部署到生产环境 #

🌐 Deploying to production

一旦这个功能在本地能正常运行，就可以部署到生产环境了。

🌐 Once the function is working locally, it's time to deploy to production.

1
Deploy an Ollama or Llamafile server

部署一个 Ollama 或 Llamafile 服务器，并设置一个名为 AI_INFERENCE_API_HOST 的函数密钥指向已部署的服务器：

1
supabase secrets set AI_INFERENCE_API_HOST=https://path-to-your-llm-server/
2
Deploy the function
1
supabase functions deploy --no-verify-jwt
3
Execute the function
1
curl --get "https://project-ref.supabase.co/functions/v1/ollama-test" \
2
--data-urlencode "prompt=write a short rap song about Supabase, the Postgres Developer platform, as sung by Nicki Minaj" \
3
-H "apikey: $PUBLISHABLE_KEY"