文档 / 保证可用性

保证可用性

通常没有必要实现这一点,因为它会增加应用的复杂性。Langfuse 提示词管理由于有多层缓存而具有高可用性,并且我们密切监控其性能(状态页)。但是,如果你需要 100% 的可用性,可以使用以下选项。

Langfuse API 具有很高的正常运行时间,并且提示词在 SDK 中本地缓存,以防止网络问题影响你的应用。

但是,在以下情况 get_prompt()/getPrompt() 会抛出异常:

没有可用的本地(新鲜或过期)缓存提示词 -> 应用新实例首次获取提示词
并且网络请求失败 -> 网络或 Langfuse API 问题(重试之后)

为保证 100% 可用性,有两种选项:

在应用启动时预取提示词,如果提示词不可用则退出应用。
提供一个 fallback 提示词,在这些情况下使用。

选项 1:预取提示词

在应用启动时预取提示词,如果提示词不可用则退出应用。

python

from flask import Flask, jsonify
from langfuse import Langfuse

# Initialize the Flask app and Langfuse client
app = Flask(__name__)
langfuse = Langfuse()

def fetch_prompts_on_startup():
    try:
        # Fetch and cache the production version of the prompt
        langfuse.get_prompt("movie-critic")
    except Exception as e:
        print(f"Failed to fetch prompt on startup: {e}")
        sys.exit(1)  # Exit the application if the prompt is not available

# Call the function during application startup
fetch_prompts_on_startup()

@app.route('/get-movie-prompt/<movie>', methods=['GET'])
def get_movie_prompt(movie):
    prompt = langfuse.get_prompt("movie-critic")
    compiled_prompt = prompt.compile(criticlevel="expert", movie=movie)
    return jsonify({"prompt": compiled_prompt})

if __name__ == '__main__':
    app.run(debug=True)

import express from "express";
import { LangfuseClient } from "@langfuse/client";

// Initialize the Express app and Langfuse client
const app = express();
const langfuse = new LangfuseClient();

async function fetchPromptsOnStartup() {
  try {
    // Fetch and cache the production version of the prompt
    await langfuse.prompt.get("movie-critic");
  } catch (error) {
    console.error("Failed to fetch prompt on startup:", error);
    process.exit(1); // Exit the application if the prompt is not available
  }
}

// Call the function during application startup
fetchPromptsOnStartup();

app.get("/get-movie-prompt/:movie", async (req, res) => {
  const movie = req.params.movie;
  const prompt = await langfuse.prompt.get("movie-critic");
  const compiledPrompt = prompt.compile({ criticlevel: "expert", movie });
  res.json({ prompt: compiledPrompt });
});

app.listen(3000, () => {
  console.log("Server is running on port 3000");
});

选项 2:降级(Fallback)

提供一个降级提示词,在这些情况下使用:

python

from langfuse import Langfuse
langfuse = Langfuse()

# Get `text` prompt with fallback
prompt = langfuse.get_prompt(
  "movie-critic",
  fallback="Do you like {{movie}}?"
)

# Get `chat` prompt with fallback
chat_prompt = langfuse.get_prompt(
  "movie-critic-chat",
  type="chat",
  fallback=[{"role": "system", "content": "You are an expert on {{movie}}"}]
)

# True if the prompt is a fallback
prompt.is_fallback

import { LangfuseClient } from "@langfuse/client";

const langfuse = new LangfuseClient();

// Get `text` prompt with fallback
const prompt = await langfuse.prompt.get("movie-critic", {
  fallback: "Do you like {{movie}}?",
});

// Get `chat` prompt with fallback
const chatPrompt = await langfuse.prompt.get("movie-critic-chat", {
  type: "chat",
  fallback: [{ role: "system", content: "You are an expert on {{movie}}" }],
});

// True if the prompt is a fallback
prompt.isFallback;