LangChain WebResearchRetriever 超出上下文长度

Question

我目前正在使用LangChain的

WebResearchRetriever

类开发代理聊天机器人。然而，

WebResearchRetriever

返回的上下文总是超过OpenAI的

gpt-3.5-turbo

模型允许的最大令牌长度，即4097个令牌。有什么方法可以阻止该代理使用大于模型上下文长度的上下文长度？以下是我的代理代码：

# Initialize properties
vectorstore = Chroma(embedding_function=OpenAIEmbeddings(), persist_directory="./chroma_db_oai")
llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.3)
memory = AgentTokenBufferMemory(memory_key="chat_history", llm=llm, return_messages=True, max_token_limit=2048)
llm.max_tokens = 2048

# Initialize Google Search Retriever
search = GoogleSearchAPIWrapper()

web_research_retriever = WebResearchRetriever.from_llm(
    vectorstore=vectorstore,
    llm=llm, 
    search=search, 
    num_search_results=1,
    text_splitter=RecursiveCharacterTextSplitter.from_tiktoken_encoder(chunk_size=512, chunk_overlap=0, model_name="gpt-3.5-turbo"),
    url_database=["a-hospital.com", "120ask.com", "dxy.cn", "zhihu.com", "baidu.com", "ewsos.com", "wikipedia.org"]
)

# Initialize the agent tools
tool = create_retriever_tool(
    web_research_retriever, 
    "medical_web_retriever",
    "Researches and returns information in the internet about any medical-related topics."
)
tools = [tool]

# Initialize the agent
# TODO: prompt engineer the system message so that it selects the context when it needs; it now does not select any concept
system_message = SystemMessage(
        content=(
            "Do your best to answer the questions. "
            "Feel free to use any tools available to look up for professional medical information."
            "If you encounter a new medical concept that has not appeared in the conversation before, e.g., a medicine or a symptom, you should carry out a web research of it."
            "If a medical concept has been asked before, you should check the pre-researched information before deciding to research about any aspects of the concept even more."
        )
)
prompt = OpenAIFunctionsAgent.create_prompt(
        system_message=system_message,
        extra_prompt_messages=[MessagesPlaceholder(variable_name="chat_history")]
    )
agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=prompt)

agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory, verbose=True, return_intermediate_steps=True)
result = agent_executor({"input": "什么是连花清瘟胶囊？"})
print(result)

Answer 1

有两件重要的事情将帮助您控制上下文长度。

1. 确保您的提示定义明确，您可以在系统消息中再添加一点，即“您的回复应该简洁，不应超过两到三句话。”

2. 将工具描述定义为

   description="useful for when you need to research short answers for any medical-related topics"