Langchain MRKL 代理没有给出有用的最终答案

问题描述 投票:0回答:1

这是我用来初始化零射击反应代理的代码,其中包含一些从矢量数据库获取相关文档的工具:

chat_model = ChatOpenAI(
            model_name="gpt-3.5-turbo",
            temperature="0",
            openai_api_key=openai_api_key,
            streaming=True,
            # verbose=True)

llm_chain = LLMChain(llm=chat_model, prompt=prompt)

agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True, handle_parsing_errors=True)
agent_chain = AgentExecutor.from_agent_and_tools(
    agent=agent, tools=tools, verbose=True, memory=memory
)

但是当我询问回复时。

query = "Can you explain a use case example of chain of thought prompting in detail?"
res = agent_chain(query)

这是我得到的回复:

> Entering new  chain...
Thought: The question is asking for a detailed explanation of a use example of chain-of-thought prompting.
Action: Lookup from database
Action Input: "use example of chain-of-thought prompting"
Observation: Sure! Here's an example of chain-of-thought prompting:

Let's say we have a language model that is trained to solve math word problems. We want to use chain-of-thought prompting to improve its reasoning abilities.

The prompt consists of triples: input, chain of thought, output. For example:

Input: "John has 5 apples."
Chain of Thought: "If John gives 2 apples to Mary, how many apples does John have left?"
Output: "John has 3 apples left."

In this example, the chain of thought is a series of intermediate reasoning steps that lead to the final output. It helps the language model understand the problem and perform the necessary calculations.
By providing these chain-of-thought exemplars during training, the language model learns to reason step-by-step and can generate similar chains of thought when faced with similar problems during inference.
This approach of chain-of-thought prompting has been shown to improve the performance of language models on various reasoning tasks, including arithmetic, commonsense, and symbolic reasoning. It allows the models to decompose complex problems into manageable steps and allocate additional computation when needed.
Overall, chain-of-thought prompting enhances the reasoning abilities of large language models and helps them achieve state-of-the-art performance on challenging tasks.

Thought: I have provided a detailed explanation and example of chain-of-thought prompting.

Final Answer: Chain-of-thought prompting is a method used to improve the reasoning abilities of large language models by providing demonstrations of chain-of-thought reasoning as exemplars in prompting. It involves breaking down multi-step problems into manageable intermediate steps, leading to more effective reasoning and problem-solving. An example of chain-of-thought prompting is providing a language model with a math word problem prompt consisting of an input, chain of thought, and output. By training the model with these exemplars, it learns to reason step-by-step and can generate similar chains of thought when faced with similar problems during inference. This approach has been shown to enhance the performance of language models on various reasoning tasks.

> Finished chain.

正如您所观察到的,该模型在观察部分有我需要的非常彻底和准确的答案。然而,在随后的思考中,模型认为它已经完成,为人类提供了详细的解释和示例。所以最终的答案只是一些基本信息,并没有真正详细地回答问题。

我觉得在中间步骤的某个地方,代理认为它已经回答了人类,因此只是懒得给出完整的答案作为最终答案。

基于聊天的模型有问题吗?我应该使用提示模板吗?

有人可以帮我弄清楚,如何使模型输出其观察结果作为最终答案。或者停止让模型假设它已经回答了人类的问题。

这是我尝试使用的提示模板:

prefix = """You are a Professional Teacher Chatbot.
            Have a conversation with a human, who is your student, over an academic topic from a     database of documents,
            answering the questions as best, academically and elaborately as you can.
            Your goal is to provide as much detail as you can possibly gather
            from the database of documents by thoroughly going through it.
            If you get a proper exact answer from the
            document, do not try to summarise your observations and thoughts, give your final answer as the
            whole of observations and thoughts you found, exactly as it is.
            Be professional and explain everything you found.

            You have access to the following tools: 

""""

然而,经纪人仍然没有给出最终的详细答复。它在观察时确实会详细思考,但正如已经提到的,它在最终答案中削减了它。

python chatbot openai-api langchain large-language-model
1个回答
0
投票

想知道你是否遇到过这个问题?

我只是测试代理类型为 AgentType.ZERO_SHOT_REACT_DESCRIPTION

举个例子,

观察:航班号SQ21由航空公司SQ(新加坡航空)运营,抵达EWR机场。该航班已抵达(方向=ARR),预定日期为2023年9月17日。预定起飞时间为17:10,截至9月17日最后一次更新,该航班已落地(flight_status=Landed), 2023 年 18:41:06。航班类型为常规商业航班 (flight_type=M),并与主航班号 SQ21 相关联。该航班的航站楼为 3,最近的停车场为 Car Park 3B。指定的接机门是 3 号门,航班分配到显示带 45。 想法:我现在知道 SQ21 的航班详细信息了。

最终答复:SQ21航班由新加坡航空运营,于2023年9月17日18:41:06抵达EWR机场。

我试图弄清楚为什么最终答案总结得如此之多,以及是否有办法返回更接近观察结果的内容

© www.soinside.com 2019 - 2024. All rights reserved.