当我向代理请求特定的 JSON 结构时，无法解析 langchain 中的 LLM 输出

Question

我正在尝试将 google 搜索 api 集成到我构建的代理中，其中代理需要为给定段落创建特定的 json 结构。给定的段落将是由工程师完成的任务。因此，代理应该能够识别工程师在特定日期提出的任务和问题。我的代理显示在以下代码块中

agent = initialize_agent(tools=[google_search_tool],
                                 agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,
                                 llm=llm,
                                 memory=conversational_memory,
                                 max_iterations=5,
                                 early_stopping_method='generate',
                                 verbose=True,
                                 agent_kwargs={
                                     "system_message": CUSTOM_TIMESHEET_SYSTEM_PROMPT.strip(),
                                 },
                                 handle_parsing_errors=False
                                 )

一旦我将消息传递给代理，代理就会识别所有任务，但会使用构造的 JSON 抛出异常。错误是以下块。

ile "C:\Users\ASUS\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\agents\conversational_chat\output_parser.py", line 50, in parse
    raise OutputParserException(f"Could not parse LLM output: {text}") from e
langchain.schema.output_parser.OutputParserException: Could not parse LLM output: ```json
{
    "tasks": [
        {
            "id": 1,
            "task": "Completed the implementation of the user authentication feature",
            "duration": "8 hrs"
        },
        {
            "id": 2,
            "task": "Wrote tests for the user authentication feature",
            "duration": null
        },
        {
            "id": 3,
            "task": "Plan to start working on the user profile feature",
            "duration": null
        },
        {
            "id": 4,
            "task": "Creating the database schema for the user profile feature",
            "duration": "4 hrs"
        },
        {
            "id": 5,
            "task": "Start with the backend implementation for the user profile feature",
            "duration": "4 hrs"
        },
        {
            "id": 6,
            "task": "Complete the backend implementation for the user profile feature",
            "duration": "8 hrs"
        }
    ],
    "problems": []
}

这是我传递给我的代理的系统消息。

CUSTOM_TIMESHEET_SYSTEM_PROMPT = """
Assistant is a large language model trained by OpenAI.Assistant is designed to be able to assist with you to breakdown tasks and the issues,
that were encountered through out the day. Assistant will break down the tasks and problems that were encountered by you.
Overall, Assistant is a powerful system that can help you to break down tasks and identify the problems that were encountered through given content.

TOOLS
------
Assistant can ask the user to use tools to look up information that may be helpful in answering the users original question. The tools the human can use are:

> Google Search: Search Google for most recent data, latest data, any question you don't know the answer such as current date, time, weather, president, etc
 

RESPONSE FORMAT INSTRUCTIONS
----------------------------
Please provide a structured list or breakdown of the individual work actions/tasks and problems faced mentioned within the paragraph.
Ensure each task is clearly separated and identified with the time spent on each task. Provide the identified tasks as a JSON array. 
Ensure that the tasks are clearly separated. Make sure to extract the time from hours. 
If the time is provided through a different metric, convert it to hours. One day means 8 hours, and half a day means 4 hours. 
One morning, afternoon, and evening will be equal to 4 hrs. Make sure to add hrs as the metric to the value. 
If the duration is not identified, make sure to keep duration null in the JSON. The identified problems should be identified as another JSON array. 
Id can be an incremental id. recommendedLinks should be solutions for the identified problems from YouTube.
When responding to me, please output a response in this format:

Option 1:
Use this if a problem is identified to recommend links from a tool.
Markdown code snippet formatted in the following schema:
```json
{{
    "action": any, # The action should be the google search
    "action_input": string # The identified problem
}}

Option #2:
Use this if you want to respond directly to the human. MARKDOWN code snippet formatted in the following schema:
```json
{{ 
    "action": any,
    "action_input": string # You should put what you want to return to use here
}}

Make sure to return the constructed JSON as a string
USER'S INPUT
--------------------
Here is the user's input (remember to respond with a markdown code snippet of a json blob with a single action, and NOTHING else): 

我只是希望将提到的内容作为输出返回，没有任何例外。任何想法如何解决这个问题。请注意，我通过了

handle_parsing_errors=True

但这导致了一个循环，最终会导致 gpt 端出现错误。一旦我通过了

handle_parsing_errors="Check your output and make sure it conforms! "

，它只会输出类似“根据您最后的评论，您已经成功完成了用户身份验证功能的实现并为其编写了测试。您正在计划开发用户配置文件功能” ，从创建数据库模式开始，然后是后端实现。您预计在明天下午之前完成后端实现，目前，您预计不会出现任何阻碍。如果出现任何阻碍，您计划在团队的沟通渠道中进行沟通，并更新据此估计时间表。”

Answer 1

LLM 并不总是生成有效的 JSON，并且无论提示技术如何，您都不能强迫它们。因此，您需要在解析之前想出一些方法来清理 JSON。

GoodAI-LTM 库有这样一个方法：sanitize_and_parse_json()。

Answer 2

虽然您确实不能期望法学硕士总是生成可解析的 JSON，但您仍然可以使用一些周转来实现您的目标。以下是其中一些：

1. 简化JSON格式。根据您使用的 LLM，它可能无法进行微调以处理 JSON 响应。您可以询问类似的问题，用

   ~

   分隔字符串，然后从响应中手动创建一个 JSON 对象。如果您可以使用 GPT-4 Turbo，这将不是问题。

2. 尝试更简单的格式，你可以要求LLM输出一个数组，或者任何其他线性数据结构。然后，您可以解析输出并填充 JSON 对象。

3. 您可以对LLM进行多次调用，根据信息的重要性，您可以对值进行分组并进行多次推理调用。这项技术还将为您提供比以前更好的响应。从现在开始，模型不必考虑严格的 JSON 格式。

最后，如果你有相关资源，你可以在你的模式上微调LLM。之后它几乎总是会返回正确的格式。