Worklog for task "Add project documentation analysis features to the AI agent"

Here is a separate, extremely interesting case with the following instruction:

Read the files

storage/unpack/cmqv7yb0i00018hzwx2uttxcb/1782515133778-VKG_Ümberpuist hoone.rar/RUS_25053_v03_PP_ГГС-4_Золоудаление/RUS_25053_v03_PP_ГГС-4_Золоудаление/Том I_25053_PP_ГГС-4_AA-AS-AR-EK-TK/25053_PP_ГГС-4_строительство/25053_PP_ГГС-4_строительство/25053_PP_ƒƒ‘-4_áâந⥫ìá⢮/25053_PP_AA/25053_PP_AA-0-01_â¨âã«ì­ë© «¨áâ.pdf

storage/unpack/cmqv7yb0i00018hzwx2uttxcb/1782515133778-VKG_Ümberpuist hoone.rar/RUS_25053_v03_PP_ГГС-4_Золоудаление/RUS_25053_v03_PP_ГГС-4_Золоудаление/Том I_25053_PP_ГГС-4_AA-AS-AR-EK-TK/25053_PP_ГГС-4_строительство/25053_PP_ГГС-4_строительство/25053_PP_ƒƒ‘-4_áâந⥫ìá⢮/25053_PP_AA/25053_PP_AA-3-01_¯®ï᭨⥫쭠ï.pdf

Write the results to the worklog.

What is interesting here first of all? These are the long paths with various mojibake characters. Let me remind you that LLMs do not inherently have variable mechanisms (I am only planning to implement this in my own version). By and large, when generating queries, the LLM must reproduce these paths byte-for-byte when passing them as parameters to tools. And it must be noted that Gemini 2.5 Flash handled this brilliantly.

Next, I have some interesting and unexpected assumptions to make based on the models' reasoning while reading the files. I will quote both queries right away.

[
  {
    "query": {
      "reasoning": "Extracting text from the title page to identify the document.",

      "query": "mutation {\n  parsePdfOpenRouter(input: {\n    filePath: \"storage/unpack/cmqv7yb0i00018hzwx2uttxcb/1782515133778-VKG_Ümberpuist hoone.rar/RUS_25053_v03_PP_ГГС-4_Золоудаление/RUS_25053_v03_PP_ГГС-4_Золоудаление/Том I_25053_PP_ГГС-4_AA-AS-AR-EK-TK/25053_PP_ГГС-4_строительство/25053_PP_ГГС-4_строительство/25053_PP_ƒƒ‘-4_áâந⥫ìá⢮/25053_PP_AA/25053_PP_AA-0-01_â¨âã«ì­ë© «¨áâ.pdf\"\n    pages: [1]\n  }) {\n    pages {\n      text\n    }\n  }\n}",

      "variables": "{}"
    }
  }
]

[
  {
    "query": {
      "reasoning": "Extracting text from the explanatory note. Let's see how many pages are in the document, and read them (if there aren't many). The first request showed 1 page, but it's better to make sure.",

      "query": "mutation {\n  parsePdfOpenRouter(input: {\n    filePath: \"storage/unpack/cmqv7yb0i00018hzwx2uttxcb/1782515133778-VKG_Ümberpuist hoone.rar/RUS_25053_v03_PP_ГГС-4_Золоудаление/RUS_25053_v03_PP_ГГС-4_Золоудаление/Том I_25053_PP_ГГС-4_AA-AS-AR-EK-TK/25053_PP_ГГС-4_строительство/25053_PP_ГГС-4_строительство/25053_PP_ƒƒ‘-4_áâந⥫ìá⢮/25053_PP_AA/25053_PP_AA-3-01_¯®ïá­¨â¥«ì­ ï.pdf\"\n  }) {\n    totalPages\n    pages {\n      pageNumber\n      text\n    }\n  }\n}",

      "variables": "{}"
    }
  }
]

What is interesting here? In general, it followed the logic embedded in the document reading skill, here is a quote:

Document Reading Strategy

It is recommended to read the document superficially first — only the first page with the totalPages query:

mutation {
  parsePdfOpenRouter(input: {
    filePath: "storage/unpack/tender-123/document.pdf"
    pages: [1]
  }) {
    totalPages
    pages {
      text
    }
  }
}

This allows you to:

1. Find out the total number of pages in the document
2. Understand the content and relevance of the document
3. Decide whether you need to read the entire document

If the document is important and the full content is required, read all pages (without the pages parameter).

And what did our agent do? It did almost everything right, but made a key mistake — it requested the page count on the first document, but read everything on a completely different document :-) Moreover, in the second query, it even clarified: The first request showed 1 page, but it's better to make sure.

Furthermore, in the first query, it couldn't even know how many pages were actually there, because it only specified to get the first page by passing pages: [1], but it did not include the totalPages parameter in the response body :-) So it received the response without information about the page count.

Yet in the second query, it did the opposite — it specified totalPages (and it got this information), but did not specify pages: [1] :-)

In short, this is a very interesting and somewhat comical example of how an agent can logically deviate from the instructions given to it, but in certain cases, considering the requirements for task execution reliability, the result may turn out to be far from comical.