Worklog for task "Add image and photo upload and recognition features"

Overall, everything is done. But a situation has arisen again which highlights that even in simple tasks with LLMs, it is impossible to completely rule out the probability of out-of-nowhere errors. In this case, the LLM failed at the very simplest point—it stupidly failed to reproduce the path to the file that needed to be recognized. Moreover, it didn't just make a mistake in a couple of characters—it lost a whole chunk of the path, and it did so twice. That is, it tried once to send the path to the tool, got an error, tried to fix it, and sent the wrong path again, even though it had this path in its context just two messages above. All of this pushes me once more to work on an old idea of mine—implementing a variable system for LLMs. But even so, ultimately the variables, even if shorter, are still sent by the LLM, which means this still doesn't exclude the probability of error, though it reduces that probability since the length of the variable will be much smaller than the final value of the variable.

But here I want to highlight another observation... Here was the original path:

cmr1pztsm0003r90q4juuycrs/chat/1782890662561-17828906555453720229472878214886.jpg

And here is the path the LLM tried to send.

cmr1pztsm0003r90q4juuycrs/chat/17828906555453720229472878214886.jpg

What catches the eye here? It threw out 1782890662561- from it, that is, the timestamp. Probably, during the training process, it was nudged toward ignoring timestamps, which happens quite often, and maybe that manifested itself here. One could assume that "This timestamp couldn't have existed before, so it could never have known about it," but it is also unlikely that it had ever seen "17828906555453720229472878214886" anywhere before, just like cmr1pztsm0003r90q4juuycrs, yet it reproduced it. Besides, part of the value in 17828906555453720229472878214886 is 17828906, which is also a timestamp. I think here specifically an attention drop effect occurred, literally point-blank blindness.