Task: Add image and photo upload and recognition features

Overall, everything is done. But a situation has arisen again which highlights that even in simple tasks with LLMs, it is impossible to completely rule out the probability of out-of-nowhere errors. In this case, the LLM failed at the very simplest point—it stupidly failed to reproduce the path to the file that needed to be recognized. Moreover, it didn't just make a mistake in a couple of characters—it lost a whole chunk of the path, and it did so twice. That is, it tried once to send the path to the tool, got an error, tried to fix it, and sent the wrong path again, even though it had this path in its context just two messages above. All of this pushes me once more to work on an old idea of mine—implementing a variable system for LLMs. But even so, ultimately the variables, even if shorter, are still sent by the LLM, which means this still doesn't exclude the probability of error, though it reduces that probability since the length of the variable will be much smaller than the final value of the variable.

But here I want to highlight another observation... Here was the original path:

cmr1pztsm0003r90q4juuycrs/chat/1782890662561-17828906555453720229472878214886.jpg

And here is the path the LLM tried to send.

cmr1pztsm0003r90q4juuycrs/chat/17828906555453720229472878214886.jpg

What catches the eye here? It threw out 1782890662561- from it, that is, the timestamp. Probably, during the training process, it was nudged toward ignoring timestamps, which happens quite often, and maybe that manifested itself here. One could assume that "This timestamp couldn't have existed before, so it could never have known about it," but it is also unlikely that it had ever seen "17828906555453720229472878214886" anywhere before, just like cmr1pztsm0003r90q4juuycrs, yet it reproduced it. Besides, part of the value in 17828906555453720229472878214886 is 17828906, which is also a timestamp. I think here specifically an attention drop effect occurred, literally point-blank blindness.

The function itself is implemented. But now we need to do something complex — the complete information lifecycle. When a user uploads a photo, they can do a lot with it, but one of the functions is to ask AI to explain it. As a result, the photo gets a description as a property of the File record itself. Next, the user has a choice — whether to make this photo public. If they do, the photo must go through moderation, and after that it can no longer be edited; it can only be unpublished. The reluctance to allow editing after this is not related to a political decision, but is purely technical — in this case, it would be necessary to change the information lifecycle: first the user proposes changes, then they must go through the moderation stage again, and only after that apply to the photo itself. Right now, this is unnecessary overhead without a real need.

After the photo is published, it is not only available as a source of public information, including being placed on a map, but also as a source of information for an AI agent, which can tell other users where interesting things are based on their requests.