Deploying hoarder-docs with    Cloudflare Pages

View logs

Deploying hoarder-landing with    Cloudflare Pages

View logs

Just linking #1315 here as I think it might be better to make it part of the first release that contains embeddings, otherwise we'll have to deal with the issue of resetting all the embeddings when switching to binary or not.

I hope you won't mind but I was wondering what were the blockers for this much awaited feature. What is the state of the code, what are questions left to answer, what can the community do to help, etc.

Because it's a major change, I'm afraid no one other than the maintainer would want to tackle it and so I think communication here would go a long way to make us help you without duplicating too much effort.

@thiswillbeyourgithub So generating the embeddings currently work. The biggest blocker for this (beside my time) is me choosing what vector storage service to use. I initially was planning on sqlite-vec but the project has been in pre-alpha for a while and it's not clear to me if it'll exit this status. Meillisearch is another candidate, however, for a long period of time, the meillisearch version in karakeep was the one before the addition of vector search, so a lot of user will have to upgrade meillisearch to make use of embeddings. So I'll need to check if there's a backward compatible way for doing this.

Another caveat is that a lot of ollama users don't have an embeddings model set, so I'll need a way to communicate that as well.

Nothing major left, but a lot of tiny things for a smooth rollout of such a feature.

I don't think that the lack of embedding model on most user's setup is an issue because ollama use is opt in. As long as you haven't set an embedding model, you won't have embeddings search and that's fine.

Fwiw: on ollama i've been happy with arctic embed 2 with quantization (not enabled by default, but ends up with a 450mb model or so) link. I would gladely contribute my Modelfile to help others set this up.

Personaly, my money is on meillisearch. If you don't mind (i.e. don't tell me shortly not to do so) I'll take a look on their repo and maybe ask them directly for guidance.

Also I'd say that for a smooth rollout it might be good to do a pre release/alpha/beta for a few weeks/month beforehand to iron out the misshaps.

Edit: I took a look at meilisearch: what do you think of a docker one liner using alpine to execute an rm command on the meillisearch db then asking the user to force reindexing the db in karakeep?

Edit: after reading issue 2570 of meillisearch (not linking to avoid making a fuss) I kinda get the feeling that they don't want to make the upgrade easy because they view "easy upgrade" as their moat. Not very reassuring but still a good product. I think the alpine way is a good solution. Or using a new volume entirely.

I was suddenly reminded of that prominent quote in the readme:

This app is under heavy development and it's far from stable.

So my opinion is now that, as long as there is no data loss (ever!), this disclaimer grants you the moral right to move faster on those roll out situations.

• ollama users will figure it out
• us contributors will help with PRs left and right to iron out the time consuming kinks (docs, guides, UI, perf, etc)
• this kind of major feature that touches both frontend and backend seems (to me) at this stage best handled by the maintainer, saving coordination overhead
• with this disclaimer: the goal is to move fast to reach quickly the "stable" state, as a lot of users are probably hesitant to join as long as the disclaimer is in place

Just my 2 cents on priorities of course. I'm aware you're very busy and am sure you'll do what's best for us all.

We are using the same OpenAI api-key and base url for both inference and embedding. I think have separate config for each of them would be better, as it allows for using different providers for each of these tasks. In addition, this also would remove the limitation of having to use a provider who supports both embeddings and chat.

We are using the same OpenAI api-key and base url for both inference and embedding. I think have separate config for each of them would be better, as it allows for using different providers for each of these tasks. In addition, this also would remove the limitation of having to use a provider who supports both embeddings and chat.

I sort of disagree as for more cutstomization you should instead run a litellm instance

We are using the same OpenAI api-key and base url for both inference and embedding. I think have separate config for each of them would be better, as it allows for using different providers for each of these tasks. In addition, this also would remove the limitation of having to use a provider who supports both embeddings and chat.

I sort of disagree as for more cutstomization you should instead run a litellm instance

Litellm is one of the llm gateway providers that support embeddings , there are others that do not support embeddings but solely focus on llm inference (openrouter, cloudflare). Adding this would be good separation of concerns imo, if needed we can default to the inference parameters if the embedding params are not set.