RAG #81
Description
Folks,
I am testing AutoGen for Retrieval Augmented Code Generation and Question Answering by following the notebook posted here, https://github.com/microsoft/FLAML/blob/main/notebook/autogen_agentchat_RetrieveChat.ipynb
I have a few follow-up questions,
- How do i specify which embedding function, e.g., openai.Embedding or HuggingFaceEmbedding, can be used in RetrieveUserProxyAgent?
- how do i configure the CharacterTextSplitter/RecursiveCharacterTextSplitter in RetrieveUserProxyAgent ?
- I encountered this error when i was going through the example notebook.
File ~/venv/GPT_venv/lib/python3.10/site-packages/autogen/retrieve_utils.py:220, in create_vector_db_from_dir(dir_path, max_tokens, client, db_path, collection_name, get_or_create, chunk_mode, must_break_at_empty_line, embedding_model)
212 for i in range(0, len(chunks), 40000):
213 collection.upsert(
214 documents=chunks[
215 i : i + 40000
216 ], # we handle tokenization, embedding, and indexing automatically. You can skip that and add your own embeddings as well
217 ids=[f"doc_{i}" for i in range(i, i + 40000)], # unique for each doc
218 )
219 collection.upsert(
--> 220 documents=chunks[i : len(chunks)],
221 ids=[f"doc_{i}" for i in range(i, len(chunks))], # unique for each doc
222 )
223 except ValueError as e:
224 logger.warning(f"{e}")
UnboundLocalError: local variable 'i' referenced before assignment