It looks like that got it past the first error, but now I get this with my docker run set to: docker run -d --gpus=all -it -p 8501:8501 --add-host=host.docker.internal:host-gateway -e LLM=ollama/Meta-Llama-3.1-8B-Instruct-Q8_0:latest -e OLLAMA_API_BASE=http://host.docker.internal:11434/ --name neuml-rag neuml/rag

ResponseNotRead: Attempted to access streaming response content, without having called read().
Traceback:
File "/usr/local/lib/python3.8/dist-packages/streamlit/runtime/scriptrunner/exec_code.py", line 85, in exec_func_with_error_handling
result = func()
File "/usr/local/lib/python3.8/dist-packages/streamlit/runtime/scriptrunner/script_runner.py", line 576, in code_to_exec
exec(code, module.dict)
File "/app/rag.py", line 612, in
app.run()
File "/app/rag.py", line 579, in run
response = st.write_stream(response)
File "/usr/local/lib/python3.8/dist-packages/streamlit/runtime/metrics_util.py", line 408, in wrapped_func
result = non_optional_func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/streamlit/elements/write.py", line 167, in write_stream
for chunk in stream: # type: ignore
File "/usr/local/lib/python3.8/dist-packages/txtai/pipeline/llm/litellm.py", line 69, in stream
yield from self.response(result if stream else [result])
File "/usr/local/lib/python3.8/dist-packages/txtai/pipeline/llm/generation.py", line 120, in response
for chunk in result:
File "/usr/local/lib/python3.8/dist-packages/litellm/llms/ollama.py", line 368, in ollama_completion_stream
raise e
File "/usr/local/lib/python3.8/dist-packages/litellm/llms/ollama.py", line 327, in ollama_completion_stream
status_code=response.status_code, message=response.text
File "/usr/local/lib/python3.8/dist-packages/httpx/_models.py", line 576, in text
content = self.content
File "/usr/local/lib/python3.8/dist-packages/httpx/_models.py", line 570, in content

API address #7

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions