Comments (3)
I've the same issue.
@dedalo944 have you found a solution? The latest version works well on Windows but not on linux distro (i use debian)
from core.
I've the same issue. @dedalo944 have you found a solution? The latest version works well on Windows but not on linux distro (i use debian)
this one into upload.py, for now
def deep_copy_upload_file(upload_file: UploadFile) -> UploadFile:
file_bytes = upload_file.file.read()
new_upload_file = UploadFile(
filename=upload_file.filename,
file=io.BytesIO(file_bytes)
)
return new_upload_file
# receive files via http endpoint
@router.post("/")
async def upload_file(
request: Request,
file: UploadFile,
background_tasks: BackgroundTasks,
chunk_size: int | None = Body(
default=None,
description="Maximum length of each chunk after the document is split (in tokens)",
),
chunk_overlap: int | None = Body(default=None, description="Chunk overlap (in tokens)"),
stray = Depends(session),
) -> Dict:
"""Upload a file containing text (.txt, .md, .pdf, etc.). File content will be extracted and segmented into chunks.
Chunks will be then vectorized and stored into documents memory.
"""
# Check the file format is supported
admitted_types = stray.rabbit_hole.file_handlers.keys()
# Get file mime type
content_type = mimetypes.guess_type(file.filename)[0]
log.info(f"Uploaded {content_type} down the rabbit hole")
# check if MIME type of uploaded file is supported
if content_type not in admitted_types:
raise HTTPException(
status_code=400,
detail={
"error": f'MIME type {content_type} not supported. Admitted types: {" - ".join(admitted_types)}'}
)
# upload file to long term memory, in the background
background_tasks.add_task(
# we deepcopy the file because FastAPI does not keep the file in memory after the response returns to the client
# https://github.com/tiangolo/fastapi/discussions/10936
stray.rabbit_hole.ingest_file, stray, deep_copy_upload_file(file), chunk_size, chunk_overlap
)
# reply to client
return {
"filename": file.filename,
"content_type": file.content_type,
"info": "File is being ingested asynchronously",
}
from core.
Fixed in develop
, coming soon to next release
from core.
Related Issues (20)
- [BUG] AttributeError: 'coroutine' object has no attribute 'get' HOT 15
- BUG @hook After_Rabbit_Hole_splitted_text HOT 5
- [Feature] Using metadata to filter declarative memory enhances response accuracy HOT 2
- [Security] Add Dependency bot HOT 1
- [Code] Add python linter HOT 2
- [Refactor] Abstract Vector Memory and simple api from plugins HOT 1
- [Feature] Bash install scripts set for the cat HOT 4
- [Feature] Fallback to handler pip errors HOT 1
- [BUG] using Gemini LLM doesn't work in main branch HOT 1
- [BUG] Experimental message endpoint is broken
- [BUG] Using Gemini LLM allows you to just get one result HOT 1
- [BUG] Hooks with same priority collide HOT 3
- [Feature] Add hook Before Websocket Connection Is Accepted HOT 1
- [Feature]: Add groq llms HOT 1
- [BUG] Interface does not return error on not pdf files HOT 1
- [Feature] WhiteRabbit expansion HOT 2
- [ERROR] AsyncCompletions.create() got an unexpected keyword argument 'api_type'" HOT 2
- [BUG] Cohere API requires array of strings for stop sequence HOT 1
- Close ws connections inactive for more than x seconds
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from core.