git submodule sync
git submodule update --init
Make sure you compile/install FlashInfer first.
make codebase
make install-server
You can debug/edit code in the build folder. When done, use python copy_back.py to copy changes back to the original src folder.
make install
HF_HUB_ENABLE_HF_TRANSFER=1 pytest -s -vv --disable-pytest-warnings -m "punica_test" build/server/tests
- to generate the client stub and service stub of grpc
The current working dir is : /gpfsnyu/scratch/yy4108/torch-MIL/build/clients/python
python -m grpc_tools.protoc -I../../proto --python_out=. --grpc_python_out=. generate.proto