Comments (17)
Are you using the llvm hash in the git submodule?
https://buildkite.com/iree
from torch-mlir.
I don't think so? Could you tell me how I can do that?
I think the following message while building (that I ignored) can have relevance here too:
realpath: /home/nithin/mlir-npcomp/build/iree/bindings/python: No such file or directory
from torch-mlir.
'git submodule init'
'git submodule update'
This will result in an llvm tree in external/llvm-project.
Or you can just checkout the git hash that is referenced (currently c89e46e) in your preferred llvm location
You will need the mlir-hlo submodule, however.
from torch-mlir.
I did initialize submodule inside mlir-npcomp and the above two errors persist.
from torch-mlir.
The above errors look like a mismatched LLVM version but it is hard to tell without further details.
I just realized that the instructions in the README are slightly out of date with respect to the LLVM dependency (they reference what we had before switching to a submodule). I'm going to fix them in a little bit, but if you could confirm the following it would help.
This section should be removed, and make sure that the LLVM_SRC_DIR variable is not set:
export LLVM_SRC_DIR=/path/to/llvm-project
# Check out last known good commit.
LLVM_COMMIT="$(cat ./build_tools/llvm_version.txt)"
(cd $LLVM_SRC_DIR && git checkout $LLVM_COMMIT)
Then git submodule update --init
.
Then proceed with the build:
./build_tools/install_mlir.sh
./build_tools/cmake_configure.sh
# Build and run tests
# ./build_tools/test_all.sh runs all of these commands.
cd build
ninja
ninja check-npcomp
from torch-mlir.
Thank you, your suggestion solved the error however I am still unable to build successfully because of the following error:
./frontends/pytorch/csrc/aten_mlir_type_default.cpp:10:10: fatal error: 'ATen/CPUGenerator.h' file not found #include <ATen/CPUGenerator.h> ^~~~~~~~~~~~~~~~~~~~~
The file ATen/CPUGenerator.h isn't present in the source-compiled pytorch. However, I was able to find the file in a conda installation of pytorch.
from torch-mlir.
The PyTorch integration is very much not working out of the box yet. There are a couple of PRs pending that Stephen and I are working through. Also, the current dispatch approach is pegged to a strategy for an older version of PyTorch and we have work to do to upgrade.
from torch-mlir.
Could you please tell me which pytorch version mlir-npcomp is based on?
from torch-mlir.
It is one of the 1.3 versions, but it is presently very version sensitive. The only version that it builds against is what is packaged/built as part of the docker image defined here: https://github.com/llvm/mlir-npcomp/pull/33/files
We're still working through getting this in a shape for general use and there aren't even presubmit builds for it yet.
from torch-mlir.
Oh alright, got it. Thank you very much for all the information and prompt replies Stella.
from torch-mlir.
You're welcome - we are actively working on this. It just isn't there yet and is suffering a lot from "works on my machine"...
from torch-mlir.
@stellaraccident @stephenneuendorffer do you know what exact pytorch 1.3 SHA works on your system ? The Dockerfile just mounts an external pytorch directory but I couldn't locate the version.
from torch-mlir.
nvm - I think I found the SHA from the implicit 19.10-py3 Docker version.
from torch-mlir.
Great! if you use the docker image, you'll also have to rebuild pytorch as well. I think some of the cmake bits are not installed properly out of the box. Let me know if you have questions.
from torch-mlir.
Sounds good will keep you posted. Meanwhile have you tried with the latest Pytorch by any chance? We have some dependency on latest pytorch and have to use it for our runtime. Thx
from torch-mlir.
We're pretty sure it won't work. Parts of the pytorch integration are generated from export information that is generated by pytorch. This export information no longer has the same format, and there is a new mechanism for exporting information about the ATen interface. The plan is to rebuild things around this new source of export information, which should be more stable.
from torch-mlir.
The project has now graduated slightly past "it works on my machine" and our build/docs are more robust (though still not perfect). I haven't seen these issue crop up for anybody in months, so closing this bug.
from torch-mlir.
Related Issues (20)
- `make_simple_dynamo_backend` does not work for single argument models. HOT 2
- Compilation of Quantized Model Failed HOT 1
- Missing dtype support for OnnxToTorch HOT 3
- Strange issue related to `pyproject.toml` when building release wheels HOT 4
- iree-compiler failed to compile gpt2
- `NotSupportedError` error when executing Readme example "TorchScript ResNet18"
- onnx.Transpose version 1 not lowered (but it's essentially the same as version 13)
- boolean indexing ops: AtenNonzeroOp, AtenIndexTensorOp, AtenMaskedSelectOp
- QuantizedMLP currently failing in CI
- migrate to from member `x.cast<T>()` to `mlir::cast<T>(x)` HOT 1
- Update version pin to onnx 1.16 HOT 5
- Include Shape Information Into DecomposeComplexOps Patterns HOT 4
- onnx.MatMul version 1 not lowered (but it's essentially the same as version 13)
- Crash with TorchBackendToLinalgOnTensorsBackendPipeline
- link out of date
- unsupported aten.meshgrid operator HOT 1
- Trying to lower elided IR from torch onnx to torch causes crash HOT 2
- [ONNX] Systematically expand ONNX functions before conversion to Torch to avoid needing bespoke conversions HOT 12
- [onnx] Attempting to lower onnx mlir with elided attributes causes crash HOT 7
- Some issues with migrating to one-shot-bufferize HOT 11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torch-mlir.