Comments (21)
flozz
commented on May 29, 2024
Hello,
I had a look on your pkgbuild. I think I found the problems. :)
The first thing is that you do not need to download assimp sources at all, it is included in the tarball you download from PyPI:
$ wget https://files.pythonhosted.org/packages/source/y/yoga/yoga-1.0.0.tar.gz
[...]
$ tar -xvzf yoga-1.0.0.tar.gz
[...]
$ ls -lah yoga-1.0.0/assimp
total 80K
drwxr-xr-x 7 fabien fabien 4,0K avril 27 20:13 .
drwxr-xr-x 5 fabien fabien 4,0K avril 27 20:13 ..
-rw-r--r-- 1 fabien fabien 345 avril 27 20:13 assimp.pc.in
drwxr-xr-x 2 fabien fabien 4,0K avril 27 20:13 cmake
-rw-r--r-- 1 fabien fabien 31K avril 27 20:13 CMakeLists.txt
drwxr-xr-x 2 fabien fabien 4,0K avril 27 20:13 cmake-modules
drwxr-xr-x 9 fabien fabien 4,0K avril 27 20:13 code
drwxr-xr-x 16 fabien fabien 4,0K avril 27 20:13 contrib
-rw-r--r-- 1 fabien fabien 4,5K avril 27 20:13 CREDITS
drwxr-xr-x 3 fabien fabien 4,0K avril 27 20:13 include
-rw-r--r-- 1 fabien fabien 3,8K avril 27 20:13 LICENSE
-rw-r--r-- 1 fabien fabien 933 avril 27 20:13 revision.h.in
The python3 setup.py build_py command seems to be useless here because it only compiles the library "in place", but the generated .so file will not be copied by the python3 setup.py install command (I do not know why setuptools have this behaviour).
As the library (.so) seems to be only compilable during the install command, you should remove the --skip-build from the command.
Here are all the command I used to generated an installation in a ./output folder:
wget https://files.pythonhosted.org/packages/source/y/yoga/yoga-1.0.0.tar.gz
tar -xvzf yoga-1.0.0.tar.gz
cd yoga-1.0.0
python3 setup.py install --root=./output --optimize=1
You should be able to reduce your pkgbuild to something like I think:
pkgname=('yoga')
_module='yoga'
pkgver='1.0.0'
pkgrel=4
pkgdesc="Yummy Optimizer for Gorgeous Assets"
url="https://github.com/wanadev/yoga"
depends=(python3 python-cffi python-pillow python-pycparser python-unidecode python-pyguetzli python-zopflipy)
makedepends=('python-setuptools')
license=('BSD')
arch=('x86_64')
source=("https://files.pythonhosted.org/packages/source/${_module::1}/$_module/$_module-$pkgver.tar.gz")
sha256sums=('a4e7010969602b42af2ca74c95126ee533e3cf9ea9fdaa0c5c316eec7d985f53')
package() {
depends+=()
cd "${srcdir}/${_module}-${pkgver}"
python setup.py install --root="${pkgdir}" --optimize=1
}
Note:
- I fixed the license name
- According to the doc, the architecture cannot be set to
anyas there is compiled parts (and this is the same forpython-pyguetzliandpython-zopflipy; onlyyoga-image-optimizerhas no compiled parts in it) - I removed assimp from the dependencies as it is included
from yoga.
flozz
commented on May 29, 2024
Hum also, I do not know arch packaging so maybe it is assumed it is present, but please note that cmake is required to build assimp :)
from yoga.
Popolon
commented on May 29, 2024
If I don't get the source, it doesn't compile at all. At least it work with default assimp in my case with main account.
"any" is not as in Debian here. It means in a PKGBUILD it can be compiled in any architectures, else makepkg will throw an error and stop unless the architecture is present or -A flag is added to makepkg, this is the habit on most pyhon packages. As these package are architecture independant, it works fine.
Here are the error when I compile with included assimp:
running build_py
-- The C compiler identification is GNU 11.1.0
-- The CXX compiler identification is GNU 11.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Shared libraries disabled
-- compiling zlib from sources
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of off64_t
-- Check size of off64_t - done
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Enabled importer formats: AMF 3DS AC ASE ASSBIN B3D BVH COLLADA DXF CSM HMP IRRMESH IRR LWO LWS M3D MD2 MD3 MD5 MDC MDL NFF NDO OFF OBJ OGRE OPENGEX PLY MS3D COB BLEND IFC XGL FBX Q3D Q3BSP RAW SIB SMD STL TERRAGEN 3D X X3D GLTF 3MF MMD
-- Disabled importer formats:
-- Enabled exporter formats: OBJ OPENGEX PLY 3DS ASSBIN ASSXML M3D COLLADA FBX STL X X3D GLTF 3MF PBRT ASSJSON STEP
-- Disabled exporter formats:
-- Configuring done
-- Generating done
-- Build files have been written to: /data/pacaur/yoga/src/yoga-1.0.0/assimp/build
[ 0%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/compress.c.o
[ 0%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/adler32.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 1%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/crc32.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 1%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/deflate.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 2%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/gzclose.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 2%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/gzlib.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 3%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/gzread.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 3%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/gzwrite.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 3%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/inflate.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 4%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/infback.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 4%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/inftrees.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 5%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/inffast.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 5%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/trees.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 6%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/uncompr.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 6%] Building C object contrib/zlib/CMakeFiles/zlibstatic.dir/zutil.c.o
cc1: attention: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C
[ 7%] Linking C static library libzlibstatic.a
[ 7%] Built target zlibstatic
[ 7%] Building CXX object code/CMakeFiles/assimp.dir/CApi/CInterfaceIOWrapper.cpp.o
[ 7%] Building CXX object code/CMakeFiles/assimp.dir/Common/Assimp.cpp.o
[ 8%] Building CXX object code/CMakeFiles/assimp.dir/Common/BaseImporter.cpp.o
[ 8%] Building CXX object code/CMakeFiles/assimp.dir/Common/BaseProcess.cpp.o
[ 9%] Building CXX object code/CMakeFiles/assimp.dir/Common/PostStepRegistry.cpp.o
[ 9%] Building CXX object code/CMakeFiles/assimp.dir/Common/ImporterRegistry.cpp.o
[...] I cut these part here as everntyhing goes without error
[ 77%] Building CXX object code/CMakeFiles/assimp.dir/AssetLib/X3D/X3DImporter.cpp.o
[ 77%] Building CXX object code/CMakeFiles/assimp.dir/AssetLib/glTF/glTFCommon.cpp.o
[ 78%] Building CXX object code/CMakeFiles/assimp.dir/AssetLib/glTF/glTFImporter.cpp.o
[ 78%] Building CXX object code/CMakeFiles/assimp.dir/AssetLib/glTF2/glTF2Importer.cpp.o
Dans le fichier inclus depuis /data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/glTF/glTFAsset.h:1159,
depuis /data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/glTF/glTFImporter.cpp:45:
/data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/glTF/glTFAsset.inl: Dans la fonction membre « void glTF::Asset::Load(const string&, bool) »:
/data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/glTF/glTFAsset.inl:1304:29: erreur: écrit 1 octet dans une région de taille 0 [-Werror=stringop-overflow=]
1304 | sceneData[mSceneLength] = '\0';
cc1plus : tous les avertissements sont traités comme des erreurs
make[2]: *** [code/CMakeFiles/assimp.dir/build.make:2372 : code/CMakeFiles/assimp.dir/AssetLib/glTF/glTFImporter.cpp.o] Erreur 1
make[2]: *** Attente des tâches non terminées....
make[1]: *** [CMakeFiles/Makefile2:169 : code/CMakeFiles/assimp.dir/all] Erreur 2
make: *** [Makefile:136 : all] Erreur 2
and then it try to make the python setup and fail too.
from yoga.
Popolon
commented on May 29, 2024
I coded a lot in C++ during several years, until 20 years ago, so the language growed, but basis are the same :).
In my point of view, problem is here:
// read the scene data
std::vector<char> sceneData(mSceneLength + 1);
sceneData[mSceneLength] = '\0';
- sceneData is a varialbe of type
std::vector<char> - You need to first declare it or to assign to him an instance of a class or memory block a malloc like function.
This glTFAsset.inl file isnt present int he original assimp package.
from yoga.
Breush
commented on May 29, 2024
We're not in java. The variable constructor is called with the provided size. My feeling is that the gcc11 warning is wrong.
from yoga.
Popolon
commented on May 29, 2024
I never programed in java, don't know it.
from yoga.
Breush
commented on May 29, 2024
Sorry for my message that looks a bit aggressive now I read it gain. What I meant is that the c++ seems correct. Can You try disabling that GCC warning?
from yoga.
Popolon
commented on May 29, 2024
I read more doc about std::vector class and examples and you are true, sorry for my stupid statement I forgot this part of C++ with 20 years without practising, just remembered new form of allocator. More strangely, if I add this useless assignment it works:
std::vector<char> sceneData(mSceneLength + 1);
sceneData[0]= 'a';
sceneData[mSceneLength] = '\0';
But if fail again for another warning, warning level should probably lowered in python build, but I don't know python build system at all.
It looks like flags should be changed here in setup.py, (line 32). which flag do you suggest with today c++ standards?
if ccompiler.get_default_compiler() == "unix":
os.environ["CPPFLAGS"] = "--std=c++11"
Output of the next fail:
[ 9%] Building CXX object code/CMakeFiles/assimp.dir/AssetLib/M3D/M3DExporter.cpp.o
Dans le fichier inclus depuis /data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/M3D/M3DWrapper.h:59,
depuis /data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/M3D/M3DExporter.cpp:69:
/data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/M3D/m3d.h: Dans la fonction « unsigned char* m3d_save(m3d_t*, int, int, unsigned int*) »:
/data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/M3D/m3d.h:5523:46: erreur: array subscript « uint64_t {aka long unsigned int}[0] » is partly outside array bounds of « unsigned char [4] » [-Werror=array-bounds]
5523 | *((uint64_t *)out) = *((uint64_t *)&weights[0]);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
/data/pacaur/yoga/src/yoga-1.0.0/assimp/code/AssetLib/M3D/m3d.h:4349:43: note: while referencing « weights »
4349 | unsigned char *out = NULL, *z = NULL, weights[M3D_NUMBONE], *norm = NULL;
| ^~~~~~~
cc1plus : tous les avertissements sont traités comme des erreurs
make[2]: *** [code/CMakeFiles/assimp.dir/build.make:2568 : code/CMakeFiles/assimp.dir/AssetLib/M3D/M3DExporter.cpp.o] Erreur 1
make[1]: *** [CMakeFiles/Makefile2:169 : code/CMakeFiles/assimp.dir/all] Erreur 2
make: *** [Makefile:136 : all] Erreur 2
from yoga.
Popolon
commented on May 29, 2024
For more information, the problem is with gcc 11.1.0
from yoga.
Popolon
commented on May 29, 2024
For more information, the current problem is with gcc 11.1.0
here is the changelog, but it looks more like a bug than a willing change in behavior https://gcc.gnu.org/gcc-11/porting_to.html
from yoga.
Breush
commented on May 29, 2024
You can try:
os.environ["CPPFLAGS"] = "--std=c++11 -Wno-error=stringop-overflow -Wno-error=array-bounds"By the way, (if above works...) can you try CXXFLAGS instead of CPPFLAGS?
from yoga.
Popolon
commented on May 29, 2024
I wonder why for, size_t (c stdlib) is used instead of std::size_t (c++). Anyway I tried both in a simple example and they are compiled without problem (using size_t or std::size_t with the same GCC version.
#include <vector>
#include "stdlib.h"
int main()
{
size_t length=150;
std::vector<char> test(length+1);
test[length]='\0';
}
Worked both with g++ -o test_vector --std=c++11 test_vector.cpp and g++ -o test_vector --std=c++11 -Lstdlib test_vector.cpp
It will not be easy to make a bug report, if this is really a bug, but not related to a problem detected elsewhere.
It pass previous errors, but fail again here (the sameway with CPPFLAGS/CXXFLAGS. I believe they are mutually interchangable in gcc. Forcing g++ instead of gcc could resolve the issue.
[ 89%] Building C object code/CMakeFiles/assimp.dir/AssetLib/Assjson/cencode.c.o cc1: erreur: l'option de la ligne de commande « -std=c++11 » est valable pour C++/ObjC++ mais pas pour C [-Werror]
Beside this problem, good news I was able to build yoga on GCC10.2.1 on RISC-V 64 architecture
from yoga.
Popolon
commented on May 29, 2024
Works if removing --std=c++11 and adding the two parameters you give to avoid the warnings.
sed -i 's/--std=c++11/-Wno-error=stringop-overflow -Wno-error=array-bounds/' setup.py
python3 setup.py build_py
python3 yoga/model/assimp_build.py
Strangely with other users or root accounts, I still have this error:
$ yoga
Traceback (most recent call last):
File "/usr/bin/yoga", line 33, in <module>
sys.exit(load_entry_point('yoga==1.0.0', 'console_scripts', 'yoga')())
File "/usr/bin/yoga", line 25, in importlib_load_entry_point
return next(matches).load()
File "/usr/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 855, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/usr/lib/python3.9/site-packages/yoga/__init__.py", line 2, in <module>
from . import model # noqa
File "/usr/lib/python3.9/site-packages/yoga/model/__init__.py", line 174, in <module>
from .assimp import assimp_import_from_bytes, assimp_export_to_bytes
File "/usr/lib/python3.9/site-packages/yoga/model/assimp.py", line 6, in <module>
from ._assimp import lib, ffi
ModuleNotFoundError: No module named 'yoga.model._assimp'
from yoga.
flozz
commented on May 29, 2024
Ok I think I have the right commands to build now:
# Get source
wget https://files.pythonhosted.org/packages/source/y/yoga/yoga-1.0.0.tar.gz
tar -xvzf yoga-1.0.0.tar.gz
cd yoga-1.0.0
# Build
python3 setup.py build
# Install in ./output/usr
python3 setup.py install --prefix=./output/usr --optimize=1 --skip-build
from yoga.
Breush
commented on May 29, 2024
I wonder why for, size_t (c stdlib) is used instead of std::size_t (c++). Anyway I tried both in a simple example and they are compiled without problem (using size_t or std::size_t with the same GCC version.
std::size_t is an alias to "c-style" size_t. They are purely equivalent. One can be assigned to the other without a cast as there is no strong typing.
#include <vector> #include "stdlib.h" int main() { size_t length=150; std::vector<char> test(length+1); test[length]='\0'; }Worked both with
g++ -o test_vector --std=c++11 test_vector.cppandg++ -o test_vector --std=c++11 -Lstdlib test_vector.cppIt will not be easy to make a bug report, if this is really a bug, but not related to a problem detected elsewhere.
Add -Wall -Wextra -Werror to your gcc command line.
It pass previous errors, but fail again here (the sameway with CPPFLAGS/CXXFLAGS. I believe they are mutually interchangable in gcc. Forcing g++ instead of gcc could resolve the issue.
They're different. They are used by the build system, not GCC directly.
- CPPFLAGS means C-PreProcessor Flags (used by both C and C++)
- CXXFLAGS means C++ Flags (C equivalent is CFLAGS).
Setting CPPFLAGS to std=c++11 seem to be an error on our side, so I thought this could fix your issue.
Ok I think I have the right commands to build now: (...)
Nice!
I just don't know why your build with GCC11.1 seems to have -Wall -Werror enabled by default, but it looks like it is what is causing all your troubles.
from yoga.
flozz
commented on May 29, 2024
I started a documentation to help packagers: https://wanadev.github.io/yoga/contributing.html#packaging-yoga
from yoga.
Popolon
commented on May 29, 2024
@Breush Thank you for all these details.
@flozz python3-pil is missing for Bullseye (tested on RISC-V flavor) in the documentation.
from yoga.
Popolon
commented on May 29, 2024
Worked both with
g++ -o test_vector --std=c++11 test_vector.cppandg++ -o test_vector --std=c++11 -Lstdlib test_vector.cpp
It will not be easy to make a bug report, if this is really a bug, but not related to a problem detected elsewhere.Add
-Wall -Wextra -Werrorto your gcc command line.
No error either. The whole line I used:
g++ -o test_vector --std=c++11 -Wall -Wextra -Werror test_vector.cpp
from yoga.
flozz
commented on May 29, 2024
@flozz python3-pil is missing for Bullseye (tested on RISC-V flavor) in the documentation.
For source install it is not a system dependency as it should be installed from PyPI (like all other Python dependencies). But I understand that on non-intel platform, installing PIL/Pillow from PyPI could be problematic as there is probably no binary distribution (wheel) so you will have to compile it (and need to install all PIL / Pillow build deps)
from yoga.
Popolon
commented on May 29, 2024
When I said from source, I mean without pip, yes. So it could be a good idea to add this in documentation for non x86 platforms if no binary are available for non-x86 platform.
from yoga.
flozz
commented on May 29, 2024
@Popolon I added some links in the documentation to help :)
I close this issue as the AUR pkg seems to be ok now. Please let me know if there is more issues ;)
from yoga.
Related Issues (20)
- Add python needs in documentation
- Add Windows compatibility HOT 1
- GUI planned ? HOT 3
- Multithreading support ? HOT 5
- High memory usage during image conversion HOT 3
- [Feature request] progression indicator HOT 6
- Need more documentation about how to locally build Yoga for development work HOT 2
- Inverted normals on a specific model HOT 6
- Fix installation documentation HOT 3
- Option to keep metadata
- Add Draco encoder HOT 1
- YOGA Image crashes when an invalid value is present in a JPEG orientation tag
- Optimize outputted meshes with meshoptimizer HOT 3
- JPEG XL support HOT 2
- Cannot open /var/www HOT 2
- Allow to do a lossless jpeg to jpeg optimization using mozjpeg only HOT 2
- JPEG → JPEG optimization: keep the original image if the output image is bigger than the input one
- Grayscale encoding mode HOT 3
- Keep metadata when optimizing same format HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yoga.