google / cdep Goto Github PK
View Code? Open in Web Editor NEWCDep is a decentralized native package dependency manager with a focus on Android. Runs on Windows, Linux, and MacOS.
License: Apache License 2.0
CDep is a decentralized native package dependency manager with a focus on Android. Runs on Windows, Linux, and MacOS.
License: Apache License 2.0
Very commonly requested as a platform API, but doesn't have a stable ABI so we can't do that. Good candidate for cdep.
The example section is almost more of an integration test than it is sample / example code meant for humans. Most packages come with more than one example use case. I'm wondering if it makes sense to do one of two or both things:
1.) make this integration test / sanity check as the user story (what it currently is).
2.) allow for more than one entry in the examples or even packaged zip files complete with common build project files.
I think this is an issue as the developer that's consuming the package shouldn't have a need for downloading the original package to make heads or tails of it (get docs and sample code in same distribution rather a separated task).
One of the biggest problems with dependency management is where to find trusted sources of libraries. This tool helps integrate the library into my project, but it does not seem to have a way of discovering what is available? Something like https://bintray.com/bintray/jcenter or https://cocoapods.org/
I noticed cdep places downloads and exploded files in $HOME/.cdep
I'd like to keep build trees independent of each other. Can I control where those folders go?
I'm concerned about races between builds, and also about ease of space management.
My library has java code and assets - Ideally I want to distribute an .aar file and the headers. Does that work?
looking at this issue
The error is:
CMake Error at /Users/gfan/proj/android-vulkan-tutorials/tutorial06_texture/.cdep/modules/cdep-dependencies-config.cmake:169 (message):
Android runtime '' is not supported by com.github.ggfan:shaderc:0.0.1.
Supported: gnustl c++
This is due to the following:
if(NOT CMAKE_SYSTEM_VERSION EQUAL 1)
set(cdep_determined_android_runtime ${CMAKE_ANDROID_STL_TYPE})
set(cdep_determined_android_abi ${CMAKE_ANDROID_ARCH_ABI})
set(cdep_determined_android_platform_level ${CMAKE_SYSTEM_VERSION})
else()
set(cdep_determined_android_runtime ${ANDROID_STL})
set(cdep_determined_android_abi ${ANDROID_ABI})
set(cdep_determined_android_platform_level ${ANDROID_PLATFORM_LEVEL})
set(cdep_supports_compiler_features FALSE)
endif()
CMAKE_SYSTEM_VERSION is 24 ( target_sdk_version), and CMAKE_ANDROID_STL_TYPE is nothing.
how to fix it? thanks!
It looks like there's only support for an include directory (zipped) and abi aware .a libs in a zip to be part of the package. It would probably be good to add support for shaders and docs; or any other assets the dependency might need.
Thus when it gets exploded you might see something like this:
license
includes/*
libs/*
docs/*
shaders/*
assets/*
I can't think of many instances where assets would need to be propagated; however, some packages might really really want to ship a standard logo.
In addition, it would probably make sense to provide a new builder entry for shader compiling.
https://bitbucket.org/eigen/eigen/
It may be appropriate to host this on bitbucket instread of github since that is where the original eigen package is
That would help a bunch in having cdep adopted by projects with C/C++ code that want to work on Android and iOS. macOS support would probably easily fall down from that.
This issue was automatically created by Allstar.
Security Policy Violation
Project is out of compliance with Binary Artifacts policy: binaries present in source code
Rule Description
Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see the Security Scorecards Documentation for Binary Artifacts.
Remediation Steps
To remediate, remove the generated executable artifacts from the repository.
Artifacts Found
Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores a project's adherence to security best practices. You may wish to run a Scorecards scan directly on this repository for more details.
Allstar has been installed on all Google managed GitHub orgs. Policies are gradually being rolled out and enforced by the GOSST and OSPO teams. Learn more at http://go/allstar
This issue will auto resolve when the policy is in compliance.
Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.
If you want to fix one more error message, when I put the lib (.a) at the root of the .zip file it complain that it couldn't find the lib under abi/mylib.a
while it is actually looking inside lib/abi/mylib.a
it took me a while to figure out that the message say abi/mylib.a instead of lib/abi/mylib.a
Unfortunately, following https://github.com/gpx1000/cdep/blob/master/doc/authoring.md#step-14----prove-the-package-can-link uses the hosts' cmake. It should really be changed to use the ndk's cmake. Otherwise system files get used for headers and packages don't build correctly when they really should.
I'm following the authoring guide and at step 13 I run the following command:
./cdep fetch upload/cdep-manifest.yml
which generates the following error:
/Users/donturner/Code/workspace-android/oboe/upload/cdep-manifest.yml: error: SHA256 for file:/Users/donturner/Code/workspace-android/oboe/upload/oboe-headers.zip did not match constant from manifest [CDEP1c882e4] /Users/donturner/Code/workspace-android/oboe/upload/cdep-manifest.yml: error: SHA256 for file:/Users/donturner/Code/workspace-android/oboe/upload/oboe-armeabi.zip did not match constant from manifest [CDEP1c882e4] Fetch complete 2 errors, exiting
If I delete the folders shown by: cdep show folders
then the original command completes successfully.
How can I force CDep to recalculate the sha256 values for the various zip files in the upload folder?
Several libraries will have ABI specific code (inline asm, POT sizes etc) and thus use code generation to create specialized header files. To avoid requiring consumers having to recompile to regenerate headers and configs, probably need to allow for headers that are specific to abi in much the same way libs are currently working.
Comes up often on Stack Overflow in "how do I use X in the NDK".
after update cdep example to Android 3.0, it complains about CMAKE_SYSTEM_VERSION, not remember seeing before. what I am doing is setting it to 23 ( the library supported version ) right before calling
and it is fine, is it correct to reset it?
https://github.com/googlesamples/android-ndk/blob/master/hello-cdep/app/src/main/cpp/CMakeLists.txt
To reproduce the error, just remove the CMAKE_SYSTEM_VERSION inside the above mentioned CMakeLists.txt file
and you should see the error message of:
Android API level 1 is not supported by com.github.ggfan:hello-cdep:1.0.0
details messages:
Call Stack (most recent call first):
/Users/gfan/dev/android-sdk/cmake/3.6.4111459/share/cmake-3.6/Modules/CMakeDetermineSystem.cmake:98 (include)
CMakeLists.txt
-- Check for working C compiler: /Users/gfan/dev/sdk_current/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang
-- Check for working C compiler: /Users/gfan/dev/sdk_current/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Users/gfan/dev/sdk_current/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++
-- Check for working CXX compiler: /Users/gfan/dev/sdk_current/ndk-bundle/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /Users/gfan/proj/android-ndk/hello-cdep/.cdep/modules/cdep-dependencies-config.cmake:72 (message):
Android API level 1 is not supported by com.github.ggfan:hello-cdep:1.0.0
Call Stack (most recent call first):
/Users/gfan/proj/android-ndk/hello-cdep/.cdep/modules/cdep-dependencies-config.cmake:80 (add_cdep_hello_cdep_dependency)
CMakeLists.txt:36 (add_all_cdep_dependencies)
-- Configuring incomplete, errors occurred!
See also "/Users/gfan/proj/android-ndk/hello-cdep/app/.externalNativeBuild/cmake/debug/x86/CMakeFiles/CMakeOutput.log".
FAILURE: Build failed with an exception.
executing external native build for cmake /Users/gfan/proj/android-ndk/hello-cdep/app/src/main/cpp/CMakeLists.txt
Manifest should be okay but actual archive download at build time isn't
This one is in the system image, but we can't update old system images. Would be good to make it easier for people to package new versions in their apps.
I got this error in gradle:
Task :app:generateJsonModelDebug FAILED
CMake Error at F:\bitbucket\Android\NewCICBrowser\app\src\main\cpp\CMakeLists.txt:380 (find_package):
By not providing "Findcdep-dependencies.cmake" in CMAKE_MODULE_PATH this
project has asked CMake to find a package configuration file provided by
"cdep-dependencies", but CMake did not find one.
Could not find a package configuration file provided by "cdep-dependencies"
with any of the following names:
cdep-dependenciesConfig.cmake
cdep-dependencies-config.cmake
Add the installation prefix of "cdep-dependencies" to CMAKE_PREFIX_PATH or
set "cdep-dependencies_DIR" to a directory containing one of the above
files. If "cdep-dependencies" provides a separate development package or
SDK, be sure it has been installed.
I already added arguments in build.gradle:
externalNativeBuild {
cmake {
arguments "-Dcdep-dependencies_DIR=../.cdep/modules",
"-DCMAKE_ANDROID_STL_TYPE=c++_static"
}
}
Please help
the $(shell ...) line is prefixed with the space character. Sometimes, the rule seems to require that at runtime, it is a recipe line and thus requires the prefix of a tab.
Unfortunately, there's not a clear cut solution to this problem; the times where the line is not a recipe will create a different error as the recipe line comes before the first target line and it all depends upon the version of GnuMake the system has as to when it happens or why.
This problem does go away on its' own if the downloaded-package already exists. Thus I suggest potentially creating a script that just downloads everything in the cdep.yml dependency list as the project will potentially need them anyway. That's an ugly hack to do it, but it does work.
As a work around to anyone that encounters this bug; the $(shell ...) line does execute so just rerun ndk-build until all your packages are downloaded for your project and all subsequent ndk-builds will just work.
Another potential work around which also is my hypothesis for why Travis hasn't seen this manifest is to run cmake over your project (just the examples will do) as that will retrieve the packages into the right places and ndk-build will just work.
In order to promote adoption of CDep it would be great to have an at-a-glance list of pros and cons of using this package manager over other options such as:
There needs to be some form of solution to capture that cFlags, cppFlags, and ldFlags could have specific requirements for a dependency.
As an example solution:
capture target_* interface family of settings and populate it within the yml. Something similar would be required for android.mk; I'd suggest creating variable entries meant to enter environment variables _CFLAGS _LDFLAGS etc variables suitable to be used in any build platform.
running git clone https://github.com/jomof/cdep-redist.git and then running cdep wrapper on that directory makes cdep unusable since it overwrites itself with zero byte files
cdep outputs text in dark gray color. Original text color is not restored after exit and stays as dark gray. tested on OSX
Use com.github.gpx1000:curl:7.56.0 as a reference. Then follow the standard steps:
cmake -H.cdep/examples/cmake/ -Bbuild/examples -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_ANDROID_NDK_TOOLCHAIN_VERSION=clang -DCMAKE_SYSTEM_NAME=Android -DCMAKE_SYSTEM_VERSION=16 -DANDROID_PLATFORM=android-21 -DANDROID_ABI=armeabi -DCMAKE_TOOLCHAIN_FILE=${ANDROID_NDK}build/cmake/android.toolchain.cmake -DCMAKE_ANDROID_STL_TYPE=c++_static -DCMAKE_ANDROID_ARCH_ABI=armeabi
cmake --build build/examples
Note that it will fail to find any of the symbols in boringssl or zlib. The reason it does this is the last link step is ordered incorrectly:
${ANDROID_NDK}/toolchains/llvm/prebuilt/linux-x86_64/bin/clang++ --target=armv5te-none-linux-androideabi --gcc-toolchain=${ANDROID_NDK}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64 --sysroot=${ANDROID_NDK}/sysroot -fPIC -isystem ${ANDROID_NDK}/sysroot/usr/include/arm-linux-androideabi -D__ANDROID_API__=21 -g -DANDROID -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv5te -mtune=xscale -msoft-float -fno-integrated-as -mthumb -Wa,--noexecstack -Wformat -Werror=format-security -O2 -g -DNDEBUG -Wl,--exclude-libs,libgcc.a --sysroot ${ANDROID_NDK}/platforms/android-21/arch-arm -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--no-undefined -Wl,-z,noexecstack -Qunused-arguments -Wl,-z,relro -Wl,-z,now -shared -Wl,-soname,libcurl_target.so -o libcurl_target.so CMakeFiles/curl_target.dir/curl.cpp.o ~/.cdep/exploded/com.github.gpx1000/curl/7.56.0/curl-armeabi.zip/lib/armeabi/libcurl.a ~.cdep/exploded/com.github.gpx1000/boringssl/0.0.0/boringssl-armeabi.zip/lib/armeabi/libssl.a ~.cdep/exploded/com.github.gpx1000/boringssl/0.0.0/boringssl-armeabi.zip/lib/armeabi/libcrypto.a ~.cdep/exploded/com.github.gpx1000/zlib/1.2.11/zlib-armeabi.zip/lib/armeabi/libz.a ~.cdep/exploded/com.github.gpx1000/boringssl/0.0.0/boringssl-armeabi.zip/lib/armeabi/libssl.a ~.cdep/exploded/com.github.gpx1000/boringssl/0.0.0/boringssl-armeabi.zip/lib/armeabi/libcrypto.a ~.cdep/exploded/com.github.gpx1000/zlib/1.2.11/zlib-armeabi.zip/lib/armeabi/libz.a -lm "${ANDROID_NDK}/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi/libgnustl_static.a" "-latomic"
That is what it should be; however, the line that is generated places ~/.cdep/exploded/com.github.gpx1000/curl/7.56.0/curl-armeabi.zip/lib/armeabi/libcurl.a after ~.cdep/exploded/com.github.gpx1000/zlib/1.2.11/zlib-armeabi.zip/lib/armeabi/libz.a
The reason this is important is the linker doesn't recognize that z's symbols or boringssl's symbols exist in their respective archive libs at the time that it is trying to resolve curl's symbols. It must have a chance to resolve curl's symbols only after it has resolved z and boringssl.
I just tried to use cdep, but it initial experience on Windows is that it doesn't work at all.
Cloned the repository as described on the readme, SHA-1: ab6e03391d339dd75d90ccce55603675986b00ab
Opened Android Studio 2.3.3. and created an empty activity with C++ support enabled
Executed the ..\cdep-redist\cdep wrapper step as described on the instructions
Inside Android Studio, uncommented the line - compile: com.github.jomof:boost:1.0.63-rev10 on the file cdep.yml
It failed with a stack trace
C:\Users\.............\Downloads\MyApplication>.\cdep
Downloading https://github.com/jomof/boost/releases/download/1.0.63-rev10/cdep-manifest.yml
FAILURE (4cb6eaf): Cannot create property=android for JavaBean=io.cdep.cdep.yml.cdepmanifest.CDepManifestYml@1b26f7b2
in 'reader', line 1, column 1:
coordinate:
^
No suitable constructor with 1 arguments found for class io.cdep.cdep.yml.cdepmanifest.Android
in 'reader', line 6, column 3:
- file: boost_1_63_0.zip
^
FAILURE (152f827): groupId '' from manifest did not agree with github url https://github.com/jomof/boost/releases/download/1.0.63-rev10/cdep-manifest.
yml
FAILURE (136e12f): version '' from manifest did not agree with version 1.0.63-rev10 github url https://github.com/jomof/boost/releases/download/1.0.63
-rev10/cdep-manifest.yml
FAILURE (6af7405): Manifest was missing coordinate
FAILURE (8313000): Manifest was missing coordinate.groupId
FAILURE (b495b50): Manifest was missing coordinate.artifactId
FAILURE (31ca81f): Manifest was missing coordinate.version
FAILURE (fa75282): Package ':' has malformed version, expected major.minor.point[-tweak] but there were no dots
FAILURE (71b3ede): Package ':' does not contain any files
Modified cdep.yml to use - compile: com.github.jomof:sqlite:3.16.2-rev51 instead, like on the instructions
It failed with another stack trace
Generating C:\Users\.........\Downloads\MyApplication\.\.cdep\modules\cdep-dependencies-config.cmake
9 errors, exiting
C:\Users\..........\Downloads\MyApplication>.\cdep
[B@52feb982mapping values are not allowed here
in 'reader', line 11, column 15:
- coordinate: :
^
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchValue(ScannerImpl.java:871)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:360)
at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:226)
at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:585)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:157)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:147)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:132)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:246)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:237)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:225)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:155)
at org.yaml.snakeyaml.composer.Composer.composeSequenceNode(Composer.java:199)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:153)
at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:122)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:105)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:120)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:450)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:381)
at io.cdep.cdep.utils.CDepSHA256Utils.convertStringToCDepSHA256(CDepSHA256Utils.java:32)
at io.cdep.cdep.generator.GeneratorEnvironment.readCDepSHA256File(GeneratorEnvironment.java:222)
at io.cdep.CDep.handleGenerateScript(CDep.java:588)
at io.cdep.CDep.goNoScope(CDep.java:186)
at io.cdep.CDep.go(CDep.java:125)
at io.cdep.CDep.main(CDep.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at io.cdep.bootstrap.Bootstrap.go(Bootstrap.java:55)
at io.cdep.bootstrap.Bootstrap.main(Bootstrap.java:34)
In the file docs/android-studio-cmake.md on Step 5 it says:
"Let's reduce the set by editing the same build.gradle file from Step 4."
But no build.gradle file was edited in Step 4 which means it's not clear which build.gradle should be edited.
https://raw.githubusercontent.com/jomof/cdep/master/boot.yml
is not availabe now, 404
I tried to follow the doc to create a cdep package and reach the point where I have to check the integrity of the .yml
So I run: ./cdep fetch upload/cdep-manifest.yml (I put the yml also under upload directory like in the doc)
And I have these errors:
FAILURE (170a12c): Cannot create property=android for JavaBean=io.cdep.cdep.yml.cdepmanifest.CDepManifestYml@604ed9f0
in 'reader', line 1, column 1:
coordinate:
^
Cannot create property=archives for JavaBean=io.cdep.cdep.yml.cdepmanifest.Android@91161c7
in 'reader', line 14, column 3:
dependencies:
^
Cannot create property=lib for JavaBean=io.cdep.cdep.yml.cdepmanifest.AndroidArchive@5c3bd550
in 'reader', line 16, column 7:
- file: jnihelper-libs.zip
^
Unable to find property 'lib' on class: io.cdep.cdep.yml.cdepmanifest.AndroidArchive
in 'reader', line 23, column 12:
lib: armeabi/libjni-framework.a
^
in 'reader', line 16, column 5:
- file: jnihelper-libs.zip
^
in 'reader', line 14, column 3:
dependencies:
^
FAILURE (6af7405): Manifest was missing coordinate
FAILURE (8313000): Manifest was missing coordinate.groupId
FAILURE (b495b50): Manifest was missing coordinate.artifactId
FAILURE (31ca81f): Manifest was missing coordinate.version
FAILURE (fa75282): Package ':' has malformed version, expected major.minor.point[-tweak] but there were no dots
FAILURE (71b3ede): Package ':' does not contain any files
FAILURE (6af7405): Manifest was missing coordinate
FAILURE (8313000): Manifest was missing coordinate.groupId
FAILURE (b495b50): Manifest was missing coordinate.artifactId
FAILURE (31ca81f): Manifest was missing coordinate.version
FAILURE (fa75282): Package ':' has malformed version, expected major.minor.point[-tweak] but there were no dots
FAILURE (71b3ede): Package ':' does not contain any files
Fetch complete
13 errors, exiting
I've attached my .yml (rename .txt because of github).
I'm specially worried about the first error:
in 'reader', line 1, column 1:
coordinate:
if "coordinate:" should not be the first line of the .yml, what should it be ?
cdep-manifest.txt
cdep is a really awesome tool for Android Project intergration. But after reading through the README and tried some demos , i found that the cdep can only cooperate with github. Will it support gitlab in future releases? Thanks very much.
I couldn't tell from the usage docs.
When following the instructions here: https://github.com/google/cdep/blob/master/doc/android-studio-cmake.md
Android Studio produced the following error:
Error:ABIs [armeabie] are not available for platform and will be excluded from building and packaging. Available ABIs are [armeabi, armeabi-v7a, arm64-v8a, x86, x86_64, mips, mips64].
This is due to a typo in the docs, "armeabie" should be "armeabi"
Like the title says.
I might have misunderstood the usage model. But it looks like cdep wants to download components during a build:
I really don't want a build system to update its source tree during a build. Is there a mode for using cdep where it just resolves package dependencies? Then at least I can segregate that step into a stage before actually building a project.
The manifest parsing is overly particular about whitespace. It would be good to denote that all prefix space is required to be a " " space not a "\t". Also there MUST be no space between a key and ":" but MUST have space between ":" and value.
This bug can be addressed by documentation. However, for sanity sake, either a tool to do the file creation for us or relaxed rules surrounding whitespace is probably a good idea.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.