macOS, Linux, and FreeBSD all support GCC 10, which has better support for concepts and <ranges>. This can help remove deps on range-v3, whose preprocessor usage can severely bloat compile times.

Describe the desired documentation change

Having documentation about how a dds project can be used from CMake is essential to allowing CMake users access to dds projects with minimal effort.

Existing Behavior and Potential Problems

The repository database file is often highly compressible, so there are potential bandwidth savings by downloading the db as a repo.db.gz instead of the uncompressed database.

Describe the scenario and circumstances that should be addressed

As currently documented, dds build system has a fairly rigid approach to dealing with dependencies, focusing on static archives for self-managed libraries and allowing system defaults for compiler's built-in standard ones.

However, for legal and/or technical reasons, dynamic linking is sometimes required. Instances of legal reasons are fairly obvious for anything deployed on end user machines, otherwise LGPL would not be considered different from GPL. The technical reasons are present in case of, for example, video games interfacing with rendering APIs, which have varying implementations depending on hardware and firmware setup.

Describe alternatives you've considered

For dependencies that one cannot (drivers) or otherwise doesn't need to modify, one could subvert the dependency management aspect of dds and copy the required header files into, depending on the application, src/ or include/ directory of their project, and then manipulate the linker flags accordingly. Compiler flags could also be used to include headers, which may or may not work depending on how it defines a compiler-defined header. One of the points of dds usage is dependency management, so these two are not satisfactory.

In case where one wants a modified DLL/SO/... and is not satisfied with static linking either due to legal obligations or other reasons (such as a wish to present a failsafe customization point), there doesn't seem to be an obvious way to do that with dds at the moment.

Describe the solution you'd like

These appear to be two separate problems which just happen to share a common root issue, and as such it appears to me that providing two separate solutions may be adequate.

For things which can be built, but should be put into a separate object for one reason or another, a configuration key that would force it to work that way would do that. This, in turn, presents another issue, as the dds packages are not supposed to be particularly configurable. The most satisfying solution, from the viewpoint of application developer, would be to allow multiple options, such as forcing static linkage of an entire library, forcing dynamic linkage of library and letting it decide about its dependencies, or mixing a selection of libraries into a single DLL/SO/..., which is in direct conflict to that. Alternatively, a library could enforce its preferred linking setup, and optionally present different versions of itself (or the package) to allow options, though given indirect dependencies and uniform library identifiers this could get out of control as well.

For things that simply cannot be built, but could benefit from being treated as a managed library, a more orthogonal solution would be needed, such as allowing import libraries / SOs as a special type of sources.

Additional context

The SDL2 library is interesting in that it uses manual dynamic linking of its dependencies, including, optionally, newer versions of itself when linked statically. However, this is not accomplished through an in-archive list of needed symbols in such modules, as its current build system requires some version of such dependencies to be installed in order to detect them at runtime. What is exactly happening there could warrant further investigation.

Describe the scenario and circumstances that should be addressed

• If all of the source files in a compiled library are identical to their previous build results, they do not need to be re-archived.
• If a generated test executable is bitwise-identical to its prior link result and the prior test run succeeded, it does not need to be re-run.

Describe the solution you'd like

dds needs to record a file hash of the object files added to an archive library, and skip generating the archive if they are identical to the previous build.

dds needs to fingerprint test executables, and, if the prior execution of the test succeeded and the file is entirely un-changed, execution of that test should be skipped.

Describe the desired documentation change

The main section title is still "dds 0.1.0-alpha.2 documentation" -- presumably this should now be "alpha.3".

When I try and follow the tutorial for setting up dds for the first time, under MSVC Developer Command Prompt and Powershell I get the following error:

PS C:\Users\luket\src\shopping-guru> dds build -t :msvc

      HALT: CONTRACT VIOLATION
      ----- ------------------
      What: A class/library/function invariant was broken.
  Expected: pkg_man.has_value()
  Location: File "D:\a\1\s\src\dds\source\dist.cpp", line 129,
            in [struct dds::sdist __cdecl dds::sdist::from_directory(const class std::filesystem::path &)]
   Message: All dirs in the repo should be proper source distributions. If you see this, it means one of the directories in the repository is not a valid sdist.
   Context: where.string() := C:\Users\luket\AppData\Local\dds\repo\[email protected]

If I run the same command under WSL using the Linux version of DDS everything works hunky dory! My file location is not as described in the error message but as shown under the PS command prompt above. ( C:\Users\luket\src\shopping-guru\src\hello-world.main.cpp)

How to install it?

Describe the bug

While neo-http supports and seems to work with HTTP "chunked" transfer-encoding, it appears that dds will get stuck on some and/or experience a stream I/O error that aborts the connection. This has only so far been seen in chunked messages coming from GitHub.

Steps to Reproduce

1. Ask dds to pull a file via HTTP from a server that may response with Transfer-Encoding: chunked (only seen on GitHub so far). This can be done with repoman or by having a repo listing that refers to a server that causes this issue.
2. If GitHub responds with a chunked encoding, dds will emit an I/O error. (Not sure what other servers might repro this.)

dds Output

[error] An error occured while downloading [https://github.com/vector-of-bool/neo-fun/archive/0.6.0.tar.gz]: Failed read in stream_io_buffers::next(): Input/output error

Desktop (please complete the following information):

• Operating System: Appears on all platforms intermittently (macOS, Linux, Windows, FreeBSD)
• Compiler: Any
• dds Version: New with HTTP support in 0.1.0-alpha.6

Describe the scenario and circumstances that should be addressed

dds doesn't cache generated binaries, instead relying on external tools such as CCache. dds has special understanding of the build process that can let it do better than CCache, and even work in scenarios in which CCache fails.

Describe the solution you'd like

What I'd like:

• dds to understand how to perform direct and preprocessed caching much as CCache does.
• dds knows about how paths can be normalized and pruned in a way that CCache cannot.
• dds can know what toolchain options are relevant to cache invalidation, whereas CCache reads the command-line verbatim. (See #39)
• CCache doesn't support MSVC, which is a huge gap in potential cache speedup.
• Stretch goals: Additional cache storage backends, i.e. S3 API, Redis, etc.

Describe the bug

If a subprocess is spawned (compiler, git, etc.) and the executable does not exist, the process will never exit after the fork() and the forked child is left forever

Steps to Reproduce

1. Execute dds with a compiler executable that does not exist.

Desktop (please complete the following information):

• Operating System: Linux (presumably any Unix-like)
• Version: 0.1.0-alpha.4

Describe the bug

Attempting to use Clang as the toolchain compiler does not work on MacOS.

Steps to Reproduce

1. Create a simple test project
2. Use dds build -t :clang
3. Get error message

dds Output

Assertion failed: (false && "No archive suffix"), function operator(), file /Users/runner/runners/2.163.1/work/1/s/src/dds/toolchain/from_dds.cpp, line 512.

Desktop (please complete the following information):

• Operating System: MacOS
• Compiler: Clang
• dds Version: 0.1.0-alpha.2

Additional context

Using dds build -t :gcc-9 works fine (using GCC provided by homebrew). Writing a custom toolchain file such as

Compiler-Id: Clang
C++-Compiler: clang++

produces the same error message as above.

Allow selection of the runtime library mode that should be used when compiling/linking

MSVC in particular has a dedicated switch for this option: /M[TD][d]. GCC has -static-libgcc and -static-libstdc++, and _GLIBCXX_DEBUG.

Describe alternatives you've considered

It is possible for a user to set these options manually with the flags, but it would be good if there was a built-in toolchain option that made it more obvious what was happening.

Describe the solution you'd like

Ideally, an option in a toolchain like so:

{
  runtime: {
    debug: true,
    static: false,
  }
}

that will set the appropriate compile/link flags. If these options aren't specified, let the compiler select a default.

Describe the scenario and circumstances that should be addressed

After compiling a file, any emitted warnings are not saved. This means that the next compile will not display the warning unless the file is recompiled. Although perhaps the user should fix the warnings before they disappear, sometimes the warnings are missed and the code is presumed to be warning free despite it very much not being so.

Describe alternatives you've considered

Many people tout this as the reason to use -Werror: if a warning stops the build, it's impossible to forget to address it. However, I disagree with -Werror for this purpose. Yes, warnings should be addressed. But sometimes, ephemeral warnings come up in the process of natural development, such as with unused variables. Halting the thought process to address the warning right away is undesirable. Warnings should be addressed, but they don't need to interrupt the programmer.

Forcing the programmer to address the warning right away can easily lead to (void)variable; committed and living forever. The variable never became used, but now the warning is hidden, and the programmer forgot to remove it.

-Werror can catch this mistake in CI, but it would be preferable to catch it before then.

Describe the solution you'd like

My ideal build system would cache the warnings generated by the compiler, emitting the warnings even if the file is not recompiled.

There is some nuance here, however. Sometimes, the sheer amount of false-positive warnings drowns out useful warnings, especially when using experimental implementations of new language features. In these cases, it may be reasonable to turn off re-emitting of warnings, perhaps simply saying "N warnings generated" instead.

Additional context

Perhaps relevant: Rust's cargo was recently changed to do exactly this:

Compiler warnings are now cached on disk. If a build generates warnings, re-running the build will now re-display the warnings.

The DDS filename extension is already being used by the DirectDraw Surface format.
This causes on windows that the files are being marked as DDS texture files

So maybe it would be better to choose a different filename extension so it is clearer if it is a DirectDraw Surface texture or a txt file for this

Describe the scenario and circumstances that should be addressed

Support tweak-headers, as outlined in here

Describe the solution you'd like

The top-most package root should be able to provide tweak-headers for the build, e.g. in a top-level conf/ directory (Maybe add to Pitchfork?), or possibly a --tweaks option on the dds build commandline. Should:

• The top-level tweak-headers dir should be added as an #include-search directory for every compilation in the build.
• Adding/removing tweak headers should invalidate incremental/cached compilations, since it can change the result of `__has_include()``. This is essential to prevent miscompilations and ODR issues.
• The tweaks-header directory is not exported as part of any source distribution. If a dependency requires a specific tweak of another dependency, it should use a compile-time assertion to check and issue a helpful message to the user.

Building dds with msvc with the following toolchain:

Compiler-ID: MSVC
C++-Version: C++20
C++-Flags: /experimental:preprocessor /D_CRT_SECURE_NO_WARNINGS /DSPDLOG_COMPILED_LIB /wd5105 /std:c++latest /DCATCH_CONFIG_CPP17_STRING_VIEW
Warning-Flags: /W3
Link-Flags: rpcrt4.lib
Optimize: True

Describe the bug

I get the following output:

[13:47:15] [error] Failed to link executable [C:\Users\chbarto\source\repos\dds\_build\test\dds\package\id.exe]. Link command was [cl.exe /nologo /EHsc "C:\Users\chbarto\source\repos\dds\_build\obj\dds\package\id.test.cpp.obj" "C:\Users\chbarto\source\repos\dds\_build\libdds.lib" "C:\Users\chbarto\source\repos\dds\_build\_test-driver\v1\catch2\catch-main.cpp.obj" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libsemver.lib" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libsqlite3.lib" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libspdlog.lib" "/FeC:\Users\chbarto\source\repos\dds\_build\test\dds\package\id.exe" /O2 /MT rpcrt4.lib] [Exited 2], produced output:
id.test.cpp.obj : error LNK2019: unresolved external symbol "public: static class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl Catch::StringMaker<class std::basic_string_view<char,struct std::char_traits<char> >,void>::convert(class std::basic_string_view<char,struct std::char_traits<char> >)" (?convert@?$StringMaker@V?$basic_string_view@DU?$char_traits@D@std@@@std@@X@Catch@@SA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string_view@DU?$char_traits@D@std@@@4@@Z) referenced in function "class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl Catch::Detail::stringify<class std::basic_string_view<char,struct std::char_traits<char> > >(class std::basic_string_view<char,struct std::char_traits<char> > const &)" (??$stringify@V?$basic_string_view@DU?$char_traits@D@std@@@std@@@Detail@Catch@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV?$basic_string_view@DU?$char_traits@D@std@@@3@@Z)
C:\Users\chbarto\source\repos\dds\_build\test\dds\package\id.exe : fatal error LNK1120: 1 unresolved externals

[13:47:15] [error] Failed to link executable [C:\Users\chbarto\source\repos\dds\_build\test\dds\util\string.exe]. Link command was [cl.exe /nologo /EHsc "C:\Users\chbarto\source\repos\dds\_build\obj\dds\util\string.test.cpp.obj" "C:\Users\chbarto\source\repos\dds\_build\libdds.lib" "C:\Users\chbarto\source\repos\dds\_build\_test-driver\v1\catch2\catch-main.cpp.obj" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libsemver.lib" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libsqlite3.lib" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libspdlog.lib" "/FeC:\Users\chbarto\source\repos\dds\_build\test\dds\util\string.exe" /O2 /MT rpcrt4.lib] [Exited 2], produced output:
string.test.cpp.obj : error LNK2019: unresolved external symbol "public: static class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl Catch::StringMaker<class std::basic_string_view<char,struct std::char_traits<char> >,void>::convert(class std::basic_string_view<char,struct std::char_traits<char> >)" (?convert@?$StringMaker@V?$basic_string_view@DU?$char_traits@D@std@@@std@@X@Catch@@SA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string_view@DU?$char_traits@D@std@@@4@@Z) referenced in function "class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl Catch::Detail::stringify<class std::basic_string_view<char,struct std::char_traits<char> > >(class std::basic_string_view<char,struct std::char_traits<char> > const &)" (??$stringify@V?$basic_string_view@DU?$char_traits@D@std@@@std@@@Detail@Catch@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV?$basic_string_view@DU?$char_traits@D@std@@@3@@Z)
C:\Users\chbarto\source\repos\dds\_build\test\dds\util\string.exe : fatal error LNK1120: 1 unresolved externals

[13:47:15] [error] Failed to link executable [C:\Users\chbarto\source\repos\dds\_build\test\dds\toolchain\from_dds.exe]. Link command was [cl.exe /nologo /EHsc "C:\Users\chbarto\source\repos\dds\_build\obj\dds\toolchain\from_dds.test.cpp.obj" "C:\Users\chbarto\source\repos\dds\_build\libdds.lib" "C:\Users\chbarto\source\repos\dds\_build\_test-driver\v1\catch2\catch-main.cpp.obj" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libsemver.lib" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libsqlite3.lib" "C:\Users\chbarto\source\repos\dds\_build\_deps\[email protected]\libspdlog.lib" "/FeC:\Users\chbarto\source\repos\dds\_build\test\dds\toolchain\from_dds.exe" /O2 /MT rpcrt4.lib] [Exited 2], produced output:
from_dds.test.cpp.obj : error LNK2019: unresolved external symbol "public: static class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl Catch::StringMaker<class std::basic_string_view<char,struct std::char_traits<char> >,void>::convert(class std::basic_string_view<char,struct std::char_traits<char> >)" (?convert@?$StringMaker@V?$basic_string_view@DU?$char_traits@D@std@@@std@@X@Catch@@SA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$basic_string_view@DU?$char_traits@D@std@@@4@@Z) referenced in function "class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl Catch::Detail::stringify<class std::basic_string_view<char,struct std::char_traits<char> > >(class std::basic_string_view<char,struct std::char_traits<char> > const &)" (??$stringify@V?$basic_string_view@DU?$char_traits@D@std@@@std@@@Detail@Catch@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBV?$basic_string_view@DU?$char_traits@D@std@@@3@@Z)
C:\Users\chbarto\source\repos\dds\_build\test\dds\toolchain\from_dds.exe : fatal error LNK1120: 1 unresolved externals

[13:47:15] [error] Linking a runtime binary (executable/shared library/DLL) failed
[13:47:15] [error]
Linking a runtime binary file failed. There are a variety of possible causes
for this error. Refer to the documentation for more information.

[13:47:15] [error] Refer: https://vector-of-bool.github.io/docs/dds/err/link-failure.html

Desktop (please complete the following information):

• Operating System: Windows
• Compiler: MSVC 19.25.28508.3 for x64
• dds Version: 0.1.0-Alpha2

Describe the scenario and circumstances that should be addressed

dds requires that a toolchain be specified for any build operation. This can be a hassle for new projects or when the user has a common toolchain that they use which is different than any built-in toolchain.

Describe alternatives you've considered

Most other build and package systems will attempt to infer information about the system in order to create their default toolchain. This is antithetical to the dds design of "refusing the temptation to guess." The most "guesswork" that dds does is through the built-in toolchains, which are hard-coded rather than platform-detection-based guesses.

Describe the solution you'd like

There should be a filepath or some search path to find a "default toolchain" to use if none is specified. There should also be tiered toolchain searching, e.g.:

• Use toolchain.dds in the working directory
• Use toolchain.dds in the user-local config path
• Use toolchain.dds if found in some set of directories

Additional context

The platform's config directory specification should be obeyed.

I tried to run dds build in a dds project, and I got the following error:

[22:25:56] [critical] Reading file: /home/straversaro/src/dds-playground/dds/toolchain.dds: No such file or directory

If I understood well the spirit of the "Offering a Helping Hand" section in https://vector-of-bool.github.io/2020/01/06/new-decade.html , it would be great if the error included a reference to the documentation on the -t option to specify the toolchain to use.

Describe the bug

In MacOS, /usr/bin/g++ actually invokes clang++. A simple test project using dds build -t :gcc compiles successfully, but fails to link as it cannot find libstdc++fs (and even if it could it probably wouldn't work).

Steps to Reproduce

1. Write a simple test project
2. Attempt to build with dds build -t :gcc
3. Receive linker error message

dds Output

22:35:15] [info ] [example] Compile: example.test.cpp
[22:35:16] [info ] [example] Compile: example.test.cpp                          -     757ms [1/1]
[22:35:16] [info ] Compilation completed in 801ms
[22:35:16] [info ] Archiving completed in 0ms
[22:35:16] [info ] [example] Link: test/example                  
[22:35:16] [info ] [example] Link: test/example                   -     86ms
[22:35:16] [error] Failed to link executable [/Users/tristan/Coding/dds-experiments/example/_build/test/./example]. Link command was [g++ -fPIC -fdiagnostics-color /Users/tristan/Coding/dds-experiments/example/_build/obj/example.test.cpp.o /Users/tristan/Coding/dds-experiments/example/_build/_test-driver/v1/catch2/catch-main.cpp.o -pthread -lstdc++fs -o/Users/tristan/Coding/dds-experiments/example/_build/test/./example] [Exited 1], produced output:
ld: library not found for -lstdc++fs
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[22:35:16] [error] Linking a runtime binary (executable/shared library/DLL) failed
[22:35:16] [error] 
Linking a runtime binary file failed. There are a variety of possible causes
for this error. Refer to the documentation for more information.

[22:35:16] [error] Refer: https://vector-of-bool.github.io/docs/dds/err/link-failure.html

Desktop (please complete the following information):

• Operating System: MacOS
• Compiler: Clang pretending to be GCC
• dds Version: 0.1.0-alpha.2

Additional context

Perhaps on MacOS, the built-in :gcc toolchain should be special-cased to use the Clang Compiler-ID setting, rather than GNU? (Which also doesn't work right now, but that's #11 )

Description

Compilation fails if dds builds more than N source files at once, where N is the current resource limit as determined by ulimit, sysctl , or similar.

Steps to Reproduce

A couple ways to trigger this error...

1. Artificially decrease soft limit and then build:

ulimit -S -n $REALLY_SMALL_NUMBER
dds build ...

2. Define a library/package that depends on & uses many other libraries and then do a clean build.

dds Output

terminate called after throwing an instance of 'std::filesystem::__cxx11::filesystem_error'
  what():  filesystem error: cannot get current path: Too many open files

System / platform

• Operating System: macOS, Linux
• Compiler: GCC
• dds Version: 0.1.0-alpha.6

Additional context

Performing build within a clean package root that exhibits this error will fail repeatedly. i.e. dds starts building file 1/X (where X = num source files) on each and every run.
If build is issued after n/X object files already exist (where 1 < n < X), then dds starts building file n + 1 and it's sometimes possible to complete successfully - depending on whether X - n < L (L = resource limit).

Known workaround

Adjusting the resource limits for the process enables successful builds.

e.g. for environments using ulimit:

ulimit -S -n $SUFFICIENTLY_LARGE_NUMBER

Describe the scenario and circumstances that should be addressed

It is common for header files to accidentally become order-dependent, or to rely on other headers that may not be available to header consumers. This can cause compilation errors for consumers that #include headers in the incorrect order.

Describe the solution you'd like

dds should, in parallel with regular compilation, run a "syntax-only" pass over the headers of a project. For headers in include/, the src/ directory should be intentionally omitted from the search-path as to prevent the public headers from relying on implementation headers.

Describe the desired documentation change

Both the built-in toolchains and inferred toolchain options have implicit default values. Those values should be documented, of course.

Describe the scenario and circumstances that should be addressed

Semantic versioning offers some guarantees about compatibility between version numbers. It would be excellent if dds could automatically test these guarantees.

Describe the solution you'd like

• A dds test-minor (or other name) command that runs the project's own tests:
  • Compiles and runs prior versions within the same major version of the project's tests using the latest version of the projects library code. All tests within the same major version should compile and run, otherwise there is a breaking change.
• A dds test-bugfix that checks that a testcase from the current project tree fails when compiled against the prior patch version, ensuring that the bugfix has a valid test case added.

Open question for test-bugfix: How to test compile-time bugfixes?

Describe the scenario and circumstances that should be addressed

If project Gadgets uses Widgets, and Gadgets identifies a bug in Widgets, it should be possible, from the Gadgets workspace, to create a "bugfixing workflow" against Widgets.

Describe the solution you'd like

Possible workflow:

• From the gadgets project dir: dds bugfix-dep init widgets, and dds will clone a copy of widgets at the latest bugfix version for the Major.Minor release depended on by widgets.
• From the widgets project dir: dds bugfix-dep check-self:
  • Compile and run the working project's unit test files against the upstream version of the code, to check that the tests fail to compile or exit with error, to validate that there have been new tests created that check the bugfix.
  • Compile and run the working project's unit tests against itself, to check that the a fix has been created.
• From the gadgets project dir: dds bugfix-dep check widgets will run dds build using the cloned copy of widgets instead of the version in the local cache, to check if the widgets changes actually contain fixes for gadgets.
• From the gadgets project dir: dds bugfix-dep patch widgets will create a patch file that contains the changes.

Describe the bug

The built-in :clang toolchain uses the -static-libgcc -static-libstdc++ link options, neither of which are supported on MacOS. The same thing happens with :gcc, as g++ is an alias for clang++ on the Mac.

Steps to Reproduce

1. Follow the first steps from the DDS tutorial
2. Attempt to build with dds build -t :clang as instructed
3. Observe linker error

dds Output

[14:04:17] [error] Failed to link executable [/path/to/example/_build/hello-world]. Link command was [clang++ -fPIC /path/to/example/_build/obj/hello-world.main.cpp.o -pthread -o/path/to/_build/hello-world -static-libgcc -static-libstdc++] [Exited 1], produced output:
clang: error: unsupported option '-static-libgcc'

[14:04:17] [error] Linking a runtime binary (executable/shared library/DLL) failed
[14:04:17] [error] 
Linking a runtime binary file failed. There are a variety of possible causes
for this error. Refer to the documentation for more information.

[14:04:17] [error] Refer: https://vector-of-bool.github.io/docs/dds/err/link-failure.html

Desktop (please complete the following information):

• Operating System: macOS
• Compiler: Clang
• dds Version: 0.1.0-alpha.3

Describe the scenario and circumstances that should be addressed

alpha.6 needs a way to search/view the available packages.

Describe the solution you'd like

A dds pkg search subcommand should allow one to specify a package name or pattern to find packages that exist matching the requested name. It should display a list of available versions and the repository they come from.

Describe the scenario and circumstances that should be addressed

If I am the author of [email protected], and I am depended on by some set of libraries that I may not even be aware of, and I want to add a bugfix or feature to widgets, then I want to check that those changes do not break my dependees.

Describe the solution you'd like

The dds package database contains sufficient information that we can find who in the repository depends on a certain version of the package. A possible workflow:

• From [email protected] working repository, run: dds test-dependees:
  • Find all dependees of widgets@[1.0.0, 1.2.3), and download, build, and execute their tests, but using the local modified version of [email protected] instead of the version of widgets in the repository.

Additional context

For low-level libraries, this could be a very large number of dependees, so there should be a way to control the set of dependees that are tested. e.g.: Test direct dependees only; test only the latest version of dependees; continually expand the test space and stop on the first failure.

I searched in dds documentation, but I could not find any detail on how to build the dds tool itself. I tried to just download a dds binary and tried to run dds built -t :gcc in the dds directory, but I get this error:

straversaro@iiticublap103:~/src/dds-playground/dds$ dds build -t :gcc
[22:16:28] [error] Dependency resolution has failed! Explanation:
[22:16:28] [error] ┌─ Given that [email protected] is not available,
[22:16:28] [error] │         and [email protected] is required,
[22:16:28] [error] ╘═       then Dependencies cannot be satisfied.
[22:16:28] [error] `dds` was unable to find a solution for the package dependencies given.
[22:16:28] [error] 
The dependency resolution algorithm failed to resolve the requirements of the
project. The algorithm's explanation should give enough information to infer
why there is no possible solution. You may need to reconsider your dependency
versions to avoid conflicts.

[22:16:28] [error] Refer: https://vector-of-bool.github.io/docs/dds/err/dep-res-failure.html

I guess I could inspect the continuous integration scripts to understand how to build dds, but it would be great if this info could be provided as part of the docs itself.

Existing Behavior and Potential Problems

DDS's key-value file format is simple and easy to use.

However, the use of a custom format is not without its downsides. Not only does the extension .dds conflict with an existing file type (#2), but even if it did not, editors will by default only recognise .dds files as plain text -- if they open them at all. Without built-in support or custom plugins, users will not get the benefit of syntax highlighting and "error squiggles" as they would for other formats.

Furthermore, the use of a custom format means that the authors of tools which wish to interact with DDS will need to write their own parsers and/or generators for .dds files.

Describe Alternatives

Re-using (a subset of) an existing, well-known, text-based data interchange format for DDS config files would allow editors to open and highlight them correctly and perform simple validation "out of the box", and would allow other tools to re-use existing libraries to read and write compatible files. Potential formats might include TOML, JSON, YAML or (shudder) XML.

The current format appears to be a near-subset of YAML -- the main difference being that YAML does not allow repeated keys, which are used in DDS (for example) to specify multiple dependencies. This could be avoided by using a sequence instead:

# Syntax highlighting for free!
Name: acme-gadgets
Version: 4.3.6
Namespace: acme

Depends:
  - acme-widgets: ^1.4.3
  - acme-gizmos: ~5.6.5
  - acme-utils: ^3.3.0

The major downside of YAML is that the specification is fearsomely complicated, with advanced features like anchors and references which are way beyond what DDS requires. A simpler alternative might be TOML, which seems to be a popular choice among recent package managers such as Cargo and Buckeroo. With TOML, the above might be become something like

[package]
Name = "acme-gadgets"
Version =  "4.3.6"
Namespace = "acme"

[depends]
acme-widgets = "^1.4.3"
acme-gizmos = "~5.6.5"
acme-utils = "^3.3.0"

The downside in this case is the need to explicitly wrap strings in quotes. Also worth noting that TOML does not allow the + character in keys, so a hypothetical toolchain.toml would need look something like:

Compiler-ID = "Clang"
Cpp-Version = 17        # Not C++-Version
Debug = false
Optimise = true
Cpp-Flags = "-fcoroutines-ts"
Warning-Flags = "-Wall -Wextra"

Existing Behavior and Potential Problems

dds will install the oldest available version of a package that matches the dependencies of the package P. If a direct dependency D has a declared version range that begins at V and (for any reason) version resolution resolves the oldest compatible version V' to be greater than V, then the effective lowest version is D@V', and not D@V (as would be expected).

Having the declared version range not match reality is a source of confusion for both package developers and users, but would otherwise silently occur. dds should issue an error that the version range is incorrect, as it means one of two things:

• The package has taken a dependency that forces an upgrade of D beyond what the user expected. If this passes silently, the developers of P will not realize that an upgrade to their direct dependencies has been forced, and may fail to document and respond to this change.
• The old version that was required has become unavailable, and dds would otherwise silently perform an upgrade. This silent upgrade spells disaster for reliable builds, and a user should be required to either fix the missing dependency or acknowledge that the upgrade is permissible.

The user should be provided with an build option to let the upgrade occur.

Describe the scenario and circumstances that should be addressed

Many Libraries have different headers and source files based on what operating system they run on (for instance GLFW). For some of those it is not trivial to simply replace them with calls from the stl. Simply disabling the files that are not intended for the current os has the potential to make the libraries compatible or trivial to port to dds.

Describe alternatives you've considered

Alternatives would be configuration steps like CMake has, but I think that works against the simple nature of dds

Describe the solution you'd like

Files that end (or possibly start) with an identifier for that operating system should only be considered part of the build if it matches the operating system it is built on.
For instance:

input.win.cpp
window.linux.cpp

(alternatively win_input.cpp or linux_window.cpp or something similiar)

Existing Behavior and Potential Problems
Currently the include directories include and src are listed as paths relative to the package root.
Some tools - in my case YouCompleteMe (which really means clangd, i suppose) -, however, will interpret these paths relative to the directory entry in compile_commands.json (i.e. _build) and therefore will not find the correct includes.

Describe Alternatives
I have been able to fix this locally by substituting in src/dds/toolchain/toolchain.cpp:103

auto shortest = shortest_path_from(inc_dir, cwd);
auto inc_args = include_args(shortest);

with

auto inc_args = include_args(inc_dir);

to make dds always generate absolute paths.

Alternatively, manually changing the directory value to the package root worked, too.

Describe the desired documentation change

I've read the docs on source distributions, the local package repository, the package catalog, and library and package dependencies. Despite this, I can't seem to wrap my head around how dds expects me to do package management.

Can we get a tutorial which uses a dependency? For example, a Hello World using fmt.

dds Should Be Able to Obtain Catalog Entries from a Remote Server

Managing the dds package catalog is currently a manual process. Having a server that can dispense catalog data would be a marked improvement.

Describe the solution you'd like

dds should be able to download catalog data from a remote server and update the local catalog. There should be support for multiple remotes. The information about available remotes and which remote creates a package entry should be stored in the database as well.

Additional context

Upload support isn't necessary for this phase.

Tasks that need completion for 0.1.0-alpha.3 release:

• Write a changelog
• Publish new docs to vector-of-bool.github.io
• Publish new FreeBSD build
• Update PMM to reference new binaries
• Create new bootstrapping checkpoint for json5 changes
• Update version numbers and references

Existing Behavior and Potential Problems

In the common case of a single library in a project, having to manage two files in annoying more than helpful, and unnecessarily confusing

Describe Alternatives

Instead, a merged package.json5 for the common cases, with a merge of depends and uses:

{
  depends: [
    "[email protected], uses: neo/sqlite3"
  ]
}

OR:

{
  depends: [
     {dep: '[email protected]', use: ['neo/sqlite3']}
  ]
}

Describe the desired documentation change

Documentation must be updated for alpha.6

The certificate for dds.pizza has expired since February 7. It should be renewed.

Description of issue

On Windows 10, initially successfully following along the Tutorials and Beginner’s Guide.

At A Hello, World Test, changing <root>/src/hello/strings.test.cpp to contain the Catch test code and compiling yields error Cannot open include file: 'catch2/catch.hpp': No such file or directory, whereas I expect it to be provided by dds.

Command: C:\Programs\bin\dds-win-x64.exe build -t :msvc

Similar error with: C:\Programs\bin\dds-win-x64.exe build -t :gcc

Steps to Reproduce

1. Download dds-win-x64.exe to C:\Programs\bin
2. Setup tutorial example in root {dds.pizza} upto and including A Hello, World Test
3. In root {dds.pizza} , run: C:\Programs\bin\dds-win-x64.exe build -t :msvc

dds Output

prompt> C:\Programs\bin\dds-win-x64.exe build -t :msvc
[15:10:29] [info ] [/dds.pizza] Compile: hello\strings.test.cpp
[15:10:29] [info ] [/dds.pizza] Compile: hello\strings.test.cpp                 -     441ms [1/1]
[15:10:29] [error] Compilation failed: {dds.pizza}\src\hello\strings.test.cpp
[15:10:29] [error] Subcommand FAILED [Exitted 2]: cl.exe /MT /EHsc /nologo /permissive- /I src /W4 /showIncludes /c "{dds.pizza}\src\hello\strings.test.cpp" "/Fo{dds.pizza}\_build\obj\hello\strings.test.cpp.obj"
Note: including file: src\hello/strings.hpp
...
Note: including file:    C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\ucrt\ctype.h
<root>\src\hello\strings.test.cpp(3): fatal error C1083: Cannot open include file: 'catch2/catch.hpp': No such file or directory

[15:10:29] [error] Compilation failed [<root>\src\hello\strings.test.cpp]
[15:10:29] [error] Source compilation failed.
[15:10:29] [error]
Source compilation failed. Refer to the compiler output.

[15:10:29] [error] Refer: https://vector-of-bool.github.io/docs/dds/err/compile-failure.html

Desktop:

• OS: Windows 10
• Compiler: MSVC, GCC
• dds Version: 0.1.0-alpha.5

Describe the bug

The generated .lmp file for a package A that uses libraries from package B should contain a Requires: A line. Because this is absent, libman import may not implicitly import transitive dependencies

Steps to Reproduce

Simplest example is to use spdlog/spdlog, which uses fmt/fmt.

Hello,
I try develop something like source package manager for c++. And I see that you use json... I was use too. But later changed this to yml - more comfortable format (you can provide commets in control files)

P.S.
If you wish we can contact and discuss problem packaging for c++... I m interesting talk (and develop) about this...

Describe the scenario and circumstances that should be addressed

[Before describing the requested feature, describe the scenario and circumstances in which it would be a benefit to the user.]

Describe alternatives you've considered

[Before describing your preferred solution, consider what alternatives might be available. If you can think of other less-preferable possibilities, describe them here and what drawbacks they may have. Otherwise, remove this section.]

Describe the solution you'd like

[Clearly describe the feature and/or workflow you'd like to see made available. If you've described alternatives, state the reason that it should be preferred. If multiple alternatives are equally beneficial, feel free to include them all here.]

Additional context

[Add any other context or screenshots about the feature request here.]

Describe the scenario and circumstances that should be addressed

Being able to inject certain package attributes (name, version, etc.) into the source code is a useful capability. Otherwise, the user must manually maintain a lot of package metadata in more than one location.

Describe alternatives you've considered

An alternative is to provide all source files in a project with the metadata in the form of preprocessor macros, but this means that any tweak to that metadata will necessitate a full rebuild of the project. This also means that downstream packages will need to see idential information to prevent ODR violations. This will also require that all such preprocessor macros be uniquely named. This would result in an explosion of macros as all sources will need to see everyone's metadata macros simultaneously.

Describe the solution you'd like

Two options:

Metadata template

Option: Provide a filename pattern that dds will treat as a metadata template, doing special find-replace semantics within and writing that to a new name based on the original. If compilable, it will be added as a compiled file to the library. If a header, it will be resolvable relative to its location in the source root. Possible template syntax:

#define MY_LIBRARY_VERSION __DDS_PACKAGE_VERSION
#define MY_LIBRARY_NAME    __DDS_LIBRARY_NAME

Downside: Will not play as nicely with IDEs, and this will allow the reliance on macros throughout the project and its dependents.

Metadata macros to specially-named compilable files

Option: If a compilable file matches a certain pattern, it will be compiled with metadata macros defined. It is up to the user to make those macros accessible via extern or functions.

Downside: The metadata values will not be available in a constexpr context to anyone except the designated source file.

Hi. Suppose this situation: project X depends on libraries Y and Z. Library Y depends on library T with version exactly 1. Library Z - on library T with version exactly 2. As well as I understand from reading https://vector-of-bool.github.io/docs/dds/guide/interdeps.html#so-if-this-is-the-case-why-use-ranges-at-all , such case is not allowed. This would be very cool to support this case. Even if this is not possible in general case, then, please, support it at least in situation when author of T cared of this. I mean the following: author of T releases every version of his library using differently named inline namespaces. Thus linking different versions of T in single binary is possible!

So, please support diamonds with different versions of common lib. If not in general case, then at least in case when author of common dependency thought about this

Describe the scenario and circumstances that should be addressed

If a file in the --tweaks-dir is not #included by any source file, it is probably a mistake by the build invoker.

Describe the solution you'd like

If any of the files in the given --tweaks-dir are unreferenced by the project's dependencies' builds, issue a warning. A hard-error is probably unwarranted.

Additional context

This will need to inspect the file-deps of each compilation. Cached/iterative compilation will require that those cached file-deps be included in considering whether a tweak-header is "unused" or not.

Existing Behavior and Potential Problems

Currently, dds considers every compilation flag to be relevant to incremental compilation. This isn't true for some flags, such as GCC/Clangs -fdiagnostics-color or MSVC's /nologo. Toolchain command-lines should be normalized and remove these to not trigger unnecessary rebuilds.

Describe the scenario and circumstances that should be addressed

dds gives special privilege to a "test-driver" library, which is currently only Catch2, and only the version that it ships. The way it links and uses the test-driver is also convoluted and slows the build.

Describe the solution you'd like

Allow a "test_uses" and "test_depends" so that a project can declare libraries that should be only available to tests, and do not act as transitive dependencies for its dependees.

Additional context

This will require that dds is able to load a different set of usage requirements for its .test.c[pp] files than for the other compiled files.

Describe the bug

The Catch test framework requires C++11, but Clang currently defaults to C++98. This leads to a large amount of "error spew" when dds attempts to compile catch-main.cpp.

Steps to Reproduce

1. Follow the tutorial steps using -t :clang
2. Try to add a Catch test as instructed
3. Observe a large number of build errors

dds Output

<very long error output snipped>

Desktop (please complete the following information):

• Operating System: macOS
• Compiler: Clang
• dds Version: 0.1.0-alpha.3

Existing Behavior and Potential Problems

dds's default toolchain behavior enables color output from GCC and Clang. This works well when dds is run from a terminal or ANSI-aware display, but this munges output otherwise.

Describe Alternatives

Not default-enabling colored output is an option, but the benefit of having colored output is stronger than occasionally appearing to be broken, and it isn't difficult to remove ANSI sequences from a string in-situ.

Note:

Users should be provided an option to force-allow ANSI output.

Describe the bug

When changing toolchain options, test-driver code should be recompiled, as those toolchain modifications may affect the ABI of the toolchain. (e.g. the runtime library used by MSVC).