request to remove `cmake`, `highway`, and `gtest` about gemma.cpp HOT 10 CLOSED

I wrote a Makefile for gemma.cpp here if anyone wants it. https://github.com/jart/gemma3/blob/main/Makefile

from gemma.cpp.

I agree cmake can be painful but the landscape of build systems + package management for C++ ... leaves something to be desired and for all its faults cmake seems to be a common denominator. What would you prefer instead?

We try to remove as many dependencies as possible, highway would be difficult since we'd have to implement portable intrinsics from scratch.

There are some things that could be simplified, eg I think the transitive footprint of sentencepiece is pretty large, if something like https://github.com/karpathy/minbpe could be swapped in that would be a win in terms of simplifying dependencies. I'd be pretty excited for that if it could be made to work.

Did you run into problems with the build process?

from gemma.cpp.

I'd suggest that use git-submodule to replace the FetchContent system. So that we could build the gamme under a system without network.

from gemma.cpp.

If @jart is fine with it, what if we adapt that Makefile + submodules for local builds? If anyone wants to submit a PR I'm happy to review.

Has anyone run into specific problems / stumbling blocks with cmake? There were some challenges with windows but the recent PRs have hopefully fixed that.

As much as I'm not a huge fan of cmake ergonomics myself, I'd like to at least have it as an option since it's widely available + other projects using gemma.cpp as a dependency may use it.

from gemma.cpp.

One thing that rubbed me the wrong way about cmake was when I did a git checkout of a different branch inside build/_deps/highway/ then make just checked out the old revision again. That's why I wrote the makefile, so I could experiment with changing transitive sources at will. I even tried using the preprocessor to roll your project up into a single 100k line stb-like header until I understood why that wouldn't be a good fit for your crtp microarchitectural support trick.

If you want the Makefile I wrote, it's yours. I'm happy to public domain it or sign any CLA you like. If you want me to send a PR, then do you want to keep cmake or should this be an additional build system sort of like what llama.cpp does? If so I'd encourage caution. Cmake isn't terrible from a user standpoint and it probably helps Microsofties who'd most likely complain about my Makefile.

from gemma.cpp.

This is a nice developer oriented discussion. I found the cmake build system easy to follow and the barrier to build and try it was very low. I have tired bazel in the past and have a sour taste about the time it took to download, setup and compile the project. That said, I can understand Gemma using Google's workflow :D. On that note thanks for including support for cmake.

I have not specifically reviewed sentence piece and will refrain from commenting on that.

As it related to Highway, it looks to be a really cool library to achieve project scalability across devices. But unfortunately I think it's a pretty involved as it relates to suggesting modifications and I'm left thinking "man!, I'd like a stripped down version without HWY". For me this has the following benefits:

1. easy to follow the code interms of reviewing the implementation - even as an awesome reference - opportunity to redo the implementation from a learning point of view
2. remove barriers to adoption - one of exciting elements of Andrej's Llama2.c is it's spelled out version. (Don't get e wrong, I found the current Gemma implement well done and clean - but it's reasonably advanced ). You can also get a sense for this by the diversity of PR's to Andrej's code base - I have noticed PRs getting sent out based on a latest improvement in llama.cp <just makes it exciting!>
3. make Gemma easy to follow for embedded software developers - sort of building on items 1 and 2, but making it easy to debug on a lone developer with a simpler cpp toolchain (in my case, for example, I kept getting absorbed into "this HWY library is really cool but how much of the implementation is abstracted out by the library" (as you can understand it's a weird combination of barrier and rabbit hole).

Ask: I'd love to implement a simplified version focussing on the 2B model - but I'm not sure where to start and how to even decouple HWY. The good thing seems to be that there's a version of weights that are not dependent on the datatypes in HWY.

As an aside, the templated implementation is really cool!

from gemma.cpp.

Fair enough about Bazel download time. As one other data point, the Highway Bazel build description is ~500 lines, and ~700 for CMake, and the latter saw about twice as many patches.

For the use case of a simple and educational codebase, isn't that already covered by https://github.com/karpathy/nanoGPT?

As the main author of Highway, I likely have some bias, but it is not clear to me what the barrier is.
In gemma.cc, the main "implementation [that] is abstracted out by the library" is MatVec, which is conceptually simple to describe. Conversely, if we leave out the Highway bits, performance truly falls off a cliff (>10x), so it's not clear to me how useful that would be.

from gemma.cpp.

For me Gemma stood out for two reasons:

1. production / integration readiness
2. did not have tensor flow / other dependencies - this was a a breadth of fresh air from the perspective of managing dependencies etc.

I was under the impression that HWY enabled device scalability and wasn't aware that incorporating Highway led to performance improvements - which seems to seal the deal.
It was also really interesting to see the library was sort of like a "drop in" for computations and assumed this was squarely aimed to support different device architectures in a smart fashion.

I'm not against incorporation Highway, cmake, bazel and understand their relevance. I was a little bit more keen about making the codebase "simple" and could not figure out the impact of HWY other than "device scaling".

Thank you for supporting cmake :D.
One cool thing that I learned was the "fetch content" in CMake. This was first time I came across this implementation, and I have already written out a proposal to this feature in my other project.

from gemma.cpp.

Thanks for sharing your impressions.
Yes, Highway is not just to allow running on any platform; we would already get that from normal C++. What it adds is access to the CPU's vector instructions, plus OpenMP-like multithreading, which we do not reliably get otherwise.

One cool thing that I learned was the "fetch content" in CMake. This was first time I came across this implementation, and I have already written out a proposal to this feature in my other project.

Nice :) Yes, this makes dependencies considerably easier to manage.

from gemma.cpp.

We can now avoid GUnit by building with -DHWY_TEST_STANDALONE=1. We're adding mention of @jart 's Makefile to the readme. Closing, feel free to reopen if you'd like to continue the discussion.

from gemma.cpp.