Probably best not to clobber files with `gguf-split -merge` about llama.cpp HOT 5 OPEN

I am not sure we want to invest on --merge as of now llama_model_loader supports sharded model loading.

from llama.cpp.

I am not sure we want to invest on --merge as of now llama_model_loader supports sharded model loading.

No problem - it's just I'm used to using cat and assumed it would follow a similar syntax rather than expand argv[2] into dbrx-16x12b-instruct-q4_0-00001-of-00010.gguf and arg[3] into dbrx-16x12b-instruct-q4_0-00002-of-00010.gguf like that.

Probably a simple test against argc that n=2 and a warning message if not, would stop people doing the same.

from llama.cpp.

Yes feel free to submit a PR

from llama.cpp.

Yes feel free to submit a PR

I'll have a look on Monday. What is the best way to deal with this:

1. Just check that there are 2 file arguments?
2. Don't clobber files by default and add an --clobber option?
3. Have an interactive "Do you really want to overwrite XXX? (y/n)" option?

(3) can be problematic for running in scripts though, but I could an an -y option to force "yes"?

from llama.cpp.

user@host:~/models$ ls -alh
-rw-rw-r-- 1 user group  43G May  2 18:43 Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00001-of-00002.gguf
-rw-rw-r-- 1 user group  38G Apr 17 19:23 Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00002-of-00002.gguf
user@host:~/models$ gguf-split --merge Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00001-of-00002.gguf Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00002-of-00002.gguf Mixtral-8x22B-Instruct-v0.1.Q4_K_M.gguf
gguf_merge: Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00001-of-00002.gguf -> Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00002-of-00002.gguf
gguf_merge: reading metadata Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00001-of-00002.gguf done
gguf_merge: reading metadata Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00002-of-00002.gguf ...gguf_init_from_file: invalid magic characters ''

gguf_merge:  failed to load input GGUF from Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00001-of-00002.gguf
user@host:~/models$ ls -alh
-rw-rw-r-- 1 user group  43G May  2 18:43 Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00001-of-00002.gguf
-rw-rw-r-- 1 user group    0 May  2 19:43 Mixtral-8x22B-Instruct-v0.1.Q4_K_M-00002-of-00002.gguf

cool, now i get to download that whole 38GB again... no idea why a program would overwrite the file it's using as input...

from llama.cpp.