Code Monkey home page Code Monkey logo

Comments (10)

przemoc avatar przemoc commented on June 8, 2024 2

I have my old Windows 7 laptop with 3rd gen Intel core i7-3520M. Windows 7 is not among supported Windows versions by MSYS2), so I quickly tested building with prebuilt llvm-mingw from Git Bash. I don't have CMake there, so I went with make.

$ export PATH="/d/Tools/llvm-mingw/bin:$PATH"
$ mkdir -p git/github.com/ggerganov
$ cd git/github.com/ggerganov
$ git clone https://github.com/ggerganov/whisper.cpp
$ cd whisper.cpp/
$ mingw32-make
...
I whisper.cpp build info:
I UNAME_S:  MINGW64_NT-6.1-7601
I UNAME_P:  unknown
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -DNDEBUG -std=c11   -fPIC -D_XOPEN_SOURCE=600 -mavx -mf16c -msse3 -mssse3
I CXXFLAGS: -I. -I./examples -O3 -DNDEBUG -std=c++11 -fPIC -D_XOPEN_SOURCE=600 -mavx -mf16c -msse3 -mssse3
I LDFLAGS:
I CC:       clang version 18.1.5 (https://github.com/llvm/llvm-project.git 617a15a9eac96088ae5e9134248d8236e34b91b1)
I CXX:      clang version 18.1.5 (https://github.com/llvm/llvm-project.git 617a15a9eac96088ae5e9134248d8236e34b91b1)
...
$ cd models/
$ ./download-ggml-model.sh base.en
$ cd ../
$ ./main -f samples/jfk.wav
whisper_init_from_file_with_params_no_state: loading model from 'models/ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:      CPU total size =   147.37 MB
whisper_model_load: model size    =  147.37 MB
whisper_init_state: kv self size  =   16.52 MB
whisper_init_state: kv cross size =   18.43 MB
whisper_init_state: compute buffer (conv)   =   16.39 MB
whisper_init_state: compute buffer (encode) =  132.07 MB
whisper_init_state: compute buffer (cross)  =    4.78 MB
whisper_init_state: compute buffer (decode) =   96.48 MB

system_info: n_threads = 4 / 4 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0

main: processing 'samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...


[00:00:00.000 --> 00:00:11.000]   And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

whisper_print_timings:     load time =   315.84 ms
whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:      mel time =    39.78 ms
whisper_print_timings:   sample time =   146.65 ms /   131 runs (    1.12 ms per run)
whisper_print_timings:   encode time =  4231.99 ms /     1 runs ( 4231.99 ms per run)
whisper_print_timings:   decode time =    23.44 ms /     2 runs (   11.72 ms per run)
whisper_print_timings:   batchd time =   628.62 ms /   125 runs (    5.03 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:    total time =  5410.97 ms

I don't see any issues.

Is it possible that you didn't remove your build directory before building using CMake with different settings than before?
Can you paste your whole output, i.e. from building and from running using the latest build with AVX2 and FMA disabled?
(Please do it as text, not image; don't forget to mark the output as code block in GitHub after pasting for better readability.)

from whisper.cpp.

przemoc avatar przemoc commented on June 8, 2024

When using prebuilt binaries or building whisper.cpp using CMake without any tweaking it is assumed that you're using relatively recent CPU, i.e. from last ~10 years. That comes from relying by default on availability of AVX2 and FMA ISA extensions that were introduced in Haswell aka 4th gen Intel core.

You have Ivy Bridge aka 3rd gen Intel core, which is older, so binaries may fail for you.

You can build using CMake with disabling AVX2 and FMA:

cmake -B build -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON
cmake --build build

Or you could use MSYS2 and build using make in one of its environment like UCRT64. ISA extensions availability is detected automatically when using make.

For MSYS2 route, CI build definition shows what packages to install (pacman -S PACKAGE_NAME...):

https://github.com/ggerganov/whisper.cpp/blob/master/.github/workflows/build.yml#L253

from whisper.cpp.

fristhon avatar fristhon commented on June 8, 2024

@przemoc thank you very much for the answer.
yea my CPU is so old! I guessed that it could be the cause of the problem.

I just tried once again with CMake and those parameters you suggested. now it gets stuck on transcribing.

from whisper.cpp.

fristhon avatar fristhon commented on June 8, 2024

Now works!
I followed your setup step by step and used git-bash and llvm-mingw, now output binary main.exe is working fine.
Thanks a lot

Sorry for screen shots, I sent images just to show nothing is happening after command.
About previous build with CMake:
yes I'm sure I cleaned the build directory in advance. actually tried once again and nothing changed it still get stuck after command runs. I think something is wrong with my CMake setup, I can see some errors about not finding pthread, don't know it is important or it is just a warning.

anyway problem solved with your help but I just send CMake logs for further debus. thanks again

cmake -B build -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON

-- Building for: Visual Studio 17 2022
-- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.19045.
-- The C compiler identification is MSVC 19.37.32824.0
-- The CXX compiler identification is MSVC 19.37.32824.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.37.32822/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.36.1.windows.1")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- x86 detected
-- Configuring done
-- Generating done
-- Build files have been written to: C:/MyDev/github/whisper.cpp/build

cmake --build build

MSBuild version 17.7.2+d6990bcfa for .NET Framework

  Checking Build System
  Building Custom Rule C:/MyDev/github/whisper.cpp/examples/CMakeLists.txt
  Building Custom Rule C:/MyDev/github/whisper.cpp/CMakeLists.txt
  ggml.c
  ggml-alloc.c
  ggml-backend.c
  ggml-quants.c
...
C:\MyDev\github\whisper.cpp\ggml-quants.c(552,22): warning C4244: 'initializing': conversion from 'int64_t' to 'const int', possible loss of data [C:\MyDev\github\whisper.cpp\build\whisper.vcxproj]
<LOT OF WARNINGS ABOUT conversion from 'int64_t' to 'const int'>
....
  whisper.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\whisper.dll
  Building Custom Rule C:/MyDev/github/whisper.cpp/examples/bench/CMakeLists.txt
  bench.cpp
  bench.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\bench.exe
  Building Custom Rule C:/MyDev/github/whisper.cpp/examples/CMakeLists.txt
  common.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\common.vcxproj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\common.vcxpr
oj]
  common-ggml.cpp
  grammar-parser.cpp
  Generating Code...
  common.vcxproj -> C:\MyDev\github\whisper.cpp\build\examples\Debug\common.lib
  Building Custom Rule C:/MyDev/github/whisper.cpp/examples/main/CMakeLists.txt
  main.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\main\main.vcxproj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\main\main.vc
xproj]
  main.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\main.exe
  quantize.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\quantize\quantize.
vcxproj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\quantize\qua
ntize.vcxproj]
  quantize.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\quantize.exe
  Building Custom Rule C:/MyDev/github/whisper.cpp/examples/server/CMakeLists.txt
  server.cpp
C:\MyDev\github\whisper.cpp\examples\common.h(188,47): warning C4244: 'initializing': conversion from 'float' to 'int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\server\server.vcxp
roj]
C:\MyDev\github\whisper.cpp\examples\common.h(188,37): warning C4244: 'initializing': conversion from 'float' to 'const int16_t', possible loss of data [C:\MyDev\github\whisper.cpp\build\examples\server\serve
r.vcxproj]
  server.vcxproj -> C:\MyDev\github\whisper.cpp\build\bin\Debug\server.exe
  Building Custom Rule C:/MyDev/github/whisper.cpp/CMakeLists.txt

from whisper.cpp.

przemoc avatar przemoc commented on June 8, 2024

Glad that building using llvm-mingw worked for you. If you use Windows 10, then I would suggest building using one of MSYS2 environments, as they are much more convenient to use. (I used llvm-mingw in previous message only because I was doing it on Windows 7.)

You pasted output from running cmake commands. Can you also paste output from running:
./build/bin/main -f samples/jfk.wav
using binary that was created with -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON options?
Just for the record.

from whisper.cpp.

fristhon avatar fristhon commented on June 8, 2024

Sure I remember to use MSYS2 instead.

this is the output of the binary that created with CMake and -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON
it stops working at last line, I waited about 20 minutes.

whisper_init_from_file_with_params_no_state: loading model from 'C:\MyDev\github\whisper.cpp\models\ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:      CPU total size =   147.37 MB
whisper_model_load: model size    =  147.37 MB
whisper_init_state: kv self size  =   16.52 MB
whisper_init_state: kv cross size =   18.43 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 14.01 MiB
whisper_init_state: compute buffer (conv)   =   16.39 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 124.33 MiB
whisper_init_state: compute buffer (encode) =  132.07 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 2.93 MiB
whisper_init_state: compute buffer (cross)  =    4.78 MB
ggml_gallocr_needs_realloc: graph has different number of nodes
ggml_gallocr_alloc_graph: reallocating buffers automatically
ggml_gallocr_reserve_n: reallocating CPU buffer from size 0.00 MiB to 90.38 MiB
whisper_init_state: compute buffer (decode) =   96.48 MB

system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0

main: processing 'C:\MyDev\github\whisper.cpp\samples\jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...

from whisper.cpp.

przemoc avatar przemoc commented on June 8, 2024

Can you do one more test? Building using cmake with --config Release:

cmake --fresh -B build -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON
cmake --build build --config Release

from whisper.cpp.

fristhon avatar fristhon commented on June 8, 2024

Tested, interesting it works in Release mode
Just for the record it is much slower than the binary that created with mingw and I think whatever the reason is it caused the Debug release get stuck.

I can see these guys are on F16C = 1 | SSE3 = 1 | SSSE3 = 1 in the mingw binary but not in the CMake one. should I pass some options to CMake to make those activate?

CMake binary took about 40 seconds to transcribe sample jfk audio

whisper_init_from_file_with_params_no_state: loading model from 'C:\MyDev\github\whisper.cpp\models\ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:      CPU total size =   147.37 MB
whisper_model_load: model size    =  147.37 MB
whisper_init_state: kv self size  =   16.52 MB
whisper_init_state: kv cross size =   18.43 MB
whisper_init_state: compute buffer (conv)   =   16.39 MB
whisper_init_state: compute buffer (encode) =  132.07 MB
whisper_init_state: compute buffer (cross)  =    4.78 MB
whisper_init_state: compute buffer (decode) =   96.48 MB

system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0

main: processing 'C:\MyDev\github\whisper.cpp\samples\jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...


[00:00:00.000 --> 00:00:11.000]   And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.


whisper_print_timings:     load time =   256.89 ms
whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:      mel time =    24.38 ms
whisper_print_timings:   sample time =   183.19 ms /   131 runs (    1.40 ms per run)
whisper_print_timings:   encode time = 34401.36 ms /     1 runs (34401.36 ms per run)
whisper_print_timings:   decode time =    95.66 ms /     2 runs (   47.83 ms per run)
whisper_print_timings:   batchd time =  5552.72 ms /   125 runs (   44.42 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:    total time = 40524.86 ms

Mingw binary took about 5 seconds to transcribe sample jfk audio

whisper_init_from_file_with_params_no_state: loading model from 'C:\MyDev\github\whisper.cpp\models\ggml-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: ftype         = 1
whisper_model_load: qntvr         = 0
whisper_model_load: type          = 2 (base)
whisper_model_load: adding 1607 extra tokens
whisper_model_load: n_langs       = 99
whisper_model_load:      CPU total size =   147.37 MB
whisper_model_load: model size    =  147.37 MB
whisper_init_state: kv self size  =   16.52 MB
whisper_init_state: kv cross size =   18.43 MB
whisper_init_state: compute buffer (conv)   =   16.39 MB
whisper_init_state: compute buffer (encode) =  132.07 MB
whisper_init_state: compute buffer (cross)  =    4.78 MB
whisper_init_state: compute buffer (decode) =   96.48 MB

system_info: n_threads = 4 / 8 | AVX = 1 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 0 | ARM_FMA = 0 | METAL = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD
 = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | CUDA = 0 | COREML = 0 | OPENVINO = 0

main: processing 'C:\MyDev\github\whisper.cpp\samples\jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, 5 beams + best of 5, lang = en, task = transcribe, timestamps = 1 ...


[00:00:00.000 --> 00:00:11.000]   And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.


whisper_print_timings:     load time =    85.62 ms
whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:      mel time =    27.78 ms
whisper_print_timings:   sample time =   443.54 ms /   131 runs (    3.39 ms per run)
whisper_print_timings:   encode time =  3418.99 ms /     1 runs ( 3418.99 ms per run)
whisper_print_timings:   decode time =    16.51 ms /     2 runs (    8.26 ms per run)
whisper_print_timings:   batchd time =   513.49 ms /   125 runs (    4.11 ms per run)
whisper_print_timings:   prompt time =     0.00 ms /     1 runs (    0.00 ms per run)
whisper_print_timings:    total time =  4570.45 ms

let me know if you needed something else to test

from whisper.cpp.

przemoc avatar przemoc commented on June 8, 2024

Thanks for testing. It seems that either one of this options happen:

  • Debug configuration is much slower than Release configuration (and 20 mins is simply not enough),
  • Debug configuration has some issue leading it to be stuck.

MSVC have limited configurability regarding x86 ISA extensions. I noticed one minor issue in ggml regarding exposing some of them for MSVC, so you may try to do and check new cmake build in Release config (do not forget about -DWHISPER_NO_AVX2=ON -DWHISPER_NO_FMA=ON) using my PR #2128, but other than seeing SSE3 = 1 | SSSE3 = 1 I do not expect any performance improvement (I'll be happy to be wrong on that).

F16C is more tricky, because it came after AVX, but before AVX2, so there is no easy way to figure out if it can be exposed in MSVC (I mean in case when you have AVX available but not AVX2).

from whisper.cpp.

fristhon avatar fristhon commented on June 8, 2024

Okay thanks for the info

from whisper.cpp.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.