Describe the bug
Rule setting is not preseved when job is saved as a Template

To Reproduce
Steps to reproduce the behavior:

1. Save a job as a template
   

2. Review the settings in the template

    {
        "Name":  "XXXXXX",
        "ID":  "1d4d3454-569b-44f3-b80c-8ec4156ec321",
        "Details":  {
                        "attack_mode":  0,
                        "increment":  null,
                        "increment_max":  null,
                        "increment_min":  null,
                        "mask":  "?a?a?a?a?a?a",
                        "rules":  null,
                        "wordlist":  "rockyou2021",
                        "wordlist2":  null,
                        "timeout":  32400,
                        "stats":  ""
                    }
    },

Expected behavior
The rules property of the template should not be null
No logs written to crackq.log

Hey,

I have encountered the following error in the crackq container log:

crackq | Traceback (most recent call last):
crackq | File "/usr/local/bin/rq", line 8, in
crackq | sys.exit(main())
crackq | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 829, in call
crackq | return self.main(*args, **kwargs)
crackq | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 782, in main
crackq | rv = self.invoke(ctx)
crackq | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1259, in invoke
crackq | return _process_result(sub_ctx.command.invoke(sub_ctx))
crackq | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
crackq | return ctx.invoke(self.callback, **ctx.params)
crackq | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 610, in invoke
crackq | return callback(*args, **kwargs)
crackq | File "/usr/local/lib/python3.6/dist-packages/rq/cli/cli.py", line 83, in wrapper
crackq | return ctx.invoke(func, cli_config, *args[1:], **kwargs)
crackq | File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 610, in invoke
crackq | return callback(*args, **kwargs)
crackq | File "/usr/local/lib/python3.6/dist-packages/rq/cli/cli.py", line 220, in worker
crackq | settings = read_config_file(cli_config.config) if cli_config.config else {}
crackq | File "/usr/local/lib/python3.6/dist-packages/rq/cli/helpers.py", line 27, in read_config_file
crackq | settings = importlib.import_module(module)
crackq | File "/usr/lib/python3.6/importlib/init.py", line 126, in import_module
crackq | return _bootstrap._gcd_import(name[level:], package, level)
crackq | File "", line 994, in _gcd_import
crackq | File "", line 971, in _find_and_load
crackq | File "", line 953, in _find_and_load_unlocked
crackq | ModuleNotFoundError: No module named 'rq_settings'

This seems to come up on the:

• docker-compose-opencl.yml
• docker-compose-opencl.wldapserver.yml
• docker-compose.test.yml

This Error doesnt show up on the dev.yml though.
It seems to be the case because the environment var 'PYTHONPATH' is on these leads to /opt/crackq/build/crackq and not /opt/crackq/build/
If u look inside the container the rq and init.py is indeed on the path /opt/crackq/build/

I'll create a pull request to fix this :)

Hi all,

I developed Cloudstomp for a similar reason (machine learning and current user password validation): https://github.com/Fmstrat/cloudstomp

It's main feature is it is built to grab the most affordable AWS GPU spot instance to run hashcat. This logic could be a great addition to CrackQ, and it is already written in Python.

Basically, launch the cli, choose a price/instance, and CloudQ gets launched in AWS.

It also supports statelessness by storing encrypted responses in S3 for if the spot instance goes down.

Thoughts?

Run the following job
Status: failed
Time started: 2020-06-19 15:50:36.651153
Time finished: 2020-06-19 15:50:36.651680
attack_mode: 1
hash_mode: 1000
mask: None
name: Test
rules: []
username: True
wordlist: rockyou
Result:
email_count: 0
notify: true

=== /var/crackq/files/crackq.conf==
[wordlists]
rockyou: /var/crackq/files/wordlists/skull/rockyou.txt

==crack.log===
INFO root 2020-06-19 16:40:41,499 Registering new job_id to current user: 953ece119a95448189eab18458c14ec1
INFO root 2020-06-19 16:40:41,503 Adding task to job queue: 953ece119a95448189eab18458c14ec1
INFO root 2020-06-19 16:40:41,504 API Job 953ece119a95448189eab18458c14ec1 added to queue
16:40:41 default: hc_worker(attack_mode=1, brain=True, crack=<crackq.run_hashcat.Crack object at 0x7f80a2adb780>, hash_file='/var/crackq/logs/953ece119a95448189eab18458c14ec1.hashes', hash_mode=1000, mask=None, mask_file=False, name='Test_Comb_01', outfile='/var/crackq/logs/953ece119a95448189eab18458c14ec1.cracked', pot_path='/var/crackq/logs/crackq.pot', rules=[], session='953ece119a95448189eab18458c14ec1', username=True, wordlist='/var/crackq/files/wordlists/skull/rockyou.txt') (953ece119a95448189eab18458c14ec1)
INFO root 2020-06-19 16:40:41,967 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-19 16:40:41,969 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-19 16:40:42,478 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-19 16:40:42,484 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-19 16:40:42,504 Running hashcat
16:40:42 Cleaning registries for queue: default

bash-4.2$ ls -al /var/crackq/files/wordlists/skull/rockyou.txt
-rw-r--r-- 1 root root 139921497 Mar 5 2018 /var/crackq/files/wordlists/skull/rockyou.txt

At the moment I finish the "NGINX Configuration" section of the guide and am able to execute the sudo docker compose -f docker-compose.nvidia.yml up --build command, everything runs smoothly and I can see all the steps of the docker compose process running.

In the build process these check boxes were ticked:

Network crackq_crackq_net Created;
Container redis Created;
Container crackq Created;
Container nginx Created.

However, at the end of the process this message appears:

Error response from daemon: could not select device driver "nvidia" with capabilities: [[gpu]]

However I do not know why the problem is occurring and can assure the machine is equipped with a Nvidia card as shown by executing command nvidia-smi:

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.171.04             Driver Version: 535.171.04   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce 940MX           Off | 00000000:01:00.0 Off |                  N/A |
| N/A   39C    P8              N/A / 200W |      4MiB /  4096MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      1988      G   /usr/lib/xorg/Xorg                            2MiB |
+---------------------------------------------------------------------------------------+

I can't add more than one user at a time to Crackq without having to restart it. I get the error "Error adding user: undefined". When looking in the logs, I find:

DEBUG    cq_api.py:1868 post 2021-01-26 22:49:53,927 Creating User: test
ERROR    app.py:1892 log_exception 2021-01-26 22:49:54,104 Exception on /api/admin/ [POST]
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 609, in do_execute
    cursor.execute(statement, parameters)
sqlite3.IntegrityError: column id is not unique
...
sqlalchemy.exc.IntegrityError: (sqlite3.IntegrityError) column id is not unique
[SQL: INSERT INTO user (id, active, username, email, last_login_at, last_seen, current_login_at, last_login_ip, current_login_ip, login_count, confirmed_at, is_admin, password) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)]
[parameters: (<memory at 0x7fbbbcff0108>, None, 'test', '[email protected]', None, None, None, None, None, None, None, 0, '{some hash}')]
(Background on this error at: http://sqlalche.me/e/13/gkpj)

Describe the feature
Allow existing templates to be edited or cloned. Sometimes a template needs just a small change, e.g. timeout and running a whole new job with fresh (not in the potfile) hashes can be time consuming.

Documented steps for modifying the on-disk JSON files would be and adequate workaround.

I recently got crackq deployed and have been performing some testing. When I submit a brute-force of an NTLM hash. The job sits in the queue for some time and then goes to a timeout status.

Describe the bug
Installation seem smooth up to point of starting container, did some digging and seems like flask-login was updated and one of function has been removed? https://stackoverflow.com/questions/77215107/importerror-cannot-import-name-url-decode-from-werkzeug-urls

Error:
`Step 16/16 : CMD ./nginx.sh
---> Using cache
---> 8b2b8c5d478a

Successfully built 8b2b8c5d478a
Successfully tagged nginx-crackq:latest
Starting redis ... done
Starting crackq ... done
Starting nginx ... done
Attaching to redis, crackq, nginx
redis | 1:C 07 Dec 2023 14:54:57.564 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see jemalloc/jemalloc#1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis | 1:C 07 Dec 2023 14:54:57.564 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis | 1:C 07 Dec 2023 14:54:57.564 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
redis | 1:C 07 Dec 2023 14:54:57.564 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis | 1:M 07 Dec 2023 14:54:57.564 * monotonic clock: POSIX clock_gettime
redis | 1:M 07 Dec 2023 14:54:57.565 * Running mode=standalone, port=6379.
redis | 1:M 07 Dec 2023 14:54:57.566 * Server initialized
redis | 1:M 07 Dec 2023 14:54:57.566 * Loading RDB produced by version 7.2.3
redis | 1:M 07 Dec 2023 14:54:57.566 * RDB age 211 seconds
redis | 1:M 07 Dec 2023 14:54:57.566 * RDB memory usage when created 0.83 Mb
redis | 1:M 07 Dec 2023 14:54:57.566 * Done loading RDB, keys loaded: 0, keys expired: 0.
redis | 1:M 07 Dec 2023 14:54:57.566 * DB loaded from disk: 0.000 seconds
redis | 1:M 07 Dec 2023 14:54:57.566 * Ready to accept connections tcp
crackq | Processing /opt/crackq/build
crackq | Preparing metadata (setup.py) ... done
crackq | Building wheels for collected packages: crackq
crackq | Building wheel for crackq (setup.py) ... done
crackq | Created wheel for crackq: filename=crackq-0.1.0-py3-none-any.whl size=38858 sha256=f9021f46111fb6f76493bd337b0e5c647641ebdb17f228b55595024bb0c0005d
crackq | Stored in directory: /tmp/pip-ephem-wheel-cache-tql01q8z/wheels/ad/45/07/34e69083a16bf36cd3c7ab183548cbf1d5f035b166b1c0fdb5
crackq | Successfully built crackq
crackq | Installing collected packages: crackq
nginx | Starting authentication failure monitor: fail2ban.
crackq | WARNING: The script crackq is installed in '/home/crackq/.local/bin' which is not on PATH.
crackq | Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
crackq | Successfully installed crackq-0.1.0
crackq | Traceback (most recent call last):
crackq | File "", line 1, in
crackq | File "/opt/crackq/build/crackq/init.py", line 6, in
crackq | from flask_login import (
crackq | File "/usr/local/lib/python3.8/dist-packages/flask_login/init.py", line 12, in
crackq | from .login_manager import LoginManager
crackq | File "/usr/local/lib/python3.8/dist-packages/flask_login/login_manager.py", line 33, in
crackq | from .utils import _create_identifier
crackq | File "/usr/local/lib/python3.8/dist-packages/flask_login/utils.py", line 14, in
crackq | from werkzeug.urls import url_decode
crackq | ImportError: cannot import name 'url_decode' from 'werkzeug.urls' (/usr/local/lib/python3.8/dist-packages/werkzeug/urls.py)
crackq exited with code 1`

I was running a brute-force on a 9-char NTLM hash, mainly because I could and the cracker was not need for other work at the time. According to CrackQ it was going to take 20 days to complete, which was fine. I check the task this morning and it had failed with the message

Task exceeded maximum timeout value (1209600 seconds)

Where is this timeout controlled?

hi,

i tried to install it on fresh Ubuntu 18.04 server but no success

Successfully built 3f4f6d78db26
Successfully tagged nginx-crackq:latest
Creating redis ... done
Creating crackq ... error

ERROR: for crackq Cannot create container for service crackq: Unknown runtime specified nvidia

ERROR: for crackq Cannot create container for service crackq: Unknown runtime specified nvidia
ERROR: Encountered errors while bringing up the project.

I want to do some testing of the "add_inapp_auth_admin" branch but I do not have spare GPU hardware to use. Any suggestions on how to get the crackq docker image running without GPU hardware?

What are the restrictions on passwords? I'd generated a password with 1Password which I'll post below (it is now used for nothing and is no longer a secret). I set this password for the initial admin user with the command given in the wiki.
sudo docker exec -it crackq /usr/bin/python3 /opt/crackq/build/crackq/add_admuser.py
What about this password makes it invalid, and could that be put somewhere in documentation?
a1m]@W9J1]q.=gcILp(Q,twI5qj8M@mY

Error received:
DEBUG cq_api.py:580 post 2021-01-26 22:07:32,503 Validation error: {'password': ['Invalid input for this field.']}

Describe the bug
Hey, When creating a new job using 0.1.2 and Ubuntu + NVIDIA the job is scheduled but fails immediately afterwards. A look at the console error message lists the Speed check failed error entry. This affects both with/without brain enabled. Ive tried launching new and previously cracked jobs. The issue doesn't seem to affect opencl, which works fine. So Im thinking its potentially an issue related to hashcat/nvidia. The issue mostly seems to come from run_hashcat.py i think.

Version

root@localhost:/opt/crackq-0.1.2# uname -a
Linux localhost 5.15.0-91-generic #101~20.04.1-Ubuntu SMP Thu Nov 16 14:22:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Running instances

root@localhost:/opt/crackq-0.1.2/build# docker ps
CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS          PORTS                                           NAMES
50c58e6fa8af   nginx-crackq    "/docker-entrypoint.…"   28 seconds ago   Up 27 seconds   80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   nginx
34c0de1d67a7   nvidia-ubuntu   "/opt/nvidia/nvidia_…"   29 seconds ago   Up 28 seconds   6379/tcp, 8081/tcp, 127.0.0.1:8080->8080/tcp    crackq
e734a0eb4655   redis:latest    "docker-entrypoint.s…"   29 seconds ago   Up 28 seconds   127.0.0.1:6379->6379/tcp

Nvidia SMI

root@localhost:/opt/crackq-0.1.2/build# nvidia-smi
Wed Jan 17 11:27:05 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.223.02   Driver Version: 470.223.02   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 50%   31C    P8    14W / 250W |     14MiB /  4043MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce ...  Off  | 00000000:02:00.0 Off |                  N/A |
| 51%   28C    P8    14W / 250W |      5MiB /  4043MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA GeForce ...  Off  | 00000000:0B:00.0 Off |                  N/A |
| 50%   25C    P8    13W / 250W |      5MiB /  4043MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1083      G   /usr/lib/xorg/Xorg                  7MiB |
|    0   N/A  N/A      1693      G   /usr/bin/gnome-shell                2MiB |
|    1   N/A  N/A      1083      G   /usr/lib/xorg/Xorg                  3MiB |
|    2   N/A  N/A      1083      G   /usr/lib/xorg/Xorg                  3MiB |
+-----------------------------------------------------------------------------+

Crash Error - Brain Enabled

root@localhost:/opt/crackq-0.1.2/build# screen -r PW
crackq    | 00:30:20 speed_check: crackq.run_hashcat.show_speed(attack_mode=0, brain=True, hash_file='/var/crackq/logs/4108b8eab7ef475093bd9f8e4190a9a4.hashes', hash_mode=100, mask='?a?a?a?a?a?a', name='Example4', pot_path='/var/crackq/logs/crackq.pot', session='4108b8eab7ef475093bd9f8e4190a9a4', speed_session='4108b8eab7ef475093bd9f8e4190a9a4_speed', username=False, wordlist2=None, wordlist='/var/crackq/files/rockyou.txt.gz') (4108b8eab7ef475093bd9f8e4190a9a4_speed)
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:21,096 Reading from config file /var/crackq/files/crackq.conf
crackq    | [nltk_data] Downloading package wordnet to /home/crackq/nltk_data...
crackq    | [nltk_data]   Package wordnet is already up-to-date!
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:21,272 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:21,307 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     schemas.py:1260 include_schema 2024-01-17 00:30:21,557 Resource 'XMLSchema.xsd' is already loaded
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:23,147 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     run_hashcat.py:116 runner 2024-01-17 00:30:23,168 Running hashcat
crackq    | INFO     crackqueue.py:47 q_add 2024-01-17 00:30:23,513 Adding task to job queue: 4108b8eab7ef475093bd9f8e4190a9a4
crackq    | 00:30:23 default: crackq.run_hashcat.hc_worker(attack_mode=0, brain=True, hash_file='/var/crackq/logs/4108b8eab7ef475093bd9f8e4190a9a4.hashes', hash_mode=100, increment=False, increment_max=None, increment_min=None, mask=None, mask_file=False, name='Example4', outfile='/var/crackq/logs/4108b8eab7ef475093bd9f8e4190a9a4.cracked', pot_path='/var/crackq/logs/crackq.pot', potcheck=False, restore=0, rules=[], session='4108b8eab7ef475093bd9f8e4190a9a4', username=False, wordlist2=None, wordlist='/var/crackq/files/rockyou.txt.gz') (4108b8eab7ef475093bd9f8e4190a9a4)
redis     | 1:M 17 Jan 2024 00:30:23.528 * 100 changes in 300 seconds. Saving...
redis     | 1:M 17 Jan 2024 00:30:23.529 * Background saving started by pid 21
nginx     | 10.255.255.10 - - [17/Jan/2024:00:30:23 +0000] "POST /api/add HTTP/1.1" 202 32 "https://192.168.1.1/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0" "-"
redis     | 21:C 17 Jan 2024 00:30:23.541 * DB saved on disk
redis     | 21:C 17 Jan 2024 00:30:23.543 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
redis     | 1:M 17 Jan 2024 00:30:23.630 * Background saving terminated with success
nginx     | 10.255.255.10 - - [17/Jan/2024:00:30:23 +0000] "GET /api/queuing/all HTTP/1.1" 200 375 "https://192.168.1.1/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0" "-"
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:24,141 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     run_hashcat.py:116 runner 2024-01-17 00:30:24,171 Running hashcat
crackq    | [nltk_data] Downloading package wordnet to /home/crackq/nltk_data...
crackq    | [nltk_data]   Package wordnet is already up-to-date!
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:24,325 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:24,359 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     schemas.py:1260 include_schema 2024-01-17 00:30:24,610 Resource 'XMLSchema.xsd' is already loaded
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:30:26,198 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     run_hashcat.py:116 runner 2024-01-17 00:30:26,215 Running hashcat
crackq    | ERROR    run_hashcat.py:188 runner 2024-01-17 00:30:26,253 Speed check failed: RuntimeError
crackq    |
crackq    | The above exception was the direct cause of the following exception:
crackq    |
crackq    | Traceback (most recent call last):
crackq    |   File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 1418, in perform_job
crackq    |     rv = job.perform()
crackq    |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 1222, in perform
crackq    |     self._result = self._execute()
crackq    |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 1259, in _execute
crackq    |     result = self.func(*self.args, **self.kwargs)
crackq    |   File "/opt/crackq/build/crackq/run_hashcat.py", line 988, in show_speed
crackq    |     hcat = runner(hash_file=hash_file, mask=mask,
crackq    |   File "/opt/crackq/build/crackq/run_hashcat.py", line 159, in runner
crackq    |     hc.hashcat_session_execute()
crackq    | SystemError: <method 'hashcat_session_execute' of 'pyhashcat.hashcat' objects> returned a result with an error set

Crash Error - Brain Disabled

crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:32:00,424 Reading from config file /var/crackq/files/crackq.conf
crackq    | [nltk_data] Downloading package wordnet to /home/crackq/nltk_data...
crackq    | INFO     run_hashcat.py:116 runner 2024-01-17 00:32:00,468 Running hashcat
crackq    | [nltk_data]   Package wordnet is already up-to-date!
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:32:00,610 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:32:00,646 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     schemas.py:1260 include_schema 2024-01-17 00:32:00,903 Resource 'XMLSchema.xsd' is already loaded
crackq    | INFO     conf.py:18 hc_conf 2024-01-17 00:32:02,478 Reading from config file /var/crackq/files/crackq.conf
crackq    | INFO     run_hashcat.py:116 runner 2024-01-17 00:32:02,498 Running hashcat
crackq    | ERROR    run_hashcat.py:188 runner 2024-01-17 00:32:02,537 Speed check failed: RuntimeError
crackq    |
crackq    | The above exception was the direct cause of the following exception:
crackq    |
crackq    | Traceback (most recent call last):
crackq    |   File "/usr/local/lib/python3.8/dist-packages/rq/worker.py", line 1418, in perform_job
crackq    |     rv = job.perform()
crackq    |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 1222, in perform
crackq    |     self._result = self._execute()
crackq    |   File "/usr/local/lib/python3.8/dist-packages/rq/job.py", line 1259, in _execute
crackq    |     result = self.func(*self.args, **self.kwargs)
crackq    |   File "/opt/crackq/build/crackq/run_hashcat.py", line 988, in show_speed
crackq    |     hcat = runner(hash_file=hash_file, mask=mask,
crackq    |   File "/opt/crackq/build/crackq/run_hashcat.py", line 159, in runner
crackq    |     hc.hashcat_session_execute()
crackq    | SystemError: <method 'hashcat_session_execute' of 'pyhashcat.hashcat' objects> returned a result with an error set

To Reproduce
Steps to reproduce the behavior:

1. Login to web and queue new job. [Note ive tried both new and previously cracked jobs]
2. Enable/Disable brain [error message is the same]
3. Submit Request
4. Job scheduled
5. See error on back end/ See error on failed jobs.

Expected behavior
I would really like it to complete the job successfully -even a temp solution.

Screenshots
Error message in the console is listed as:

Additional context
This was an upgrade from 0.1.0 -> 0.1.1 -> 0.1.2. 0.1.1 Never worked for me due to the library issues.

After completed a crack job it was placed in "Failed Jobs" with the message of notify

I am trying to test a hash that previously worked but now I get a "No hashes loaded" message.

==Logs==
INFO root 2020-06-15 14:42:01,496 Registering new job_id to current user: 24c7e62890324e9c816ac98bb4db5b66
INFO root 2020-06-15 14:42:01,500 Adding task to job queue: 24c7e62890324e9c816ac98bb4db5b66
INFO root 2020-06-15 14:42:01,501 API Job 24c7e62890324e9c816ac98bb4db5b66 added to queue
14:42:01 default: hc_worker(attack_mode=3, brain=True, crack=<crackq.run_hashcat.Crack object at 0x7f2d5de6fc88>, hash_file='/var/crackq/logs/24c7e62890324e9c816ac98bb4db5b66.hashes', hash_mode=1000, mask='?2?2?2?2?2?2?2?2', mask_file=False, name='TestJob6', outfile='/var/crackq/logs/24c7e62890324e9c816ac98bb4db5b66.cracked', pot_path='/var/crackq/logs/crackq.pot', rules=[], session='24c7e62890324e9c816ac98bb4db5b66', username=True, wordlist=None) (24c7e62890324e9c816ac98bb4db5b66)
INFO root 2020-06-15 14:42:01,856 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 14:42:01,858 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 14:42:02,470 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 14:42:02,476 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 14:42:02,493 Running hashcat
INFO root 2020-06-15 14:42:07,501 Running hashcat
WARNING root 2020-06-15 14:42:13,880 Callback Triggered: WARNING
WARNING root 2020-06-15 14:42:13,881 Failed to parse hashes using the 'native hashcat' format.
WARNING root 2020-06-15 14:42:13,883 24c7e62890324e9c816ac98bb4db5b66: Failed to parse hashes using the 'native hashcat' format.

After submitting a job, i get Status update failure in the logs until I click "Pause/Stop" on the UI. Then it shows completed.

==Logs===
INFO root 2020-06-15 17:47:02,050 Registering new job_id to current user: 529fd0c07a4f429fa39fe529376a275f
INFO root 2020-06-15 17:47:02,053 Adding task to job queue: 529fd0c07a4f429fa39fe529376a275f
INFO root 2020-06-15 17:47:02,055 API Job 529fd0c07a4f429fa39fe529376a275f added to queue
17:47:02 default: hc_worker(attack_mode=3, brain=True, crack=<crackq.run_hashcat.Crack object at 0x7fc0c11ffe80>, hash_file='/var/crackq/logs/529fd0c07a4f429fa39fe529376a275f.hashes', hash_mode=1000, mask='?2?2?2?2?2?2?2?2?2', mask_file=False, name='TestJob10', outfile='/var/crackq/logs/529fd0c07a4f429fa39fe529376a275f.cracked', pot_path='/var/crackq/logs/crackq.pot', rules=[], session='529fd0c07a4f429fa39fe529376a275f', username=False, wordlist=None) (529fd0c07a4f429fa39fe529376a275f)
INFO root 2020-06-15 17:47:02,538 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 17:47:02,540 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 17:47:03,060 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 17:47:03,066 Reading from config file /var/crackq/files/crackq.conf
INFO root 2020-06-15 17:47:03,087 Running hashcat
INFO root 2020-06-15 17:47:08,093 Running hashcat
WARNING root 2020-06-15 17:47:34,794 Status update failure
WARNING root 2020-06-15 17:47:44,804 Status update failure
WARNING root 2020-06-15 17:47:54,816 Status update failure
WARNING root 2020-06-15 17:48:04,827 Status update failure
INFO root 2020-06-15 17:48:09,784 Stopping job: 529fd0c07a4f429fa39fe529376a275f
WARNING root 2020-06-15 17:48:14,838 Status update failure
INFO root 2020-06-15 17:48:14,839 Stopping Job: 529fd0c07a4f429fa39fe529376a275f

Hi,

Do you have some integration with Kubernetes?

Most of the design patterns could be solved by using GPUs on kubernetes worker nodes (with labels).

And kubernetes jobs could be spawned.

Just thinking loud here...

Describe the bug
Cannot add user ID via UI with SQL auth

To Reproduce
Steps to reproduce the behavior:

Expected behavior
SQL-based user ID is created

Debug output

ERROR    app.py:1892 log_exception 2023-07-31 19:51:49,207 Exception on /api/admin/ [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 163, in dispatch_request
    return meth(*args, **kwargs)
  File "/opt/crackq/build/crackq/cq_api.py", line 390, in wrap
    return func(args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/flask_login/utils.py", line 272, in decorated_view
    return func(*args, **kwargs)
TypeError: post() missing 1 required positional argument: 'user_id'

Prerequisites

Enable debugging:
sudo docker exec -it crackq /bin/sed -i 's/INFO/DEBUG/g' /opt/crackq/build/crackq/log_config.ini
Unable to do as the crackq container fails to start.

Prior to reaching this point I also had to tweak the Nvidia + Ubuntu Dockerfile as Python3.7 and Python3.7-Dev are not available on Ubuntu 20.04 and would throw an error when running ./install.sh /docker/nvidia/ubuntu . I changed these to Python3.8 and Python3.8-Dev which fixed the issue.

I needed to uncomment ENV DEBIAN_FRONTEND noninteractive otherwise install.sh would hang on setting up a timezone for tzdata.

I also needed to change FROM nvidia/cuda:runtime-ubuntu20.04 to specify a version number. Using the most recent, I went with FROM nvidia/cuda:11.6.0-runtime-ubuntu20.04.

# Update & install packages for installing hashcat
RUN apt-get update && \
    apt-get install -y wget p7zip gcc g++ make build-essential git libcurl4-openssl-dev libssl-dev zlib1g-dev python3.8 \
    python3.8-dev python3-pip libldap2-dev libsasl2-dev libssl-dev xmlsec1 libxmlsec1-openssl

Describe the bug

Running the sudo docker-compose -f docker-compose.nvidia.yml up --build command, Crackq is unable to start. The error thrown is after a long traceback through python imports is the following:
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.8/dist-packages/markupsage/__init__.py

See picture below for the entire traceback.

To Reproduce

Steps to reproduce the behavior:

1. Pull the Crackq Repo

2. Following the readme, install the latest Docker and Docker-compose

# Install Docker
sudo apt-get update
sudo apt-get install \
  ca-certificates \
  curl \
  gnupg \
  lsb-release

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

# Install Docker-Compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o 
/usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

#Verify Install
docker-compose --version

3. Download the latest Nvidia Server Drivers.

This may vary depending on GPU's. I'm using 7 ZOTAC 1080ti's and I installed the recommended drivers shown by running the following.

# Check your GPU/Drivers
ubuntu-drivers devices

# If ubuntu-drivers isn't installed, install the following and run the previous command again:
sudo apt install ubuntu-drivers-common

# Based off the output, install the recommended nvidia drivers
# Using a server version prevents the GUI from being installed
sudo apt install nvidia-<driver>-server 

# Reboot the system
sudo reboot

# Check Driver Change and system recognizes the GPU's
nvidia-smi

#If the previous command fails, in my case it was due to Nvidia-persistenced not running. I fixed it by running the following:
sudo -i 
nvida-smi -pm 1
exit

# Confirm by running the previous command again and receiving the expected output
nvidia-smi

4. Install Nvidia Docker

This part took some trial and error as the Crackq readme says the following:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
sudo apt-get install nvidia-container-runtime

However per the Nvidia-Docker Docs, they recommend installing nvidia-docker2 which resulted in some problems for me. Instead I followed the Crackq readme and did the following after installing nvidia-container-runtime per Nvidia-Container-Runtime.

# Create SystemD drop-in file 
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo tee /etc/systemd/system/docker.service.d/override.conf <<EOF
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --host=fd:// --add-runtime=nvidia=/usr/bin/nvidia-container-runtime
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

# Verify GPU's can be recognized by a container
sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

5. Run Install.sh

sudo ./install.sh docker/nvidia/ubuntu

6. Configuration

I ran through the configuration portion. Did everything documented in the Crackq Configuration

# Gen Secret Key
python3 -c 'import secrets; print(secrets.token_urlsafe())'

# Copy to crackq.conf
[app]
SECRET_KEY: secret_key_generated_above

# Didn't add any additional wordlists, just used the rockyou.txt that comes with install
# Moved the config file to proper location
sudo mv crackq.conf /var/crackq/files/
sudo chown crackq:crackq /var/crackq/files/crackq.conf
sudo chmod 640 /var/crackq/files/crackq.conf

I skipped any type of authentication setup and modified the custom nginx config and added my own certs to the proper directory

sudo cp ./cfg/crackq_nginx.conf /var/crackq/files/nginx/
sudo cp cert.pem /var/crackq/files/nginx/conf.d/certificate.pem
sudo cp priv.pem /var/crackq/files/nginx/conf.d/private.pem

Expected behavior

After all this I should be able to run the application with either:
sudo docker-compose -f docker-compose.nvidia.yml up --build

Or
sudo docker-compose -f docker-compose.nvidia.yml up -d

Debug output

This is the output I receive from the docker logs as the containers were starting:

Additional context

To add, I was able to temporarily workaround this by modifying the Nvidia/Ubuntu/Dockerfile again and including this command
RUN pip3 install markupsafe==2.0.1 per this issue forum.

However this then led to an issue where Pypal also failed to import. Unfortunately I don't have the output from the docker logs, however the logs threw a similar traceback stack as the one above listing that it was unable to locate pypal.

If I missed anything please let me know.

Describe the bug
When using the runtime flavor of NVIDIA images,
https://github.com/f0cker/crackq/blob/675a5b62191cd999b3f3a5304138ef021800e156/docker/nvidia/ubuntu/Dockerfile#L1C2-L1C2

hashcat does not recognize NVIDIA T4 GPUs, even though nvidia-smi does.

To Reproduce
Steps to reproduce the behavior:

1. Build containers.
2. Open shell in crackq
3. Run nvidia-smi

sudo docker exec -it crackq /bin/bash

crackq@crackq:/opt/crackq/build$ nvidia-smi
Tue Aug  8 13:12:18 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.125.06   Driver Version: 525.125.06   CUDA Version: 12.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
| N/A   66C    P0    30W /  70W |      2MiB / 15360MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla T4            Off  | 00000000:00:05.0 Off |                    0 |
| N/A   67C    P0    28W /  70W |      2MiB / 15360MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  Tesla T4            Off  | 00000000:00:06.0 Off |                    0 |
| N/A   67C    P0    30W /  70W |      2MiB / 15360MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  Tesla T4            Off  | 00000000:00:07.0 Off |                    0 |
| N/A   67C    P0    30W /  70W |      2MiB / 15360MiB |      8%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

4. Run haschat -I or a benchmark.

clGetPlatformIDs(): CL_PLATFORM_NOT_FOUND_KHR

ATTENTION! No OpenCL-compatible or CUDA-compatible platform found.

You are probably missing the OpenCL or CUDA runtime installation.

Expected behavior
Hashcat recognizes CUDA-compatible GPUs.

Additional context
This seems to work if you use the devel flavor of image, e.g.
FROM nvidia/cuda:12.2.0-devel-ubuntu20.04

Per NVIDIA:

Three flavors of images are provided:

base: Includes the CUDA runtime (cudart)
runtime: Builds on the base and includes the [CUDA math libraries](https://developer.nvidia.com/gpu-accelerated-libraries), and [NCCL](https://developer.nvidia.com/nccl). A runtime image that also includes [cuDNN](https://developer.nvidia.com/cudnn) is available.
devel: Builds on the runtime and includes headers, development tools for building CUDA images. These images are particularly useful for multi-stage builds.

Hi,

You guys stated "We're talking hundreds of billions of guesses per second in many cases."
https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/introducing-password-cracking-manager-crackq/

Only that?

For each password, "Multi One Password" tool uses by default a random 248 long characters salt that contains (uppercase\lowercase\digit) characters only and a random 128 long characters salt that contains hex symbols only!

So, here is a quick demonstration on how "hundreds of billions of guesses per second" is really nothing:

999 000 000 000 guesses in 1 second! (999 Billions!)

100000000 seconds = 3.1709791984 years

16^128 (All the 128 possible combinations between the 16 hex symbols!)
62^248 (All the 248 possible combinations between (Uppercase\Lowercase\Digit) chars!)

999 000 000 000 x 100000000 = below

99 900 000 000 000 000 000 guesses in 3 years!

16^128 = 1.340781e+154 = below

13407810000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Even in 3 years you are far far away to guess all the (16^128) possibilities alone!

Well, for (62^248) which is "Infinity" or "Error" in some calculators, I wish you Good Luck!

RTX 2080 Ti GPU (Year 2018, $999 US Dollars)

10 x RTX 2080 Ti GPU ($9 990 US Dollars) can guess 25054.2 MH/s SHA2-512 ~= 25 Billions per second!
https://www.onlinehashcrack.com/tools-benchmark-hashcat-gtx-1080-ti-1070-ti.php#

It means that 100 x RTX 2080 Ti Gpu ($99 900 US Dollars) can guess 250 Billions! (Still less than 999 Billions!)

By the way, I would like you guys to crack the below hash string generated in "Multi One Password" tool!

58530dfe0bc9c45fc074f56bb05a4d991cc9aa38144774b3014277ae353d7fa0a18aa6b6a33bf7b76ca313c3aa3f8d0e9913e7fcefab3dd3f0461a68a40bb587

The password is 10 characters long and contains only lowercase chars!

(Note) From the demonstration above, it would take you far and far more than 3 years to find the original password!

for aaaaaaaaaa + (16^128 + 62^248) = An Eternity
+
for aaaaaaaaab + (16^128 + 62^248) = An Eternity
+
for aaaaaaaaac + (16^128 + 62^248) = An Eternity
+...
for zzzzzzzzzz + (16^128 + 62^248) = An Eternity

Even if the "Random Code (Salt)" was made available to you guys, it would still be almost impossible for you to crack the above SHA512 hashed string because "Multi One Password" tool uses by default 15000 to 16000 iterations!

15000 iterations requires at least 1 second for the "Final Password" to be generated! (For average computers!)

26^10 (All the 10 possible combinations between the 26 Lowercase characters!)

26^10 = 1.411671e+14 = 141167100000000 total guesses

141167100000000 x 1 second = 141167100000000 seconds = 4476379 years

So, it would take you guys 4476379 years to crack the SHA512 hashed string above even if the "Random Code (Salt)" was made available to you! (well for 25000 or higher iterations even an eternity wouldn't be enough!)

[Note]:

for 15000 iterations, a super server can generate "Final Passwords" in 1 milliseconds = 0.001 seconds
141167100000000 x 0.001 second = 141167100000 seconds = 4476 years

for 15000 iterations, a super server can generate "Final Passwords" in 0.1 milliseconds = 0.0001 seconds
141167100000000 x 0.0001 second = 14116710000 seconds = 447 years

and son on ...!

Since 18 Feb 2020, TeraHash(Click here) still didn't crack the bellow 1 character long SHA512 hashed password (which is a lowercase letter) with its $1.4 Million configuration of 448 x GeForce RTX 2080 GPUs:

6d78101f3965681a61ab72365de3f9052d6da65f5a42ea6d4a6e68f02b81d32825388837ee9d61e5314cfca90d5638316c1465634fcad42c8cf1a744cc924947

That's really embarrassing!

I'm getting an error when attempting to generate a report from a completed job. Below is a screenshot as well as /var/crackq/logs/crackq.log at the time of the error.

Everything else works great - hashes crack properly, etc.

Thanks!

The Discord invite link in the README has expired.

sudo ./install.sh docker/nvidia/centos

pyhashcat.c:910:16: error: redefinition of 'struct usage_sort'
typedef struct usage_sort
^
In file included from pyhashcat.c:21:0:
hashcat/include/usage.h:12:16: note: originally defined here
typedef struct usage_sort
^
pyhashcat.c:916:3: error: conflicting types for 'usage_sort_t'
} usage_sort_t;
^
In file included from pyhashcat.c:21:0:
hashcat/include/usage.h:18:3: note: previous declaration of 'usage_sort_t' was here
} usage_sort_t;
^
pyhashcat.c:918:12: error: static declaration of 'sort_by_usage' follows non-static declaration
static int sort_by_usage (const void *p1, const void *p2)
^
In file included from pyhashcat.c:21:0:
hashcat/include/usage.h:22:5: note: previous declaration of 'sort_by_usage' was here
int sort_by_usage (const void *p1, const void *p2);
^
error: command 'gcc' failed with exit status 1
running install
running build
running build_ext
building 'pyhashcat' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Ihashcat/include -Ihashcat/deps/OpenCL-Headers -Ihashcat/OpenCL -Ihashcat -Ihashcat/deps/zlib -Ihashcat/deps/LZMA-SDK/C -I/usr/include/python3.6m -c pyhashcat.c -o build/temp.linux-x86_64-3.6/pyhashcat.o -std=c99 -DWITH_BRAIN -Wimplicit-function-declaration
pyhashcat.c:910:16: error: redefinition of 'struct usage_sort'
typedef struct usage_sort
^
In file included from pyhashcat.c:21:0:
hashcat/include/usage.h:12:16: note: originally defined here
typedef struct usage_sort
^
pyhashcat.c:916:3: error: conflicting types for 'usage_sort_t'
} usage_sort_t;
^
In file included from pyhashcat.c:21:0:
hashcat/include/usage.h:18:3: note: previous declaration of 'usage_sort_t' was here
} usage_sort_t;
^
pyhashcat.c:918:12: error: static declaration of 'sort_by_usage' follows non-static declaration
static int sort_by_usage (const void *p1, const void *p2)
^
In file included from pyhashcat.c:21:0:
hashcat/include/usage.h:22:5: note: previous declaration of 'sort_by_usage' was here
int sort_by_usage (const void *p1, const void *p2);
^
error: command 'gcc' failed with exit status 1

I love the design and feel of this application. I have a couple of old servers that are not being actively used. Is it possible to run the service in a way that distributes the work load across multiple physical servers?

Looking at the help the following are char mask that can be used
?1 = ?l?d, ?2 = ?l?d?u, ?3 = ?l?d?s but I am looking for a mask for ?l?u?d?s
Is there a way to define other masks?