makeasnek / findthemag2 Goto Github PK

The issue I have with editing/modifying config.py is that I then have to deal with merging my custom settings anytime I download or pull the latest version from the repository. Instead of having users directly edit config.py, I propose treating that file as a "default settings" file, and then having the user create another file (ex. config-local.py) where all site specific customizations are put.
Main.py would then source the config.py file and then, if it is present, source config-local.py. This would allow the user to store their custom settings and not have merge issues whenever a new version of FTM is downloaded/pulled.

This does not stop the user from editing config.py if that is what they want to do. At the same time, if they just want the defaults they leave config.py and do not create config-local.py. When they want a custom setting they can copy the relevant setting out of config.py into config-local.py and change the value.

This will also make it easier if the project eventually gets and installer and installs to the standard locations, as the config-local.py could exist in the users home directory.

I'd like to request that adding timestamps to all log messages be considered.

Currently without date/timestamps it is hard to relate entries in the debug.log file to entries in the BOINC event log. Also when just looking at the debug.log file it is difficult to track changes over days.

It would also be useful if on startup the very first thing FTM did is log a startup entry which includes a separator along with date/time and the version number.

I thing this would make parsing and monitoring the log much easier.

When using Python 3.12, it appears to trigger syntax errors on first running the script. However the script still continues as normal following that and no other noticeable problems occur. I have tested this in both the 3.0 release and the latest commit at the time of posting 8772d49

C:\Users\david\Desktop\FindTheMag2-main>python main.py
C:\Users\david\Desktop\FindTheMag2-main\main.py:145: SyntaxWarning: invalid escape sequence '\d'
  '("low24h":)(\d*.\d*)',
C:\Users\david\Desktop\FindTheMag2-main\main.py:149: SyntaxWarning: invalid escape sequence '\d'
  '(data-field="regularMarketPrice" data-trend="none" data-pricehint="\d" value=")(\d*\.\d*)',
C:\Users\david\Desktop\FindTheMag2-main\main.py:154: SyntaxWarning: invalid escape sequence '\$'
  '(data-coin-id="243" data-coin-symbol="grc" data-target="price.price">\$)(\d*\.\d*)(</span>)',
C:\Users\david\Desktop\FindTheMag2-main\main.py:271: SyntaxWarning: invalid escape sequence '\P'
  BOINC_DATA_DIR = "C:\ProgramData\BOINC\\"
C:\Users\david\Desktop\FindTheMag2-main\main.py:281: SyntaxWarning: invalid escape sequence '\R'
  Path.home(), "AppData\Roaming\GridcoinResearch\\"
C:\Users\david\Desktop\FindTheMag2-main\main.py:1021: SyntaxWarning: invalid escape sequence '\s'
  cleaned_response = re.sub("\s*", "", allow_response)
C:\Users\david\Desktop\FindTheMag2-main\main.py:1939: SyntaxWarning: invalid escape sequence '\ '
  print("""
C:\Users\david\Desktop\FindTheMag2-main\main.py:4534: SyntaxWarning: invalid escape sequence '\|'
  """

Error message:

Press enter key to start controlling BOINC. Press Ctrl+C to quit

Starting control of BOINC...
Traceback (most recent call last):
  File "/home/piadm/FTM2/FindTheMag2/main.py", line 2620, in <module>
    loop.run_until_complete(prefs_check(rpc_client))
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/piadm/FTM2/FindTheMag2/main.py", line 1518, in prefs_check
    max_gb=int(float(parsed.get('disk_max_used_gb',0)))
AttributeError: 'bool' object has no attribute 'get'

• Running on a Raspberry Pi: 64bit OS
• BOINC installed and running
• No Gridcoin Wallet installed
• Pulled latest FTM2 that included the security fix
• Same install procedure works on another RPi (installed that one first and followed the same directions to install this one)

Global prefs file

<global_preferences>

    <host_specific/>
    <source_project>http://bam.boincstats.com/</source_project>
    <mod_time>1668647832</mod_time>
    <run_if_user_active>1</run_if_user_active>
    <run_gpu_if_user_active>1</run_gpu_if_user_active>
    <suspend_if_no_recent_input>0</suspend_if_no_recent_input>
    <cpu_scheduling_period_minutes>60</cpu_scheduling_period_minutes>
    <idle_time_to_run>3</idle_time_to_run>
    <leave_apps_in_memory/>
    <work_buf_min_days>0.1</work_buf_min_days>
    <work_buf_additional_days>1</work_buf_additional_days>
    <max_cpus>20</max_cpus>
    <max_ncpus_pct>100</max_ncpus_pct>
    <suspend_cpu_usage>0</suspend_cpu_usage>
    <cpu_usage_limit>100</cpu_usage_limit>
    <disk_interval>60</disk_interval>
    <disk_max_used_gb>100</disk_max_used_gb>
    <disk_max_used_pct>50</disk_max_used_pct>
    <disk_min_free_gb>1</disk_min_free_gb>
    <vm_max_used_pct>75</vm_max_used_pct>
    <ram_max_used_busy_pct>60</ram_max_used_busy_pct>
    <ram_max_used_idle_pct>90</ram_max_used_idle_pct>
    <max_bytes_sec_down>0</max_bytes_sec_down>
    <max_bytes_sec_up>0</max_bytes_sec_up>
    <hangup_if_dialed/>
    <start_hour>0</start_hour>
    <end_hour>0</end_hour>
    <net_start_hour>0</net_start_hour>
    <net_end_hour>0</net_end_hour>
  </global_preferences>

• Setting logging to DEBUG gives no additional information.

What I can't tell is if this is due to FTM not being able to talk to the boinc client or not.
FTM cannot read the gui_rpc_auth.cfg, but this is the same on both RPis

Let me know if there is any further information I can provide.

If user answers 'no' to sidestaking, sidestake_check will still prompt for sidestake percentage with check_type = 'FOUNDATION'.

        while answer not in ['Y', 'N']:
            print('Error: Y or N not entered. Try again please :)')
            answer = input("")
        if answer == 'N':
            if check_type=='FOUNDATION':
                print('Ok no problem, it is your choice after all!')
            if check_type=='DEVELOPER':
                print('Ok no problem, it is your choice after all!')
                return
        print(message2)
        answer = input("")

The return should be moved out 1 level, or a return added to the first if statement.

I'm getting this issue when trying to run the script:

************************************************************************************************************************************************************************************************
*Sleep reason: NONE                                         *Info: Error checking for updates invalid update file       GRC Price: 0.009*
When you started using this tool, your average mag/hr was: 0.0007 now it is 0.0007 That's an increase of 0.02%!
Hours crunched for you vs dev: 0.0|0.0 
Use Ctrl+C to exit FTM and return BOINC to previous config
************************************************************************************************************************************************************************************************

Debug log has entries of "ERROR:root:Error getting info from goingecko" on all my RPis. Current price on coingecho is 0.021 and FTM shows 0.019.

    # Get price from coingecko
    url = 'https://www.coingecko.com/en/coins/gridcoin-research'
    regex = re.compile('(Current price of Gridcoin is USD \$)(\d*\.\d*)( with)')
    resp = ''

Looking at the page source of the above URL I can find no match to the regex. Perhaps they changed the wording on the page?

Output

Traceback (most recent call last):
  File "D:\FindTheMag2\main.py", line 2535, in <module>
    ALL_BOINC_PROJECTS=loop.run_until_complete(get_all_projects(rpc_client))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 649, in run_until_complete
    return future.result()
  File "D:\FindTheMag2\main.py", line 1498, in get_all_projects
    for project in project_status_reply:
TypeError: 'bool' object is not iterable```

Installed per instructions, xmltodict does install, but when I run the program I get:

PS D:\BOINC\Find\FindTheMag2-3.2> python main.py D:\BOINC\Find\FindTheMag2-3.2\main.py:4968: SyntaxWarning: invalid escape sequence '\|' """ Error loading some required modules. Make sure you have installed the modules in requirements.txt as documented in the README No module named 'xmltodict' PS D:\BOINC\Find\FindTheMag2-3.2>

Proof of xmltodict install:

PS D:\BOINC\Find\FindTheMag2-3.2> pip install -r requirements.txt Requirement already satisfied: certifi in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 1)) (2023.7.22) Requirement already satisfied: charset-normalizer in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 2)) (3.3.2) Requirement already satisfied: DateTime in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 3)) (5.4) Requirement already satisfied: exceptiongroup in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 4)) (1.2.0) Requirement already satisfied: idna in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 5)) (3.4) Requirement already satisfied: iniconfig in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 6)) (2.0.0) Requirement already satisfied: packaging in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 7)) (23.2) Requirement already satisfied: pluggy in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 8)) (1.4.0) Requirement already satisfied: pytest in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 9)) (8.0.2) Requirement already satisfied: pytz in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 10)) (2024.1) Requirement already satisfied: requests in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 11)) (2.31.0) Requirement already satisfied: tomli in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 12)) (2.0.1) Requirement already satisfied: typing in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 13)) (3.7.4.3) Requirement already satisfied: urllib3 in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 14)) (2.2.1) Requirement already satisfied: xmltodict in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 15)) (0.13.0) Requirement already satisfied: zope.interface in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 16)) (6.2) Requirement already satisfied: nest-asyncio in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 17)) (1.6.0) Requirement already satisfied: colorama in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from pytest->-r requirements.txt (line 9)) (0.4.6) Requirement already satisfied: setuptools in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from zope.interface->-r requirements.txt (line 16)) (69.1.0)

As previous issue, tried with newest update, same issue. Included requested python and pip versions

`PS C:\Users\tempe\Downloads\FindTheMag2-3.3\FindTheMag2-3.3> pip install -r requirements.txt
Requirement already satisfied: certifi in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 1)) (2023.7.22)
Requirement already satisfied: charset-normalizer in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 2)) (3.3.2)
Requirement already satisfied: DateTime in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 3)) (5.4)
Requirement already satisfied: exceptiongroup in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 4)) (1.2.0)
Requirement already satisfied: idna in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 5)) (3.4)
Requirement already satisfied: iniconfig in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 6)) (2.0.0)
Requirement already satisfied: packaging in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 7)) (23.2)
Requirement already satisfied: pluggy in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 8)) (1.4.0)
Requirement already satisfied: pytest in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 9)) (8.0.2)
Requirement already satisfied: pytz in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 10)) (2024.1)
Requirement already satisfied: requests in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 11)) (2.31.0)
Requirement already satisfied: tomli in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 12)) (2.0.1)
Requirement already satisfied: typing in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 13)) (3.7.4.3)
Requirement already satisfied: urllib3 in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 14)) (2.2.1)
Requirement already satisfied: xmltodict in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 15)) (0.13.0)
Requirement already satisfied: zope.interface in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 16)) (6.2)
Requirement already satisfied: nest-asyncio in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from -r requirements.txt (line 17)) (1.6.0)
Requirement already satisfied: colorama in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from pytest->-r requirements.txt (line 9)) (0.4.6)
Requirement already satisfied: setuptools in c:\users\tempe\appdata\local\programs\python\python312\lib\site-packages (from zope.interface->-r requirements.txt (line 16)) (69.1.0)

PS C:\Users\tempe\Downloads\FindTheMag2-3.3\FindTheMag2-3.3> python main.py
C:\Users\tempe\Downloads\FindTheMag2-3.3\FindTheMag2-3.3\main.py:4967: SyntaxWarning: invalid escape sequence '|'
"""
Error loading some required modules. Make sure you have installed the modules in requirements.txt as documented in the README
No module named 'xmltodict'

PS C:\Users\tempe\Downloads\FindTheMag2-3.3\FindTheMag2-3.3> python --version
Python 3.12.2

PS C:\Users\tempe\Downloads\FindTheMag2-3.3\FindTheMag2-3.3> pip --version
pip 24.0 from C:\Users\tempe\AppData\Local\Programs\Python\Python312\Lib\site-packages\pip (python 3.12)
PS C:\Users\tempe\Downloads\FindTheMag2-3.3\FindTheMag2-3.3>`

Following up on this issue. I'm still not receiving any new tasks. Also, All of my projects are showing "Won't get new tasks". There isn't a single project getting new tasks.

Have several hosts with the same hardware. Is it possible to controll them all in some way thru one installation of FindMyMag2?

FindTheMag2 miscalculates the credit/hr and therefore efficiency of multi-threaded tasks that 'hide' their threads under one executable, such as MilkyWay@home.

MilkyWay is actually using 4 or 5 threads, despite showing up as 1 task in BOINC. This results in CPU time being 4-5 times higher than wall time. It appears that credit/hr is calculated using wall time and not CPU time, but this results in MilkyWay being rewarded 4-5 times the credit/hr compared to projects that divide each thread into individual tasks.

When I run the program, I keeps seeing this status for all of my projects and BOINC is not adding any new tasks to crunch. What's going on here?

A strange one, that I only happened to stumble upon when I was testing for my other ticket #40. It requires you to gracefully terminate the script without a valid Gridcoin data directory, which is an extremely specific set of steps that no one is likely to ever come across in the typical use of the script, so you could consider this to be the lowest possible priority to fix. This happens on all python versions 3.8 to 3.12, and in both 3.0 release and the latest commit at the time of posting 8772d49

C:\Users\david\Desktop\FindTheMag2 new>python main.py
Gridcoin data dir does not appear to exist. If you have it in a non-standard location, please edit config.py so we know where to look
Press enter to continue or CTRL+C to quitProgram exiting gracefully. Please be patient this may take a few minutes
Traceback (most recent call last):
  File "C:\Users\david\Desktop\FindTheMag2 new\main.py", line 4212, in <module>
    input("Press enter to continue or CTRL+C to quit")
  File "C:\Users\david\Desktop\FindTheMag2 new\main.py", line 567, in safe_exit
    if os.path.exists(override_dest_path):
                      ^^^^^^^^^^^^^^^^^^
NameError: name 'override_dest_path' is not defined

Traceback (most recent call last):
  File "D:\FindTheMag2\main.py", line 2645, in <module>
    verification_result = loop.run_until_complete(verify_boinc_connection(rpc_client))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\base_events.py", line 649, in run_until_complete
    return future.result()
  File "D:\FindTheMag2\main.py", line 1520, in verify_boinc_connection
    response = await rpc_client._request(req)
  File "D:\FindTheMag2\libs\pyboinc\rpc_client.py", line 78, in _request
    return await self._raw_client.request(req)
  File "D:\FindTheMag2\libs\pyboinc\_raw_client.py", line 82, in request
    return await self.receive()
  File "D:\FindTheMag2\libs\pyboinc\_raw_client.py", line 63, in receive
    buff = [await self._read()]
  File "D:\FindTheMag2\libs\pyboinc\_raw_client.py", line 57, in _read
    return await self._reader.readuntil(separator=END_OF_MESSAGE)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.3056.0_x64__qbz5n2kfra8p0\lib\asyncio\streams.py", line 613, in readuntil
    raise exceptions.IncompleteReadError(chunk, None)
asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of undefined expected bytes

debug.log

Traceback (most recent call last):
File "C:\Users\jmjon\FindTheMag2-3.0\main.py", line 4602, in
boinc_loop(False, rpc_client)
File "C:\Users\jmjon\FindTheMag2-3.0\main.py", line 3545, in boinc_loop
) = generate_stats(
^^^^^^^^^^^^^^^
File "C:\Users\jmjon\FindTheMag2-3.0\main.py", line 2189, in generate_stats
combined_stats, unapproved_projects = add_mag_to_combined_stats(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\jmjon\FindTheMag2-3.0\main.py", line 1618, in add_mag_to_combined_stats
found_mag_ratio = mag_ratios.get(project_url)
^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

main.py
Line 425, 426

    else:
        DATABASE['TABLE_STATUS']='Error getting info from goingecko'
        log.error('Error getting info from goingecko')
    # Return average of all found prices

Noticed this in the log as I'm getting continual errors logged related to retrieving the latest GRC price. Not sure why the

Hi!

Could you please explain in the readme what a host group is and what is its purpose, in general and for the use of FindTheMag2? I am contributing to Arikado pool, where there is no such concept, and I do not understand whether the lack of host groups is a limitation.

Also, I do not understand from the readme or running the tool just for stats if the stats it provides are provided for all the hosts together or separate, per host.

Thanks!

Is "messagex" correct?

But the real reason for this issue... the "Project requested delay ..."

Looking at main.py, it looks like this message was removed the positive list due to projects sending this message out even when there is work, etc. However might I suggest adding the message to ignore_phrases so that the log entry is not continually generated and backoff is still returned as false.

What triggered this error was a user error... I forgot to logoff/logon after adding my user to the boinc group so the user did not have the required access to the gui_rpc_auth.cfg file.
However the following crash occurred when FTM attempted to log the permissions error:

Press enter key to start controlling BOINC. Press Ctrl+C to quit

Starting control of BOINC...
Traceback (most recent call last):
  File "/home/piadm/FTM2/FindTheMag2/main.py", line 2638, in <module>
    print_and_log('Error connecting to BOINC client, does your gui_rpc_auth.cfg specify a password or a non-standard port?\n If so, be sure to include it in your config.py')
TypeError: print_and_log() missing 1 required positional argument: 'log_level'

Information:

• RaspberryPi running Debian 11 Pi OS
• No wallet installed
• Boinc installed and configured
• FTM2 pulled from git 2023-04-09 VERSION=2.2

I modified main.py to include the log level "ERROR" in the call to test the error handling:

diff --git a/main.py b/main.py
index 62a671f..39a9495 100644
--- a/main.py
+++ b/main.py
@@ -2635,7 +2635,7 @@ if __name__ == '__main__':
 
     verification_result = loop.run_until_complete(verify_boinc_connection(rpc_client))
     if not verification_result:
-        print_and_log('Error connecting to BOINC client, does your gui_rpc_auth.cfg specify a password or a non-standard port?\n If so, be sure to include it in your config.py')
+        print_and_log('Error connecting to BOINC client, does your gui_rpc_auth.cfg specify a password or a non-standard port?\n If so, be sure to include it in your config.py','ERROR')
         print('You can find your gui_rpc_auth.cfg at {}'.format(auth_location))
         print('Linux users: make sure your username is in the BOINC group so FTM can access your BOINC config file')
         print('sudo usermod -aG boinc your_username_here')
@@ -2655,4 +2655,4 @@ if __name__ == '__main__':
     # While we don't have enough tasks, continue cycling through project list and updating. If we have cycled through all projects, get_highest_priority_project will stall to prevent requesting too often
     boinc_loop(False,rpc_client)
     # Restore user prefs
-    safe_exit(None,None)
\ No newline at end of file
+    safe_exit(None,None)

This allowed the program to continue slightly but the following was generated; I'm not sure if this is intentional or if there is a more graceful way to exit on error.

Press enter key to start controlling BOINC. Press Ctrl+C to quit
Starting control of BOINC...
Error connecting to BOINC client, does your gui_rpc_auth.cfg specify a password or a non-standard port?
 If so, be sure to include it in your config.py
You can find your gui_rpc_auth.cfg at /var/lib/boinc-client/gui_rpc_auth.cfg
Linux users: make sure your username is in the BOINC group so FTM can access your BOINC config file
sudo usermod -aG boinc your_username_here
Note that you will need to restart your computer after changing your group permissions
Press enter to quit
Traceback (most recent call last):
  File "/home/piadm/FTM2/FindTheMag2/main.py", line 2644, in <module>
    loop.run_until_complete(prefs_check(rpc_client))
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/piadm/FTM2/FindTheMag2/main.py", line 1532, in prefs_check
    response = await rpc_client._request(req)
  File "/home/piadm/FTM2/FindTheMag2/libs/pyboinc/rpc_client.py", line 78, in _request
    return await self._raw_client.request(req)
  File "/home/piadm/FTM2/FindTheMag2/libs/pyboinc/_raw_client.py", line 82, in request
    return await self.receive()
  File "/home/piadm/FTM2/FindTheMag2/libs/pyboinc/_raw_client.py", line 63, in receive
    buff = [await self._read()]
  File "/home/piadm/FTM2/FindTheMag2/libs/pyboinc/_raw_client.py", line 57, in _read
    return await self._reader.readuntil(separator=END_OF_MESSAGE)
  File "/usr/lib/python3.9/asyncio/streams.py", line 629, in readuntil
    raise exceptions.IncompleteReadError(chunk, None)

Let me know if you need any further information or when you put a fix in if you want me to test it.

Every project is showing "Won't get new tasks" on my PC. This started happening once World Community Grid starting having issues.

Relevant stacktrace below, topic as above.

Starting control of BOINC...
Traceback (most recent call last):
  File "D:\Tools\FindTheMag2-main\main.py", line 2587, in <module>
    shutil.copy(override_path,override_dest_path)
  File "C:\Users\sythi\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 417, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:\Users\sythi\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 254, in copyfile
    with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\ProgramData\\BOINC\\global_prefs_override.xml'
Press any key to continue . . .

/var/home/fennectech/FindTheMag2/main.py:4967: SyntaxWarning: invalid escape sequence '|'
"""

Error message is missing a single quote in the word "don't"

Starting control of BOINC...
BOINC is configured to use less than 10GB, this tool will not run with <10GB allocated in order to prevent requesting base project files from projects too often.
If you have configured BOINC to be able to use >=10GB and still get this message, it is because you are low on disk space and BOINC is responding to settings such as "don     use greater than X% of space" or "leave x% free"
Press enter to quit

Line 1539: "don\t" should be "don't"

        print('If you have configured BOINC to be able to use >=10GB and still get this message, it is because you are low on disk space and BOINC is responding to settings such as "don\t use greater than X% of space" or "leave x% free"')

piadm@labpi1:~/FTM2/FindTheMag2 $ python main.py 
Gridcoin data dir does not appear to exist. If you have it in a non-standard location, please edit config.py so we know where to look
Press enter to continue or CTRL+C to quit
Traceback (most recent call last):
  File "/home/piadm/FTM2/FindTheMag2/main.py", line 2524, in <module>
    rpc_client = loop.run_until_complete(setup_connection(boinc_ip,boinc_password,boinc_port)) # setup BOINC RPC connection
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/piadm/FTM2/FindTheMag2/main.py", line 277, in setup_connection
    rpc_client = await init_rpc_client(boinc_ip, boinc_password, port=port)
  File "/home/piadm/FTM2/FindTheMag2/libs/pyboinc/rpc_client.py", line 31, in init_rpc_client
    await c.connect()
  File "/home/piadm/FTM2/FindTheMag2/libs/pyboinc/rpc_client.py", line 57, in connect
    await self._raw_client.connect()
  File "/home/piadm/FTM2/FindTheMag2/libs/pyboinc/_raw_client.py", line 35, in connect
    self._reader, self._writer = await asyncio.open_connection(self.host, self.port, family=AF_INET, limit=1024*5000)
  File "/usr/lib/python3.9/asyncio/streams.py", line 52, in open_connection
    transport, _ = await loop.create_connection(
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1056, in create_connection
    raise exceptions[0]
  File "/usr/lib/python3.9/asyncio/base_events.py", line 1041, in create_connection
    sock = await self._connect_sock(
  File "/usr/lib/python3.9/asyncio/base_events.py", line 955, in _connect_sock
    await self.sock_connect(sock, address)
  File "/usr/lib/python3.9/asyncio/selector_events.py", line 502, in sock_connect
    return await fut
  File "/usr/lib/python3.9/asyncio/selector_events.py", line 537, in _sock_connect_cb
    raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 31416)

• Restarted the Pi after it lost network connectivity.
• Connected via ssh, and launched FTM
• FTM crashed when the connection to the local boinc client failed

FTM should handle this error gracefully rather than just crashing.

Started FTM again about a minute later and it launched fine. I suspect that as this is a Pi3 that the boinc client was not up and running when I launched FTM the first time.

I had assumed that setting a project as ignored would set it to "won't get new tasks" permanently, instead it has it's weight set to zero and still updates with the rest of the projects. In my case this ended up reducing my mag until I caught on.

Not really an issue, just a bit of confusion that could use some tuning. My setup was boinc on Windows 10, using BOINC/stats as a manager. I had set weights for different projects before starting FTM3.

It was not clear in the install instructions (readme.me) about how to prepare before starting FTM3. Was I, as a user, supposed to set my individual projects to 1 weight before starting? I had 4 projects set at 100, the rest at 1 prior to starting. I did not see a change in weight when FTM3 started.

Instead of setting the weights at BOINC/stats, I disconnected the manager, went to the individual projects, set them all to 1, and continued running FTM3.

Does FTM3 control weight on individual projects? Does being connected to BOINC/stats interfere with FTM3's control of weights? Do I wait for FTM3 to adjust weights going forward for the increased earnings?

Thank you for your great work, I really do appreciate you doing this!

Cheers

I don't know how feasible this would be to even do, if it even can be done at all. Really the main thing I would want is some external thing that is aware of both BONIC and Folding@Home, and be able to balance them both accordingly, so there's no concern for them having to fight for resources.

AS F@H has its own crunching and control method though, tying to get compatibility with BONIC is always going to be a challenge. Though surely there is some external way you can control it, and hopefully even a solution using python?

When running with SCRIPTED_RUN:bool=True the script prints that it will work for dev on Linux. On MacOS it refuses to start due to that it can't find the sidestake.

The error shown is "Error communicating with Gridcoin wallet! 'Contract Contents'".

The debug message right at the start of the function indicates that the cached results are being returned... before the decision is made to return the cached version or an updated version.

    # Return cached version if we have it and requested it < 24 hrs ago
    log.debug('Returning cached version of gridcoinstats data')
    delta=datetime.datetime.now()-DATABASE.get('LASTGRIDCOINSTATSPROJECTCHECK',datetime.datetime(1993,3,3))
    if abs(delta.days)<1 and 'GSPROJECTLIST' in DATABASE and 'GSRESOLVERDICT' in DATABASE:
        return DATABASE['GSPROJECTLIST'],DATABASE['GSRESOLVERDICT']

The debug message should either be moved down into the if statement (having one there would be a useful thing), or changed to reflect having entered the get_approved_project_urls_web() function.

FYI:
I found this when trying to debug why Universe@home has not been picked up as being back on the whitelist. None of the 12 instances of FTM has picked this up despite Universe being re-whitelisted for the past several days. I have not restarted any of the FTM instances recently.

It would be useful to have a debug message message near the final return in the function indicating that the updated results from the website are being returned. This way you could identify both possible normal outcomes of the function.

So my crunching setu is a Windows server PC with GUI, as I find it a easier to use the Gridcoin wallet GUI rather then have to do everything in a terminal. The purpose I want is to have the script be able to run itself on login, as I have Windows set to restart itself after applying any updates that need it.

My first thought was to create a bat file that runs main.py and put that in the startup folder, and then set SCRIPTED_RUN to True, so that way it doesn't pause on start waiting for user input. The running of the script worked fine, and it even defaults to displaying the output in a terminal window which is what I wanted/prefered. However an immediate problem was apparent, which is the script would put create and store its output files wherever it was run from. If that location is the startup folder then obviously this is not ideal. This seems to happen regardless of what the SCRIPTED_RUN parameter is set to.

Personally, I would say to always have to store its output files in the same directory as wherever main.py is located, or even you could create a subdirectory next to the script and store it all in there if you wanted to make it a bit cleaner. I'm not sure how to set the output directory myself though, as there doesn't appear to be a parameter in the confy.py to set it.

Another possible problem/oversight I found is when SCRIPTED_RUN is set to True, that regardless of how the script was run, in the Info section I get the message about consider donating via sidestake, which I have already done, and that gets picked up correctly if I run the script with it set to False, as it says in the user/pause prompt how its recognised that it has been done. I can't tell if it isn't able to detect the sidestake and is running the dev crunching, or if is just simply the message that appears when it shouldn't be, as everything is otherwise working as it should be.

The calculation of the increase in % is not correct.

When you started using this tool, your average mag/hr was: 0.0036 now it is 0.0053 That's an increase of 0.49%!

On my two Mac's the settings aren't restored when exiting gracefully. Have added my user to the boinc_master group but still not working.

Getting the following error:

Restoring original preferences...
Permission error restoring original BOINC preferences [Errno 1] Operation not permitted: '/Library/Application Support/BOINC Data/global_prefs_override.xml'
Be sure you have permission to edit this file
Linux users try  'sudo usermod -aG boinc your_username_here' to fix this error
Note that you will need to restart your machine for these changes to take effect

If I launch the script with sudo python3 main.py it works as intended.

FTM2 crashes on my Ubuntu PC with the following message.

Traceback (most recent call last):
  File "/home/daniel/FindTheMag2/main.py", line 2666, in <module>
    boinc_loop(False,rpc_client)
  File "/home/daniel/FindTheMag2/main.py", line 2279, in boinc_loop
    custom_sleep(30,rpc_client,dev_loop=dev_loop)  # There's no reason to loop through all projects more than once every 30 minutes
  File "/home/daniel/FindTheMag2/main.py", line 1744, in custom_sleep
    if loop.run_until_complete(is_boinc_crunching(boinc_rpc_client)):
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/daniel/FindTheMag2/main.py", line 262, in is_boinc_crunching
    reply = await run_rpc_command(rpc_client, 'get_cc_status')
  File "/home/daniel/FindTheMag2/main.py", line 743, in run_rpc_command
    response = await rpc_client._request(req)
  File "/home/daniel/FindTheMag2/libs/pyboinc/rpc_client.py", line 78, in _request
    return await self._raw_client.request(req)
  File "/home/daniel/FindTheMag2/libs/pyboinc/_raw_client.py", line 82, in request
    return await self.receive()
  File "/home/daniel/FindTheMag2/libs/pyboinc/_raw_client.py", line 63, in receive
    buff = [await self._read()]
  File "/home/daniel/FindTheMag2/libs/pyboinc/_raw_client.py", line 57, in _read
    return await self._reader.readuntil(separator=END_OF_MESSAGE)
  File "/usr/lib/python3.10/asyncio/streams.py", line 614, in readuntil
    raise exceptions.IncompleteReadError(chunk, None)
asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of undefined expected bytes