sumokoind high CPU Usage about sumokoin HOT 45 CLOSED

run sumokoin daemon with this flag --prep-blocks-threads 1 i.e.
./sumokoind --prep-blocks-threads 1

from sumokoin.

Thanks @sumogr, that has made a difference. Still runs high initially, then calms down.

from sumokoin.

This is still causing issues.
I am now getting timeout errors from my nodejs-pool longRunner and blockManager modules.

Any further ideas?

from sumokoin.

I didnt know you were running a pool, you didnt mention it in your first post. Increase the number of threads on the flag. Try --prep-blocks-threads 3 (4 is default). If all else fails try a server with more cores

from sumokoin.

Apologies, I should have mentioned it. I've now reverted back to running it without the switch, and when I last looked, it was running a lot better. I will try your suggestion if/when it falls over again.

Thanks

from sumokoin.

Is anyone else experiencing performance issues?
What sort of hardware specs do you have?

from sumokoin.

I have built 4 other coin pools and all of them are fine with the same spec. Xeon Quad Core E3-1225 V3, 4GB ram.

Sumo is the only one giving me problems.

Now I'm thinking of abandoning it.

from sumokoin.

Are they cryptonote coin pools? Ok, please close this issue

from sumokoin.

Yes all cryptonote pools.

Why do you want me to close this?

I'm just trying to understand what the hardware requirements are. I be live hardware is sufficient.

from sumokoin.

@BKdilse Generally speaking, for small pools, 2 vCPUs (Intel with AES support) should be enough. You should still close the issue as it only marginally relates to sumokoin daemon. Try a server with more CPU power (better CPUs, more cores, etc.). As far as your nodejs issues are concerned, they don't belong in this forum.

from sumokoin.

@jpenner123 I have 1 miner on the Pool, and 4 cores with AES support. I don't have issues with nodejs. I am questioning the daemon as that is hogging the cpu.

from sumokoin.

Try restarting your daemon with this flag this time --max-concurrency 1 (and increase that number till your cpu gets hogged)

from sumokoin.

@BKdilse On my pool, the initial usage goes up for a few seconds, then goes down to between 0.1% and 0.7% (also Intel AES Support). Once in a while I do see peeks that go way up but that is normal. Try using tools like iostat / strace in order to figure out what's going on on your system.

from sumokoin.

@sumogr thanks, I'll try that setting.

@jpenner123 Mine does fluctuate, but mostly remains very high. The pool UI therefor does not see block updates for ages, sometimes 45 minutes.

I'll try some more testing tonight.

from sumokoin.

@BKdilse Is it actually sumokoind causing your problem, or the wallet rpc? I'm having issues with the wallet rpc blocking when running a pool, but haven't identified a cause yet. sumokoind does use a fair amount of cpu, but tends to max out at 2-3 full cores on a 24 core server. That's based on blocks and block template requests from the pool; cpu usage on both of those was improved in post 0.10 Monero releases.

from sumokoin.

@bobfeldbauer I believe it's the daemon and not wallet. Api does not update quick enough. I will spend more time over the weekend with the suggestions given.

from sumokoin.

@sumogr I'm trying out --max-concurrency setting. Do you know what if defaults to, without using this. I have 4 cores, does it use 4 threads max, or does it have nothing to do with CPU cores?

from sumokoin.

@sumogr @bobfeldbauer Guys, this is what I have found:

Daemon command line:
/usr/local/src/sumokoin/build/release/bin/sumokoind --rpc-bind-ip 0.0.0.0 --restricted-rpc --prep-blocks-threads 3 --max-concurrency 1

The above command line does lower the CPU Usage slighty, but problem still exists.
It's not the daemon on it's own causing the issue, as CPU usage is low. As soon as I start wallet RPC, the CPU shots up for the Daemon process, and stays up, hovering at around 95-100%

No errors in the wallet logs.

from sumokoin.

Just reverted back to:

/usr/local/src/sumokoin/build/release/bin/sumokoind --rpc-bind-ip 0.0.0.0 --restricted-rpc

Now performance is perfect. Puzzled. Something is going on here, not sure what. CPU is hovering at 3-4%.

from sumokoin.

have you compiled everything yourself? can you please try running everything with the compiled binaries? Lets try this approach

from sumokoin.

@sumogr Yes, I compiled it myself, a few weeks ago. I had to use the make release-static option due to SSL errors. I am running the compiled libraries.

from sumokoin.

I am sure that it will run perfectly from the compiled binaries (I mean binaries not libraries :) ). The SSL errors you were getting were fixed with the last comits. If you dont want to run from the binaries try to recompile please.

from sumokoin.

By "compiled binaries", I take it you are referring to the ones the devs have compiled?

OK, so at this moment, it's been running perfectly for the last 30 minutes. I will leave it for a few days, and if/when problems re-occurs, I'll try to compile again with the latest source. If that doesn't work, I'll try the compiled binaries.

Thanks, closing for now.

from sumokoin.

Implementing this would improve things, whether it directly fixes this specific issue or not. monero-project/monero@935f746

There are probably at least a dozen blockchain sync and rpc issues present in 0.10 Monero which were fixed in later releases though. Are they being tracked in any way to fix them in Sumokoin?

from sumokoin.

Compiled latest version, and it's worse !!!
As soon as I start my sumo-wallet-rpc, the daemon dies with "Aborted (core dumped)"

I then downloaded the compiled version, which works, but has the same CPU hogging issue.

Any further suggestions?

from sumokoin.

Devs confirmed that they are scanning the code to find the issue that throttles the cpu usage to a few users, as most likely this is a bug.
Can you try something in the meantime that may sound silly but most likely will provide a temporary solution till the fix?

1. install cpulimit: sudo apt-get -y install cpulimit
2. Run top command and find the PID of the sumokoind
3. Run sudo cpulimit --pid (daemon PID) --limit (input an integer equal to the max percentage of cpu power you want the daemon to be consuming e.g 15)
   Run the above on a screen and detach it in the background
   Hope that gives a temporary solution till the fix

from sumokoin.

I've given the server a reboot today, and it appears to be performing a little better.

btw, the overall CPU usage is 30-35%. The sumokoind process is using up around 140%.

I'll give your suggestion a try when I see very bad performance.

from sumokoin.

ok. Leave the issue open

from sumokoin.

I can confirm that a similar behavior is happening for me as well on E3-1270v5.

from sumokoin.

There is a pull, that is being tested atm before being merged, solving the high cpu usage (along with other things) if you cant wait or want to test it yourself before merging please compile from here
https://github.com/haruto-tanno/sumokoin
Edit: Its now merged so please try to recompile from official

from sumokoin.

I'll give that a go tonight. Thanks @sumogr

from sumokoin.

@sumogr Here are the results from the new compiled version:

1. Start Daemon, completed synching the last few blocks.
2. Start Nodejs Pool modules
3. CPU usage is acceptable now
4. Daemon crashes out "Aborte, core dumped".

Tried a few times, and had same issue.

Then I tried 3rd time, and when it crashed out, I simply restarted the daemon, and it is working so far.

Question is, what is causing it to dump core?

UPDATE: Tried again, and getting random results. Sometimes the sumo wallet rpc crashses daemon, and other times it does not.

I have got it running for now, but am concerned that it may die on me when miners are using pool. Please advise...

from sumokoin.

I take that back, it's still crashing out, so it's actual worse than before, just isn't stable. Any suggestions?

NOTE: Reverted back to old saved Daemon for now, until you advise....

from sumokoin.

Really strange, the official pool is running well with the latest code atm without any restrict. What's your OS? I suggest Ubuntu 16.04 64-bit and compile code clean:

make clean && make release-static

Of course, 4GB RAM is not ideal for running a pool but CPU should be fine.

from sumokoin.

@quangvu3 OS is Ubuntu 16.04.4 LTS 64-bit.

Trying now with your suggestion.

from sumokoin.

@quangvu3 That's working a lot better. Not crashed, and low resource usage.

I have noticed something which is probably not related... The daemon shows last block synched as 113095, but my pools network stats show 113094 found an hour ago (Other pools showing same delay). Not noticed that much delay previously. Any thoughts?

UPDATE: It just updated, showing latest block.

from sumokoin.

Latest Daemon update has fixed the performance issue, thanks to all involved in this. :)

from sumokoin.

Few days running, and the issue is back.
Ubuntu top shows this: CPU147.7% MEM 57.6% process sumokoind

Any further suggestions?

from sumokoin.

Hello,

I have a similar issue with sumokoind CPU usage. I'm also running other cryptonote coin daemon without the same high-cpu usage. And Sumokoin Pool hashrate isn't high (only 1KH/s).

from sumokoin.

I can also confirm (again) the same high CPU usage.

from sumokoin.

Can anyone confirm the issue after upgrading binaries to v0.3.0.0 (Aomori)?

from sumokoin.

Yes. I can. Same issue. Out of all my 10 coin pools, sumo is the highest cpu usage pool.

from sumokoin.

While it is understandable that Sumokoin daemon may consume a little more CPU than others (due to larger RingCT size), it has never been a problem at official pool. Here is a screenshot from top command, rarely more than 5%:

from sumokoin.

What hardware is it running on?

from sumokoin.

Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz, 32GB RAM

 cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 58
model name      : Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz
stepping        : 9
microcode       : 0x1c
cpu MHz         : 2587.353
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm epb retpoline kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts
bogomips        : 6784.56
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual

from sumokoin.