Terrible performance compared to Blagominer on large system #332
Comments
|
I am no expert on creepMiner but I recall some recommended settings in the install guides. You should probably review/tweak the settings for your best performance. To me though, your intensity/readers seems a bit high. Note, your images just give a 404 error. |
|
I never was able to locate any install guides, just the Wiki which explained the items in the config file. I have tried various things in the config file to no avail. I'll just stick to Blago's miner which just works. As per my original post, so see the images, you have to cut and paste the URLs I posted. Not sure why they don't work directly like on all other forums. |
|
Hi peteraclausen, I was also having read issues on a SAN had slow read times ~120s on 50TB LUN's but can read / write at 2000MB/s and it did not telly up. It all comes down to two things / hardware layout and mining.config setup. Please jump on discord and look me up, lets discuss there then update this once we find you an optimal solution. |
|
I also report performance issue on Case 311 and never get resolve. Reading this post make more sense because my plot is also large (140TB): Notice that CreepMiner read fast when there's 6 or less Hard Drive. When I try add more, the reading speed start to get slower. In my case with 17 drives, Blago read all plots never less than 100MB/s. CreepMiner read most plots with 60~90MB/s. 6 Hard Disks reading samples:
17 Hard Drives reading samples:
|
|
set "bufferChunkCount" to 4096 -> 4K reads perform way better then 16byte reads. |
|
It takes forever to read if I set "bufferChunkCount" to 4096 |
|
hey guys, @ChrOst and @jretropie actually @ALL could you please enable the benchmark feature and post the benchmark.csv for review? |
|
My findings were:
|
|
I am not sure what ChrOst trying to come up with your findings.......
Notice that when I set CPU to AVX or AVX2, it made no different on performance. From my experience with Blago miner, Xplotter, and TurboPlotter, performance should pretty much double when switching from AVX to AVX2. |
|
that just means its not a processing blottleneck, but getting the data to the processor bottleneck. one thing i can add to the whole blago vs creep debacle: |
|
@jretropie they are as fast as they can get using the CPU. using BufferChunkSize 16 decreases it by 100MB/s, setting it to 8 decreases it to 540MB/s. Currently struggeling with a CUDA Card. |
|
@ChrOst I am not sure what you trying to do here but 757MB/s is VERY slow if you have 14 Disks. Consider Blago reads every disk more than 100MB/s (100 x 14). You should get at least 1400MB/s total. |
|
@jretropie yes. it can read over 1600MB/s. BUT: the CPU won't do the Job during mining. As it has to calculate some stuff after reading. And that Xeon just can do about 800MB/s. |
|
Interesting stuff. This is my experience: On non avx capable cpu's I have found creepminer to be much faster than blago - over 1/3rd if I remember correctly. This was on an AMD Phenom 6core (X6 1090T). @ChrOst |
|
Hey, just to report I have the same issue. Could be windows related, as I only see win guys complaining. I run 20 disks, blago ~27s, creepminer ~52s, roughly double the time. SMR drives. cpu is i7-7700 using avx2. I played around with all params, no luck. Neither my CPU nor my HDD I/O is maxed will running creep. Hyperthreadding issue? I/O issue? no idea. back to blago for now. |
|
Here some perfmon capture I took. I marked blago red and creep green. %Interrupt and %Privileged time is highlighted. We can clearly see that creepminer generates a lot more interrupts when reading the plots (up to 50% cpu usage). I don't now the reason, but to me it seems like the "slow performance" is definitely coming from the way the plots are being read. |
|
I would like to report the same issue, our build runs with Ubuntu 16.04 using CreepMiner latest with NVIDIA Cuda on Titan XP, 200TB scans are well over 1 minute consistently. When we switch to JMiner, scans on the same plot same hardware is performed in 27 to 30 seconds Same hardware on Windows 10 x64 with Blago miner on AVX2 is about 22 to 24 seconds Something worth to mention, Creepminer assumes and report the Plot size to be around 190TB, but when using Jminer it reports 204TB on the same hardware, so for whatever reason the size detected is inconsistent 14TB delta is HUGE! Anybody have done any performance test with Burst to scan using EXT4 vs BTRFS, vs XFS, how about switching the Linux IO scheduler running on noop, anticipatory, deadline, cfq Thanks! |
|
@sgabay242 |
|
@sgabay242 I can confirm the same finding, much longer scan times vs jMiner even o a small plot of 17TB. jMiner completes it in just over 9 seconds when creepMiner requires 20 seconds to complete it. 2 GPUs have been tested R7 250 and RX 480 (I actually get the same times on both GPUs of 9 seconds on my small plot with jMiner). |
|
1.9.0.7 compared to 1.9.0.5 seems to have a performance problem too. |
|
Thank you everybody for reporting. If you are brave enough, you can test the current pre-compiled dev version - thanks in advance for every bug report. Please bear in mind, that the whole read/calculate algorithm changed - so most likely you also have to change your config. Most significant options are: maxBufferSizeMB and bufferchunkcount. |
|
Just to quickly report back. |
|
I'm in the process of replotting for poc2... so I can't compare the perfomance directly right now |
|
The 14 version seems to have the same problem with wrong deadlines.
|
|
"poc2StartBlock" <-- what does this say in your mining.conf? change it to 502000 please |
|
It is 502000.
|
|
clinfo |grep "Global memory size" vs "maxBufferSizeMB" ? |
|
@ChrOst I think I have played with all the possible options I can think of and I am still getting the error: |
|
I got to the bottom of the |
|
As this is not related to the original performance issue I guess we shouldn't discuss this here on and on again. The Performance seemed to be fixed in the dev build. |
|
I tried 1.9.0.16 today (most of my drives are now done with poc2) |
|
Just an update if you are interested. I have been running the dev build 1.9.0.16 for a few days now and so far no issues. It runs fine, scan times are around 18sec (a few seconds faster than blago) for my 100TB. |
I have a large miner with 108 hard drives and a total capacity of 407.5TB. My server consists of a dual E5-2660 V4 CPU server with 64GB of RAM. All 108 drives are directly connected via LSI 9211-8i and 9200-8e HBA's via SAS2 backplane expanders. Using the AVX2 version of the Blago miner, my plots are scanned in about 20 seconds at a speed of about 5,000 MB/s.
Switching to creepMiner, my performance slows to a crawl. I have the following in my mining.conf file:
"bufferChunkCount" : 16, "cpuInstructionSet" : "AVX2", "getMiningInfoInterval" : 3, "gpuDevice" : 0, "gpuPlatform" : 0, "intensity" : 108, "maxBufferSizeMB" : 55000, "maxPlotReaders" : 108, "processorType" : "CPU", "rescanEveryBlock" : false,Upon startup, I do see the following:
Total plots size: 407.51TB
Mining Intensity: 108
Max plot readers: 108
Processor type: CPU
CPU instruction set: AVX2
creepminer takes about 300 seconds to scan my plots compared to about 20 seconds for Blago miner. So something seems off to say the least.
Here's a view from creepMiner after completing a scan:
http://www.cstone.net/~dk/creepminerscanspeed.PNG
And from Blago miner for comparison:
http://www.cstone.net/~dk/dualargonscan5.PNG
EDIT: for some reason the above links don't work unless you cut and paste them into a new tab in your browser.
Perhaps my hardware is not compatible with creepMiner for some reason?
The text was updated successfully, but these errors were encountered: