Long 2 are a joke
log in

Advanced search

Message boards : Number crunching : Long 2 are a joke

Previous · 1 · 2 · 3 · Next
Author Message
[SG]Felix
Avatar
Send message
Joined: 25 Dec 17
Posts: 63
Credit: 21,692,929
RAC: 9
Message 4855 - Posted: 3 Jan 2019, 16:43:20 UTC
Last modified: 3 Jan 2019, 16:49:30 UTC

for long3 i have runtimes of under 2 hours.
note that the shown runtimes aren't correct, because boinc has problems when you load 9 wus and only compute one after the other. once i have wus which i got alone without other wus, the runtimes are correct. right time is the time in seconds shown at the bottom of the stderr out

heres my app-config.xml for an i7-5820k:

<app_config>

<app_version>
<app_name>srbase</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase2</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase3</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase4</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase5</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase6</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase7</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase8</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase9</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase10</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase11</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>

<app_version>
<app_name>srbase12</app_name>
<cmdline>-t9</cmdline>
<avg_ncpus>9</avg_ncpus>
</app_version>
</app_config>

last result for long3 here

Profile marmot
Avatar
Send message
Joined: 17 Nov 16
Posts: 97
Credit: 126,410,450
RAC: 21,911
Message 4906 - Posted: 2 Feb 2019, 1:18:57 UTC
Last modified: 2 Feb 2019, 1:21:07 UTC

Given my longs 3 threads and still running up against deadlines on some of them.

Why does BOINC start a WU with only 12 hours left that took 3 days to run?

Guessing BOINC is ignorant and only the server here would have the data on average run times and would have to determine the WU couldn't complete and do the cancellation on it's end, right?

I had self abort 8+ longs per machine by using my own deadline calculations.

A longer deadline or more aggressive server cancellations would be appreciated so I can reduce my management time.

Profile marmot
Avatar
Send message
Joined: 17 Nov 16
Posts: 97
Credit: 126,410,450
RAC: 21,911
Message 4907 - Posted: 2 Feb 2019, 1:24:53 UTC - in response to Message 4766.
Last modified: 2 Feb 2019, 1:32:55 UTC

Mine work using <cmdline>-t4</cmdline>

Wuprop only counts 1 thread though.



That REALLY sucks... I was going to increase thread count to 6 until noticing your comment and now really want to reduce it back to 1 and abort all long WU that don't have 6 days deadline.

One of my main goals it to complete 100,000 hours per work unit at WuProp.

Reduce hours counted by 1/3 on longs will mean it might be 4 years to reach 100,000 hours since WU here are intermittent and I shut down work in the summer to avoid contributing to global warming with A/C since our community still uses coal and natural gas fired power plants.



@Rebirther, why not triple the deadline on longs and then there will be much less wasted work, and a higher probability of work 24/7 instead of these long gaps of no WU's?

Profile PDW
Send message
Joined: 15 Oct 15
Posts: 41
Credit: 1,078,223,594
RAC: 43,916
Message 4910 - Posted: 2 Feb 2019, 10:12:55 UTC - in response to Message 4907.

Mine work using <cmdline>-t4</cmdline>

Wuprop only counts 1 thread though.



That REALLY sucks... I was going to increase thread count to 6 until noticing your comment and now really want to reduce it back to 1 and abort all long WU that don't have 6 days deadline.

One of my main goals it to complete 100,000 hours per work unit at WuProp.

Reduce hours counted by 1/3 on longs will mean it might be 4 years to reach 100,000 hours since WU here are intermittent and I shut down work in the summer to avoid contributing to global warming with A/C since our community still uses coal and natural gas fired power plants.

What Conan (and others) posted saying to add the following line did make the hours all count for Wuprop...

<avg_ncpus>4</avg_ncpus>

Just make the number the same as your <cmdline>-t4</cmdline> number.

Profile marmot
Avatar
Send message
Joined: 17 Nov 16
Posts: 97
Credit: 126,410,450
RAC: 21,911
Message 4915 - Posted: 3 Feb 2019, 2:52:15 UTC - in response to Message 4910.



<avg_ncpus>4</avg_ncpus>

Just make the number the same as your <cmdline>-t4</cmdline> number.


Thanks, did that already.

The long1's were getting about 330 hours/day and that was what I expect from single thread, 1 per machine, but they are set to 3 threads and so it seemed like WUprop wasn't counting them correctly.

Some new long1-3 came down today and I'll try and get accurate accounting.

Profile Michael Goetz
Avatar
Send message
Joined: 1 Jan 15
Posts: 25
Credit: 1,197,466
RAC: 0
Message 5126 - Posted: 20 Apr 2019, 17:19:35 UTC
Last modified: 20 Apr 2019, 17:21:24 UTC

I'm running some long2 and long3 tasks right now, on a non-AVX computer. The long2 tasks will take about 11 days to run. They also were queued up for a few days, so they'll finish about 4 days after the deadline. Starting them late is my fault.

There's 3 different things I'd like to mention, however:

1) The discussion in this thread about multi-threading confuses me. Does multithreading even work? If so, then how? At PrimeGrid, our wrapper is modified to specifically pass the "-t#" parameter to LLR, but in doing so it has to convert "-t#" to "-t #". Unless the wrapper here passes the parameter to LLR, and converts it to the correct format, I don't see how you could actually get multithreading to work. When I tried it, CPU temperatures dropped dramatically, indicating that only one core was being used.

2) Looking in the in-progress stderr.txt file, it's obvious that the BOINC manager is not reporting the correct progress done amount.

3) Maybe the deadline should be long enough to let computers without AVX to finish?

[SG]Felix
Avatar
Send message
Joined: 25 Dec 17
Posts: 63
Credit: 21,692,929
RAC: 9
Message 5127 - Posted: 20 Apr 2019, 17:25:50 UTC - in response to Message 5126.
Last modified: 20 Apr 2019, 17:27:53 UTC



1) The discussion in this thread about multi-threading confuses me. Does multithreading even work? If so, then how? At PrimeGrid, our wrapper is modified to specifically pass the "-t#" parameter to LLR, but in doing so it has to convert "-t#" to "-t #". Unless the wrapper here passes the parameter to LLR, and converts it to the correct format, I don't see how you could actually get multithreading to work. When I tried it, CPU temperatures dropped dramatically, indicating that only one core was being used.



it works the same way as it works at Primegrid

with -t8 my i7-5820k needs about 11-12 hours for one task

Profile Michael Goetz
Avatar
Send message
Joined: 1 Jan 15
Posts: 25
Credit: 1,197,466
RAC: 0
Message 5128 - Posted: 20 Apr 2019, 18:09:02 UTC - in response to Message 5127.
Last modified: 20 Apr 2019, 18:09:14 UTC



1) The discussion in this thread about multi-threading confuses me. Does multithreading even work? If so, then how? At PrimeGrid, our wrapper is modified to specifically pass the "-t#" parameter to LLR, but in doing so it has to convert "-t#" to "-t #". Unless the wrapper here passes the parameter to LLR, and converts it to the correct format, I don't see how you could actually get multithreading to work. When I tried it, CPU temperatures dropped dramatically, indicating that only one core was being used.



it works the same way as it works at Primegrid

with -t8 my i7-5820k needs about 11-12 hours for one task


I stand corrected. PEBKAC

MT does indeed work.

Shadak
Send message
Joined: 29 Nov 18
Posts: 5
Credit: 629,213
RAC: 0
Message 5129 - Posted: 20 Apr 2019, 18:10:50 UTC - in response to Message 5126.
Last modified: 20 Apr 2019, 18:11:53 UTC



3) Maybe the deadline should be long enough to let computers without AVX to finish?


at least they should be realistic. long2 have a deadline of 12 days. thats 288hrs. the run time of long2 is avg 110.10 (min 0.62 - max 200.93) hrs
I'm not sure, but this is a much to low deadline, when you don't use tickle up. I have long2 running since 10 days which will reach soon the 90hr mark and which still have 3 more days till dead line so I hope I can finish them before someone else with multithreading finishes them...

but I decided not to run anymore long wus. the risk of getting no credits is much to high.
____________

Profile marmot
Avatar
Send message
Joined: 17 Nov 16
Posts: 97
Credit: 126,410,450
RAC: 21,911
Message 5131 - Posted: 21 Apr 2019, 2:19:23 UTC - in response to Message 5126.
Last modified: 21 Apr 2019, 2:21:28 UTC

When I tried it, CPU temperatures dropped dramatically, indicating that only one core was being used.


Noticed this also, there appears to be a 1 to 4 minute delay (depending on the computer's stats) where the long1 and long2 (haven't observed all WU's for the behavior but no noticeable lag on shorts) run a single core as the data set is being initialized.

Just mentioning so that anyone else maybe new to SRBase multithreading know about this lag and don't conclude they've made a mistake after seeing single core execution for 2 minutes.

[AF>Le_Pommier] Jerome_C2005
Send message
Joined: 1 Sep 16
Posts: 9
Credit: 1,493,204
RAC: 9,527
Message 5132 - Posted: 21 Apr 2019, 9:46:30 UTC

I just can't understand why the MT is not enabled / config by default and distributed with the standard app, it is just insane to try to run these monsters without the MT on.

Profile rebirther
Volunteer moderator
Project administrator
Project developer
Project tester
Project scientist
Avatar
Send message
Joined: 2 Jan 13
Posts: 7228
Credit: 42,729,227
RAC: 34
Message 5133 - Posted: 21 Apr 2019, 10:23:40 UTC - in response to Message 5132.

I just can't understand why the MT is not enabled / config by default and distributed with the standard app, it is just insane to try to run these monsters without the MT on.


Its only defined by the user, smaller runtimes are not affected by the -t command and are slower in runtime. Also every user has different amount of cores.

Profile Michael Goetz
Avatar
Send message
Joined: 1 Jan 15
Posts: 25
Credit: 1,197,466
RAC: 0
Message 5135 - Posted: 21 Apr 2019, 10:53:21 UTC - in response to Message 5131.

When I tried it, CPU temperatures dropped dramatically, indicating that only one core was being used.


Noticed this also, there appears to be a 1 to 4 minute delay (depending on the computer's stats) where the long1 and long2 (haven't observed all WU's for the behavior but no noticeable lag on shorts) run a single core as the data set is being initialized.

Just mentioning so that anyone else maybe new to SRBase multithreading know about this lag and don't conclude they've made a mistake after seeing single core execution for 2 minutes.


That was because I wasn't doing it correctly. When done right, it does work.

[AF>Le_Pommier] Jerome_C2005
Send message
Joined: 1 Sep 16
Posts: 9
Credit: 1,493,204
RAC: 9,527
Message 5136 - Posted: 21 Apr 2019, 18:40:04 UTC - in response to Message 5133.
Last modified: 21 Apr 2019, 18:43:30 UTC

I just can't understand why the MT is not enabled / config by default and distributed with the standard app, it is just insane to try to run these monsters without the MT on.


Its only defined by the user, smaller runtimes are not affected by the -t command and are slower in runtime. Also every user has different amount of cores.


There are many mt projects that provide tasks/programs able to detect and use the available cores with their mt apps depending on each member machine. Obviously here it would only be interesting for these "long" tasks, not all apps.

Having to play around with config file (and I remember this one is really not easy to do) really limits the potential number of users willing to try to do this.

Profile marmot
Avatar
Send message
Joined: 17 Nov 16
Posts: 97
Credit: 126,410,450
RAC: 21,911
Message 5138 - Posted: 21 Apr 2019, 23:42:59 UTC - in response to Message 5136.
Last modified: 21 Apr 2019, 23:48:42 UTC



Having to play around with config file (and I remember this one is really not easy to do) really limits the potential number of users willing to try to do this.


Agreed.

Longs really shouldn't be run on a 2 core machine, which would generally be 8 year old Core 2 Duos, so excluding them would be setting an appropriate boundary.

So assuming minimum machine is a 4 thread CPU, and the user might be wanting to surf and do other activities, longs could default to MT 3. This is how I set up my app_config. On 4 core machine it leaves 1 core open to capture the short, average or regulars. On 8 core machine, 2 longs run and leave 2 cores available for the other WU's or another project.

Having longs default to MT 4 is also a decent option given a majority of commonly owned CPU's thread counts are divisible by 4.

Certain BOINC server software version have the preferences ability to decide core count of multi thread work units. You had mentioned your server software was an older version, so maybe this option isn't available.

Profile Michael Goetz
Avatar
Send message
Joined: 1 Jan 15
Posts: 25
Credit: 1,197,466
RAC: 0
Message 5139 - Posted: 22 Apr 2019, 12:01:02 UTC - in response to Message 5138.

Longs really shouldn't be run on a 2 core machine, which would generally be 8 year old Core 2 Duos, so excluding them would be setting an appropriate boundary.


Most modern Core i3 CPUs are 2c/4t.

All but the highest end laptops tend to be 2c/4t. Most Mobile Core i5 CPUs are 2c/4t, unlike their desktop counterparts. Unless I'm mistaken, many mobile Core i7s are also 2c/4t.

You don't have to go back 8 years to find dual core CPUs in common use.

Shadak
Send message
Joined: 29 Nov 18
Posts: 5
Credit: 629,213
RAC: 0
Message 5142 - Posted: 23 Apr 2019, 18:05:24 UTC

so. all my long2 are now marked for resending...I'm 2-3 days away from finishing...
____________

Profile marmot
Avatar
Send message
Joined: 17 Nov 16
Posts: 97
Credit: 126,410,450
RAC: 21,911
Message 5145 - Posted: 24 Apr 2019, 4:52:28 UTC - in response to Message 5139.
Last modified: 24 Apr 2019, 5:05:34 UTC

Longs really shouldn't be run on a 2 core machine, which would generally be 8 year old Core 2 Duos, so excluding them would be setting an appropriate boundary.


Most modern Core i3 CPUs are 2c/4t.

All but the highest end laptops tend to be 2c/4t. Most Mobile Core i5 CPUs are 2c/4t, unlike their desktop counterparts. Unless I'm mistaken, many mobile Core i7s are also 2c/4t.

You don't have to go back 8 years to find dual core CPUs in common use.


When I said Core2 Duo, the implication was 2c/2t. But to be specific, 2c/2t machines. The benchmarks of 6th -> 9th gen 2c/4t are much greater than the Core2 Duos and can do the work of a long1 or long2, slowly.
BOINC will see 4 threads unless an owner actually turned off hyperthreading.

I wouldn't want to be doing long SRBase work units on even a modern 2c machine with hyper threading turned off and no available processing power to even use Chromium. LLR is pretty aggressive even set to idle priority.

BTW, only 1st gen i7 600 series were 2c/4t, all later have been 4+ real cores. 2nd, 3rd, 4th and 5th gen i3/i5s had quite a lot of low power 2 cores sold (desktop i5's were 4 core), 6th gen and later (after Jan 2015) are 2/4, 4 and 6.

Profile Michael Goetz
Avatar
Send message
Joined: 1 Jan 15
Posts: 25
Credit: 1,197,466
RAC: 0
Message 5146 - Posted: 24 Apr 2019, 13:44:24 UTC - in response to Message 5145.

Longs really shouldn't be run on a 2 core machine, which would generally be 8 year old Core 2 Duos, so excluding them would be setting an appropriate boundary.


Most modern Core i3 CPUs are 2c/4t.

All but the highest end laptops tend to be 2c/4t. Most Mobile Core i5 CPUs are 2c/4t, unlike their desktop counterparts. Unless I'm mistaken, many mobile Core i7s are also 2c/4t.

You don't have to go back 8 years to find dual core CPUs in common use.


When I said Core2 Duo, the implication was 2c/2t. But to be specific, 2c/2t machines. The benchmarks of 6th -> 9th gen 2c/4t are much greater than the Core2 Duos and can do the work of a long1 or long2, slowly.
BOINC will see 4 threads unless an owner actually turned off hyperthreading.

I wouldn't want to be doing long SRBase work units on even a modern 2c machine with hyper threading turned off and no available processing power to even use Chromium. LLR is pretty aggressive even set to idle priority.

BTW, only 1st gen i7 600 series were 2c/4t, all later have been 4+ real cores. 2nd, 3rd, 4th and 5th gen i3/i5s had quite a lot of low power 2 cores sold (desktop i5's were 4 core), 6th gen and later (after Jan 2015) are 2/4, 4 and 6.


We'll just have to agree to disagree. There's a lot in what you said that I completely disagree with.
____________
Want to find one of the largest known primes? Try PrimeGrid. Or help cure disease at WCG.

Profile marmot
Avatar
Send message
Joined: 17 Nov 16
Posts: 97
Credit: 126,410,450
RAC: 21,911
Message 5147 - Posted: 25 Apr 2019, 2:52:29 UTC - in response to Message 5146.
Last modified: 25 Apr 2019, 3:52:29 UTC

Longs really shouldn't be run on a 2 core machine, which would generally be 8 year old Core 2 Duos, so excluding them would be setting an appropriate boundary.


Most modern Core i3 CPUs are 2c/4t.

All but the highest end laptops tend to be 2c/4t. Most Mobile Core i5 CPUs are 2c/4t, unlike their desktop counterparts. Unless I'm mistaken, many mobile Core i7s are also 2c/4t.

You don't have to go back 8 years to find dual core CPUs in common use.


When I said Core2 Duo, the implication was 2c/2t. But to be specific, 2c/2t machines. The benchmarks of 6th -> 9th gen 2c/4t are much greater than the Core2 Duos and can do the work of a long1 or long2, slowly.
BOINC will see 4 threads unless an owner actually turned off hyperthreading.

I wouldn't want to be doing long SRBase work units on even a modern 2c machine with hyper threading turned off and no available processing power to even use Chromium. LLR is pretty aggressive even set to idle priority.

BTW, only 1st gen i7 600 series were 2c/4t, all later have been 4+ real cores. 2nd, 3rd, 4th and 5th gen i3/i5s had quite a lot of low power 2 cores sold (desktop i5's were 4 core), 6th gen and later (after Jan 2015) are 2/4, 4 and 6.


We'll just have to agree to disagree. There's a lot in what you said that I completely disagree with.


Unless you are referring to the earlier suggestions about default MT on LLR for SRBase long WU's, the only opinion I present in the last message was not wanting to run SRBase on even a modern 2 core machine with no hyper-threading.

The rest of my response was all factual taken from Intel database sources and sales figures presented in Wikipedia from tech articles.
All the machines I'm referring to are less than 6 GFlops (http://srbase.my-firewall.org/sr5/cpu_list.php) and it's unclear how many have completed a long WU in the last year (doubtful any completed one until the recent grace period was added). It would take some DB analysis.
EDIT: I see about 1200 of 61000 machines in the database that would be effected, most of which are likely not doing long WU's anyway, so that the rest of the workforce can see a 3x to 4x boost in speed. Not seeing a downside.

So, I assume you would love people to run SRBase on 2 core machines as that's the only thing you could be disagreeing with.

Previous · 1 · 2 · 3 · Next
Post to thread

Message boards : Number crunching : Long 2 are a joke


Main page · Your account · Message boards


Copyright © 2014-2024 BOINC Confederation / rebirther