v5.22 Experimental optimized app

Message boards : Number crunching : v5.22 Experimental optimized app

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 13 · Next

AuthorMessage
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 150 - Posted: 14 Nov 2007, 14:14:05 UTC
Last modified: 19 Feb 2013, 18:25:43 UTC

The project is open source, it's relatively easy to compile your own executable and wrapper. Enigma source can be found at http://www.bytereef.org and the wrapper is BOINC default one (it comes with BOINC sources inside boinc/samples); wrapper has no effect on speed so you can use the default one or one of my upgraded wrappers with working progress indicator to run your app. Executable built for specific system/hardware usually is much faster (from 15-20% up, in some cases custom build app can be twice as fast) than the default one, so if you feel you can do it, try it.

Linux: I've tried compiling enigma-suite with gcc versions from 3.2 up to the latest, 3.2 was the fastest one when I used only safe cflags for specific processors, however latest gcc can be faster if you add more cflags for advanced optimizations. If you have Intel processor, Intel C/C++ compiler is recommended. It's free for personal use and the results are impressive - optimized executable can be up to twice as fast as the default app.

Windows: It's relatively easy to compile enigma with dev-cpp (just create an empty windows console app c project and add all source files), then you can play with compiler options. It's also possible to replace the compiler included in dev-cpp package with newer MinGW 5.1.x which gives some minor speed improvements - I've done that by replacing all compiler binaries.
Building enigma suite with Visual Studio and/or Visual C++ is a bit harder, because you'll have to add additional header files to add support for getopt.



As alternative, you can try one of these precompiled executables:

Please use it only if you know what you are doing. It's very easy to trash large numbers of workunits if you do something wrong.

Average speed increase is at about 15-20%, in some cases up to 30%.

How to switch app
download two archives:

Windows:

Last update: 19.02.2013
http://dl.getdropbox.com/u/1912652/c2d.zip - a newer version of app_info.xml, contains only the new app (the old one is unused). The enigma executable packed in this archive is an Intel C2D executable for Win32 (runs also on x64), if you want to run it on other processor, replace the executable with other from the archive below:

https://dl.dropbox.com/u/1912652/exe.zip - pack of recompiled executables, use one from this archive if the c2d version does not work for you.
http://dl.getdropbox.com/u/1912652/enigma_076_windows_intelx86.exe - executable for Intel Atom.

My old Windows benchmark http://dl.getdropbox.com/u/1912652/win_bench.7z

It uses compiled AutoIt script to run and control enigma executable; there's no progress or GUI, only AutoIt tray icon while it's running. When finished, it shows the CPU time (in seconds) in a small window.
How to use: replace the enigma.exe with the executable you want to test and click benchmark.exe


Linux:

Last update: 18.02.2013
https://dl.dropbox.com/u/1912652/app_test_522.tgz - app xml with all files
https://dl.dropbox.com/u/1912652/enigma_athlon64_64bit.tgz - 64bit executable for Athlon 64/Athlon 64 X2.
http://tjm.boo.pl/enigma/test.tgz - pack of execs compiled with gcc 3.2

Linux users: it's very easy to compile your own executable, either with gcc or Intel C/C++ compiler (which is free for personal use). Read posts below.

How to switch app:

Shutdown BOINC, open first archive and replace files in project folder:



then open second archive, pick one executable, and replace the old one in project folder.





When you start BOINC again, it should resume previous tasks with new app.
M4 Project homepage
M4 Project wiki
ID: 150 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rebirther
Avatar

Send message
Joined: 10 Sep 07
Posts: 18
Credit: 2,832,564
RAC: 0
Message 151 - Posted: 14 Nov 2007, 14:15:14 UTC - in response to Message 150.  

Why do you not using the option "Run test applications?" for further testing?
You can add this to the Enigma preferences, Boinc doesnt need the app_info.xml anymore.

Edit:
Copied all in folder, restarted Boinc resuming only from zero, waiting to finish something. (C2D)

Edit2:
finished some tasks, speedup around 5-18% on a C2D. Validation ok.
Restart with Boinc from zero took only the half time, so checkpointing is working but the time is not running from the last end point.
ID: 151 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 152 - Posted: 14 Nov 2007, 14:16:02 UTC - in response to Message 151.  

Rebirther wrote:

Edit2:
finished some tasks, speedup around 5-18% on a C2D. Validation ok.
Restart with Boinc from zero took only the half time, so checkpointing is working but the time is not running from the last end point.


Yep, that's a known problem - checkpointing works, but CPU time is not saved. It's already on the to-do list, not working progress indicator is also there.
I was going to work on this last weekend, but I had to fix some minor bugs on the server side first.


M4 Project homepage
M4 Project wiki
ID: 152 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile rebirther
Avatar

Send message
Joined: 10 Sep 07
Posts: 18
Credit: 2,832,564
RAC: 0
Message 153 - Posted: 14 Nov 2007, 14:16:20 UTC - in response to Message 152.  

The application for C2D does a great job, no problems seen, if any other with different cpu can report success the new application can go public!



ID: 153 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Anonymous
Avatar

Send message
Joined: 11 Sep 07
Posts: 19
Credit: 474
RAC: 0
Message 154 - Posted: 14 Nov 2007, 14:20:05 UTC - in response to Message 153.  

I noticed that when I installed the optimized app for Pentium 4 on my computer, the "Application" column in BOINC Manager shows "enigma_m4_naval_2" instead of "Enigma 0.76 5.17" like it did before. I guess it probably doesn't matter--and it could be a nice way to tell if you're using the optimized app or not--but if you want to, I think you can fix it by including the following line in the app_info.xml file:

<user_friendly_name>Enigma 0.76 5.17</user_friendly_name>

It goes right underneath the line near the beginning of the file that says:
<name>enigma_m4_naval_2</name>


If you'd still like to retain the ability to tell whether you're using the optimized app or not at a glance, then you can, instead of typing "Enigma 0.76 5.17" for the <user_friendly_name> value, type "Optimized Enigma 0.76 5.17".

ID: 154 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
[B^S] Ed Dillio

Send message
Joined: 10 Sep 07
Posts: 1
Credit: 33,326
RAC: 0
Message 155 - Posted: 14 Nov 2007, 14:21:51 UTC - in response to Message 154.  

It is working good for me on my P4 computer
ID: 155 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 156 - Posted: 14 Nov 2007, 14:22:59 UTC - in response to Message 154.  
Last modified: 14 Nov 2007, 14:23:20 UTC

Anonymous wrote:

<user_friendly_name>Enigma 0.76 5.17</user_friendly_name>

It goes right underneath the line near the beginning of the file that says:
<name>enigma_m4_naval_2</name>



5.17 is added by BOINC, I've just set user-friendly app name to Enigma 0.76-Opt.
What's the speed increase when running optimized app on p4 ? On my Celeron D 2.13@3.65 it was up to 30% faster last time I checked.
M4 Project homepage
M4 Project wiki
ID: 156 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tom Philippart

Send message
Joined: 15 Sep 07
Posts: 9
Credit: 44,303
RAC: 0
Message 157 - Posted: 14 Nov 2007, 14:23:47 UTC - in response to Message 156.  

41% faster on my Athlon 64 X2!

Thanks for this great app! :D
ID: 157 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 158 - Posted: 14 Nov 2007, 14:25:53 UTC - in response to Message 157.  

Tom Philippart wrote:
41% faster on my Athlon 64 X2!


Are you sure ? Maybe it was just different work unit lenght ?
There are 4 types of work units, type0 is short (1h 03 minutes on my PIV 1,5) , 1 and 2 are medium length (2h 07m and 2h 36m) and type 3 is the longest (4h 24m).

EDIT:
To check work unit type, take a look at it's name



enter it's unique ID here: http://kfd.no-ip.org/enigma/wu_lookup.php
and check -t column.



for newer work units (ID higher than 215k) the first number is work unit type.




M4 Project homepage
M4 Project wiki
ID: 158 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tom Philippart

Send message
Joined: 15 Sep 07
Posts: 9
Credit: 44,303
RAC: 0
Message 159 - Posted: 14 Nov 2007, 14:26:15 UTC - in response to Message 158.  

The WUs I crunched with the new client are all type 0, the older ones don't say anything in the type field

ID: 159 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Todd Dawson

Send message
Joined: 19 Sep 07
Posts: 1
Credit: 15,750
RAC: 0
Message 160 - Posted: 14 Nov 2007, 14:27:15 UTC - in response to Message 159.  

computer smart I`m not can I detech and get the optimized app?
some help
dumb farm boy


ID: 160 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 161 - Posted: 14 Nov 2007, 14:27:41 UTC - in response to Message 160.  

Nope, optimized app cannot be installed as default, because it doesn't work on all systems/hardware configurations.

M4 Project homepage
M4 Project wiki
ID: 161 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Viking69
Avatar

Send message
Joined: 9 Oct 07
Posts: 3
Credit: 102,079
RAC: 0
Message 243 - Posted: 20 Nov 2007, 11:23:29 UTC - in response to Message 161.  

how does this optimized app do with all the other BOINC projects? Is the same improvement seen in these or is there a decrease or no change in performance on everything but enigma?

Have your optimizasions been approved but the Berkley team?

What's UP ?
ID: 243 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 244 - Posted: 20 Nov 2007, 11:23:52 UTC - in response to Message 243.  

It's application optimization, not the BOINC client, so it has no effects on other projects.
M4 Project homepage
M4 Project wiki
ID: 244 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
zombie67 [MM]
Avatar

Send message
Joined: 2 Sep 07
Posts: 25
Credit: 15,424,373
RAC: 0
Message 245 - Posted: 20 Nov 2007, 11:24:24 UTC - in response to Message 244.  

TJM wrote:

Nope, optimized app cannot be installed as default, because it doesn't work on all systems/hardware configurations.


How about enabling the "Run test applications?" setting in the project preferences?  You could set it up as a test application, and just rename the setting to something like "check this if your processor supports SSE2 (or whatever is the requirement).
Dublin, CA
Team SETI.USA
ID: 245 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile TJM
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 25 Aug 07
Posts: 843
Credit: 267,994,998
RAC: 0
Message 246 - Posted: 20 Nov 2007, 11:24:55 UTC - in response to Message 245.  

Then I'd have to run 5 test apps (for Athlon 64, Athlon TB/Athlon XP, C2D, PIV/Celeron D, Coppermine/Tualatin), unless there's an option to replace executables on client side. If a test app requires it's own validator, then it would be even worse, because running 6 validators (or 12 if I add another app) is a waste of server resources.
Btw these apps are not really optimized, just compiled with different -march settings. It's not the best way to optimize apps, but the speed increase is significant (usually between 15 and 40%, depends on processor).

M4 Project homepage
M4 Project wiki
ID: 246 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
zombie67 [MM]
Avatar

Send message
Joined: 2 Sep 07
Posts: 25
Credit: 15,424,373
RAC: 0
Message 247 - Posted: 20 Nov 2007, 11:25:23 UTC - in response to Message 246.  

TJM wrote:

Then I'd have to run 5 test apps (for Athlon 64, Athlon TB/Athlon XP, C2D, PIV/Celeron D, Coppermine/Tualatin), unless there's an option to replace executables on client side. If a test app requires it's own validator, then it would be even worse, because running 6 validators (or 12 if I add another app) is a waste of server resources.
Btw these apps are not really optimized, just compiled with different -march settings. It's not the best way to optimize apps, but the speed increase is significant (usually between 15 and 40%, depends on processor).


Ah right, different flavors.  Forgot about that.  Ah well.
Dublin, CA
Team SETI.USA
ID: 247 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Anonymous
Avatar

Send message
Joined: 11 Sep 07
Posts: 19
Credit: 474
RAC: 0
Message 248 - Posted: 20 Nov 2007, 11:26:13 UTC - in response to Message 247.  

TJM wrote:

Then I'd have to run 5 test apps (for Athlon 64, Athlon TB/Athlon XP, C2D, PIV/Celeron D, Coppermine/Tualatin), unless there's an option to replace executables on client side. If a test app requires it's own validator, then it would be even worse, because running 6 validators (or 12 if I add another app) is a waste of server resources.
Btw these apps are not really optimized, just compiled with different -march settings. It's not the best way to optimize apps, but the speed increase is significant (usually between 15 and 40%, depends on processor).

BOINC server software for a while has had a feature where you can insert multiple applications, assigning each one an ID number, and you can have it display in the project preferences checkboxes for each one (rather than just a simple beta apps or no beta apps option). If you want to see this in action, you can see PrimeGrid--they're the only project I've seen that uses it (I'm sure there's others out there, but since I haven't participated in them, I wouldn't know.) They have 6 different applications there, so by checking the different boxes, you can decide which mix of work you get (you get a mix of the ones you've selected--the project administrator must set "weights" for each application, sort of like resource shares, which determines how heavy the mix is on one app or another--useful if some apps' tasks take longer than others.) If such a system was implemented here, you'd want to set the non-optimized one to be the only one checked by default for new users, and then add a warning to the project preferences page saying to make sure to only select one, or else you'll be getting a mix of work optimized for different processors.  :)

Unfortunately, though, I think BOINC requires separate pools of work for each "application" definined for the project (since usually this is used to determine between different types of work, not different types of optimizations for the same app)--I guess you'd just have to allocate some of the work for different processor types, then. (As long as you've got enough work in the system to keep the computers using each optimization from running dry on their "pool" of work, you'd be OK, though.) You could do that a few different ways: maybe simply take a mix of all the different types of workunits (i.e. the ones with hceyz72_0_, _1_, _2_, and _3_ respectively), or assign certain types to certain processors (maybe assign all the _3_'s to Core 2's, since they take the longest, and Core 2's would finish them the quickest).

It wouldn't be a perfect solution, but it would probably be better than the traditional system of manual executable replacement and app_info.xml files.
ID: 248 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile tahanko

Send message
Joined: 21 Oct 07
Posts: 3
Credit: 37,532
RAC: 0
Message 287 - Posted: 24 Nov 2007, 19:57:36 UTC
Last modified: 24 Nov 2007, 19:59:05 UTC

hello
are you working on a new optimized application or this is the only optimalization which can be done with the code, because on "my" C2D its about 20% faster than the non-optimized app.
thanks for your answer
ID: 287 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Crunch3r

Send message
Joined: 22 Nov 07
Posts: 5
Credit: 151,336
RAC: 0
Message 289 - Posted: 24 Nov 2007, 20:13:45 UTC - in response to Message 287.  

hello
are you working on a new optimized application or this is the only optimalization which can be done with the code, because on "my" C2D its about 20% faster than the non-optimized app.
thanks for your answer


Well, i can tell you one thing, that there should be a 64 bit linux and a 64 bit windows app. Those a currently beeing tested.

About other optimized applications, we'll see what the outcome of those tests are ;)

HTH

Crunch3r


ID: 289 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · 3 · 4 . . . 13 · Next

Message boards : Number crunching : v5.22 Experimental optimized app




Copyright © 2024 TJM