Here you can find information, how to measure speed of your machine running spinpack.
First you have to download spinpack-2.15.tgz (or better version). Uncompress and untar the file, configure the Makefile, compile the sources and run the executable. Here is an example:
gunzip -c spinpack-2.15.tgz | tar -xf -
cd spinpack
# --- small speed test --- (1CPU, MEM=113MB, DISK=840MB, nud=30,10)
./configure --nozlib
make speed_test
sh -c "( cd ./exe; time ./spin ) 2>&1 | tee speed_test_small"
# --- big speed test --- (16CPUs, MEM=735MB, DISK=6GB, nud=28,12)
./configure --mpt --nozlib
make speed_test; grep -v small exe/daten.i1 >exe/daten.i
sh -c "( cd ./exe; time ./spin ) 2>&1 | tee speed_test_big"
Send me the output files together with the characteristic data
of your computer for comparisions. Please also add the output of
grep FLAGS= Makefile and cpu.log if you have.
The next table gives an overview about computation time for a N=40 site system (used for speed test). First column marks the up- and down-spins (nud) given in daten.i. Other columns list the time needed for writing the matrix (SH) and for the first 40 iterations (i=40) showed by the output. The star (*) marks the default configuration when using make speed_test (see above). The double star is an example for the big speed_test (see above).
nud SH-time i=40-time CPUs machine time=[hh:]mm:ss(+-ss) dflt: v2.15 -O2 -------+---------+---------+--+--------------------------------------- 32,8 2:18 5:47 1 SunFire-880-SparcIII-750MHz-g++2.95 v2.17 -mv8 -O2 lt=2:21 32,8 1:37 3:42 1 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast lt=1:03 32,8 1:42(2) 6:07(3) 1 Celeron-1.1GHz-gcc -lgz v2.15 disk=22MB/s . 32,8 0:50 2:04 1 Pentium4-2.5GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse B_NL2=4 32,8 1:16 2:46 1 AthlonXP-1.7GHz-gcc-3.2 v2.17 -O2 -g -pg -march=athlon-xp -m3dnow (hda=7MB/s) 32,8 1:03 2:21 1 AthlonXP-1.7GHz-gcc-3.2 v2.17 -O2 -march=athlon-xp -m3dnow (hda=55MB/s) 32,8 1:51 6:35 1 Xeon-2GHz-v2.16-gcc-3.2 -O2 -march=i686 -msse2 32,8 1:14 4:28 1 Xeon-2GHz-v2.17-gcc-3.2 -O2 -march=i686 -msse (-msse2 => asm-err) 32,8 1:21 3:13 1 GS160-Alpha-731MHz-cxx v2.17 -fast -g3 -pg Compaq C++ V6.3-008 32,8 4:39 20:09 1 O2100-MIPS-250MHz-CC v2.15 -lz -64 -O2 32,8 2:45 18:02 2 O2100-MIPS-250MHz-CC v2.15 -lz -64 -O2 32,8 1:08 18:25 8 O2100-MIPS-250MHz-CC v2.15 -lz -64 -O2 32 threads 32,8 3:48 15:21 1 O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30) 32,8 2:16 8:44 2 O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30) 32,8 1:17 6:10 4 O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30) 32,8 1:11 5:14 7 O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30) 32,8 3:35 7:19 1 O2100-MIPS-250MHz-CC v2.17 -64 -O2 B_NL2=2 (MIPSPro v7.30) 32,8 3:16 7:04 1 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA B_NL2=2 (MIPSPro v7.30) 32,8 3:40 7:07 1 O2100-MIPS-250MHz-CC v2.17v1 ssrun -64 -O2 -IPA B_NL2=2 (MIPSPro v7.30) lt=2:02 gethblock=89s latency???=81s/60*13e6=0.1us (unsorted) 32,8 4:05 7:31 1 O2100-MIPS-250MHz-CC v2.17v3 ssrun -64 -O2 -IPA B_NL2=2 (MIPSPro v7.30) lt=2:05 gethblock=91s latency???=83s/60*13e6=0.1us (sorted) 32,8 4:08 7:45 1 O2100-MIPS-250MHz-CC v2.17v2 ssrun -64 -O2 -IPA B_NL2=2 (MIPSPro v7.30) lt=2:01 gethblock=103s latency???=89s/60*13e6=0.11us (XY_OLD) 32,8 3:11 6:39 1 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA B_NL2=0 (MIPSPro v7.30) 32,8 3:13 5:56 1 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30) 32,8 1:59 4:54 2 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30) 32,8 1:23 4:36 4 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30) 32,8 1:06 4:40 7 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30) ---------------------------- 30,10 23m 76m 1 Pentium-1.7GHz-gcc v2.15 -lgz 30,10 21m 50m 1 Pentium-1.7GHz-gcc v2.15 30,10 15:15 38:29 1 AthlonXP-1.7GHz-gcc-3.2 v2.17 -O2 -march=athlon-xp -m3dnow (hda=55MB/s) 30,10 11:51 26:31 1 Pentium4-2.5GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse B_NL2=4 30,10 15:59 40:09 1 Xeon-2GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse -g -pg 30,10 14:26 34:08 1 Xeon-2GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse 30,10 8:16 25:09 2 Xeon-2GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse (slow nfs-disk) 30,10 8:14 29:52 4 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast -lz (sun4u) lt=4:14 4 threads 30,10 19:31 1:03:28 1 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast (sun4u) lt=9:50 16 threads 2048s/40*168e6=0.30us 30,10 27:28 1:14:28 1 SunFire-880-SparcIII-750MHz-g++2.95 v2.17 -mv8 -O2 (sun4u) lt=22:32 4 threads 30,10 7:24 26:45 4 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast (sun4u) lt=4:11 4 threads 4*910s/40*168e6=0.54us 30,10 7:12 26:28 4 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast -O4 (sun4u) lt=4:05 4 threads 4*911s/40*168e6=0.54us 30,10 3:44 16:58 8 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast (sun4u) lt=4:23 16 threads 8*532s/40*168e6=0.63us * 30,10 24m 64m 1 GS160-Alpha-731MHz-cxx v2.15 30,10 19:00 48:14 1 GS160-Alpha-731MHz-cxx v2.17 -fast -g3 -pg (42% geth_block, 27% b_smallest, 16% ifsmallest3) 30,10 21:12 50:37 1 GS160-Alpha-731MHz-cxx v2.17 -fast 30,10 19:36 59:44 1 GS160-Alpha-731MHz-cxx v2.17 -fast 16 threads 30,10 12:15 36:16 2 GS160-Alpha-731MHz-cxx v2.17 -fast 16 threads 30,10 8:24 24:17 3 GS160-Alpha-731MHz-cxx v2.17 -fast 16 threads 30,10 8:21 25:24 3 GS160-Alpha-731MHz-cxx v2.17 -fast -pthread 16 threads 30,10 7:40 26:36 3 GS160-Alpha-731MHz-cxx v2.17 -fast -pthread 4 threads 30,10 7:48 53:00 10 GS160-Alpha-731MHz-cxx v2.15 30,10 3:50 18:23 16 GS160-Alpha-731MHz-cxx v2.15 ( 64 threads) 30,10 3:33 15:19 16 GS160-Alpha-731MHz-cxx v2.15 (128 threads) 30,10 4:24 19:51 16 GS160-Alpha-731MHz-cxx v2.15 (128 threads, zlib) 30,10 1:01:10 4:25:28 1 O2100-MIPS-250MHz-CC v2.15 -O3 -lz 30,10 50:06 3:12:22 1 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lz 30,10 30:14 2:00:42 2 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lz 30,10 41:50 1:35:44 1 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA 30,10 54:00 1:45:15 1 O2100-MIPS-250MHz-CC v2.17v3 ssrun -64 -O2 -IPA lt=00:20:33 geth_bl=2200s latency???=2030s/60*168e6=0.20us (XY_NEW+sortH) 30,10 47:06 1:36:56 1 O2100-MIPS-250MHz-CC v2.17v3 ssrun -64 -O2 -IPA lt=00:20:40 geth_bl=2090s latency???=1928s/60*168e6=0.19us (XY_NEW) 30,10 26:52 1:14:28 2 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA (HBLen=1024 about same) 30,10 16:50 1:13:51 8 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA ---------------------------- 28,12 24h 40h 1 O2100-MIPS-250MHz-CC v1.4 28,12 3:42:27 10:32:52 2 O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA 28,12 171m 7h 1 Pentium-1.7GHz-gcc v2.15 28,12 5h 10h 1 GS160-Alpha-731MHz-cxx v2.15 28,12 3:03:00 10:04:03 1 GS160-Alpha-731MHz-cxx v2.17pre -fast 28,12 1:13:27 5:45:12 3 GS160-Alpha-731MHz-cxx v2.17 -fast -pthread 16 28,12 81m 5.7h 9 GS160-Alpha-731MHz-cxx v2.15 ** 28,12 57:39 5:29:57 16 GS160-Alpha-731MHz-cxx v2.15 (16 threads) 28,12 59:22 2:51:54 16 GS160-Alpha-731MHz-cxx v2.15 (128 threads) . 28,12 59:42 3:44:56 4 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast -lz (sun4u) lt=24:16 4 threads 4*250ns/access 28,12 53:29 3:17:55 4 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast (sun4u) lt=24:03 4 threads 28,12 53:54 3:18:20 4 SunFire-880-SparcIII-750MHz-CC-5.3 v2.17 -fast -O4 (sun4u) lt=24:06 4 threads ---------------------------- 27,13 9h - 1 GS160-Alpha-731MHz-cxx v2.15 27,13 53h 96h 1 O2100-MIPS-250MHz-CC v1.4 26,14 107h 212h 1 O2100-MIPS-250MHz-CC v1.4 20,20 21h 515h 1 O2100-MIPS-250MHz-CC v1.4
Next figure shows the computing time for different older program versions and computers (I update it as soon as I can). The computing time depends nearly linearly from the matrix size n1 (time is proportional to n1^1.07, n1 is named n in the figure).
Memory usage depends from the matrix dimension n1. For the N=40 sample two double vectors and one 5-byte vector is stored in the memory, so we need n1*21 Bytes, where n1 is approximatly (N!/(nu!*nd!))/(4N). Disk usage is mainly the number of nonzero matrix elements hnz times 5 (disk size for tmp_l1.dat is 5*n1 and is not included here). The number of nonzero matrix elements hnz depends from n1 by hnz=10.4*x^1.07, which was found empirically. Here are some examples:
nu,nd n1 memory hnz disk (zip) (n1*21=memory, hnz*5=disk) -----+---------------+---------------------- 34,6 24e3 432kB 526e3 2.6MB 1.3MB 32,8 482e3 11MB 13e6 66MB 34MB 30,10 5.3e6 113MB 168e6 840MB 444MB small speed test 28,12 35e6 735MB 1.2e9 6GB 3.6GB big speed test 27,13 75e6 1.4GB 2.8e9 14GB 26,14 145e6 2.6GB 5.5e9 28GB 20,20 431e6 7.8GB 18e9 90GB
A typical cpu load for a N=40 site system looks like this:
Data are generated using the following tiny script:
#!/bin/sh
while ps -o pid,pcpu,time,etime,cpu,user,args -p 115877;\
do sleep 30; done | grep -v CPU
115877 is the PID of the process. You have to replace it.
Alternativly you can activate a script activated by daten.i (edit it).
The machine was used by 5 users, therefore peak load is only
about 12CPUs. 735MB memory and 6GB diskspace were used.
You can see the initialization process (20min),
the matrix generation (57min) and the first 4 iterations (4x8min).
The matrix generation is most dependend from CPU power.
The iteration time mainly depends from the disk speed (try: time cat exe/tmp/ht* >/dev/null) and the
speed of random memory access. You can improve
disk speed using striped disks or files (AdvFS). The maximum number
of threads was limited to 16, but this can be changed (see src/config.h).
Figure shows dataflow during iterations for 2 CPUs.