Speed test using spinpack (under construction 2003/01/20)

Here you can find information, how to measure speed of your machine running spinpack.

First you have to download spinpack-2.15.tgz (or better version). Uncompress and untar the file, configure the Makefile, compile the sources and run the executable. Here is an example:

       gunzip -c spinpack-2.15.tgz | tar -xf -
       cd spinpack

       # --- small speed test --- (1CPU, MEM=113MB, DISK=840MB, nud=30,10)
       ./configure --nozlib
       make speed_test 
       sh -c "( cd ./exe; time ./spin ) 2>&1 | tee speed_test_small" 

       # --- big speed test --- (16CPUs, MEM=735MB, DISK=6GB, nud=28,12)
       ./configure --mpt --nozlib
       make speed_test; grep -v small exe/daten.i1 >exe/daten.i
       sh -c "( cd ./exe; time ./spin ) 2>&1 | tee speed_test_big"
   

Send me the output files together with the characteristic data of your computer for comparisions. Please also add the output of grep FLAGS= Makefile and cpu.log if you have.

Computation time

The next table gives an overview about computation time for a N=40 site system (used for speed test). First column marks the up- and down-spins (nud) given in daten.i. Other columns list the time needed for writing the matrix (SH) and for the first 40 iterations (i=40) showed by the output. The star (*) marks the default configuration when using make speed_test (see above). The double star is an example for the big speed_test (see above).

   nud     SH-time  i=40-time CPUs machine          time=[hh:]mm:ss(+-ss) dflt: v2.15 -O2
  -------+---------+---------+--+---------------------------------------
   32,8      2:18     5:47    1  SunFire-880-SparcIII-750MHz-g++2.95 v2.17 -mv8 -O2 lt=2:21
   32,8      1:37     3:42    1  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast    lt=1:03
   32,8      1:42(2)  6:07(3) 1  Celeron-1.1GHz-gcc -lgz v2.15 disk=22MB/s .
   32,8      0:50     2:04    1  Pentium4-2.5GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse B_NL2=4
   32,8      1:16     2:46    1  AthlonXP-1.7GHz-gcc-3.2 v2.17 -O2 -g -pg -march=athlon-xp -m3dnow (hda=7MB/s)
   32,8      1:03     2:21    1  AthlonXP-1.7GHz-gcc-3.2 v2.17 -O2 -march=athlon-xp -m3dnow (hda=55MB/s)
   32,8      1:51     6:35    1  Xeon-2GHz-v2.16-gcc-3.2 -O2 -march=i686 -msse2
   32,8      1:14     4:28    1  Xeon-2GHz-v2.17-gcc-3.2 -O2 -march=i686 -msse (-msse2 => asm-err)
   32,8      1:21     3:13    1  GS160-Alpha-731MHz-cxx v2.17 -fast -g3 -pg Compaq C++ V6.3-008
   32,8      4:39    20:09    1  O2100-MIPS-250MHz-CC v2.15 -lz -64 -O2
   32,8      2:45    18:02    2  O2100-MIPS-250MHz-CC v2.15 -lz -64 -O2
   32,8      1:08    18:25    8  O2100-MIPS-250MHz-CC v2.15 -lz -64 -O2  32 threads
   32,8      3:48    15:21    1  O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30)
   32,8      2:16     8:44    2  O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30)
   32,8      1:17     6:10    4  O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30)
   32,8      1:11     5:14    7  O2100-MIPS-250MHz-CC v2.17 -lz -64 -Ofast -IPA (MIPSPro v7.30)
   32,8      3:35     7:19    1  O2100-MIPS-250MHz-CC v2.17 -64 -O2          B_NL2=2  (MIPSPro v7.30)
   32,8      3:16     7:04    1  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA  B_NL2=2  (MIPSPro v7.30)
   32,8      3:40     7:07    1  O2100-MIPS-250MHz-CC v2.17v1 ssrun -64 -O2 -IPA  B_NL2=2  (MIPSPro v7.30) lt=2:02  gethblock=89s  latency???=81s/60*13e6=0.1us (unsorted)
   32,8      4:05     7:31    1  O2100-MIPS-250MHz-CC v2.17v3 ssrun -64 -O2 -IPA  B_NL2=2  (MIPSPro v7.30) lt=2:05  gethblock=91s  latency???=83s/60*13e6=0.1us (sorted)
   32,8      4:08     7:45    1  O2100-MIPS-250MHz-CC v2.17v2 ssrun -64 -O2 -IPA  B_NL2=2  (MIPSPro v7.30) lt=2:01  gethblock=103s latency???=89s/60*13e6=0.11us (XY_OLD)
   32,8      3:11     6:39    1  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA  B_NL2=0  (MIPSPro v7.30)
   32,8      3:13     5:56    1  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30)
   32,8      1:59     4:54    2  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30)
   32,8      1:23     4:36    4  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30)
   32,8      1:06     4:40    7  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lpthread (MIPSPro v7.30)
   ----------------------------
   30,10      23m      76m    1  Pentium-1.7GHz-gcc v2.15 -lgz
   30,10      21m      50m    1  Pentium-1.7GHz-gcc v2.15
   30,10    15:15    38:29    1  AthlonXP-1.7GHz-gcc-3.2 v2.17 -O2 -march=athlon-xp -m3dnow (hda=55MB/s)
   30,10    11:51    26:31    1  Pentium4-2.5GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse B_NL2=4
   30,10    15:59    40:09    1  Xeon-2GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse -g -pg
   30,10    14:26    34:08    1  Xeon-2GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse
   30,10     8:16    25:09    2  Xeon-2GHz-gcc-3.2 v2.17 -O2 -march=i686 -msse (slow nfs-disk)
   30,10     8:14    29:52    4  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast -lz (sun4u) lt=4:14  4 threads
   30,10    19:31  1:03:28    1  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast     (sun4u) lt=9:50 16 threads   2048s/40*168e6=0.30us
   30,10    27:28  1:14:28    1  SunFire-880-SparcIII-750MHz-g++2.95 v2.17 -mv8 -O2  (sun4u) lt=22:32 4 threads
   30,10     7:24    26:45    4  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast     (sun4u) lt=4:11  4 threads  4*910s/40*168e6=0.54us
   30,10     7:12    26:28    4  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast -O4 (sun4u) lt=4:05  4 threads  4*911s/40*168e6=0.54us
   30,10     3:44    16:58    8  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast     (sun4u) lt=4:23 16 threads  8*532s/40*168e6=0.63us
 * 30,10      24m      64m    1  GS160-Alpha-731MHz-cxx v2.15
   30,10    19:00    48:14    1  GS160-Alpha-731MHz-cxx v2.17 -fast -g3 -pg (42% geth_block, 27% b_smallest, 16% ifsmallest3)
   30,10    21:12    50:37    1  GS160-Alpha-731MHz-cxx v2.17 -fast
   30,10    19:36    59:44    1  GS160-Alpha-731MHz-cxx v2.17 -fast 16 threads
   30,10    12:15    36:16    2  GS160-Alpha-731MHz-cxx v2.17 -fast 16 threads
   30,10     8:24    24:17    3  GS160-Alpha-731MHz-cxx v2.17 -fast 16 threads
   30,10     8:21    25:24    3  GS160-Alpha-731MHz-cxx v2.17 -fast -pthread 16 threads
   30,10     7:40    26:36    3  GS160-Alpha-731MHz-cxx v2.17 -fast -pthread  4 threads
   30,10     7:48    53:00    10 GS160-Alpha-731MHz-cxx v2.15
   30,10     3:50    18:23    16 GS160-Alpha-731MHz-cxx v2.15 ( 64 threads)
   30,10     3:33    15:19    16 GS160-Alpha-731MHz-cxx v2.15 (128 threads)
   30,10     4:24    19:51    16 GS160-Alpha-731MHz-cxx v2.15 (128 threads, zlib)
   30,10  1:01:10  4:25:28    1  O2100-MIPS-250MHz-CC v2.15 -O3 -lz
   30,10    50:06  3:12:22    1  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lz
   30,10    30:14  2:00:42    2  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA -lz
   30,10    41:50  1:35:44    1  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA
   30,10    54:00  1:45:15    1  O2100-MIPS-250MHz-CC v2.17v3 ssrun -64 -O2 -IPA lt=00:20:33 geth_bl=2200s latency???=2030s/60*168e6=0.20us (XY_NEW+sortH)
   30,10    47:06  1:36:56    1  O2100-MIPS-250MHz-CC v2.17v3 ssrun -64 -O2 -IPA lt=00:20:40 geth_bl=2090s latency???=1928s/60*168e6=0.19us (XY_NEW)
   30,10    26:52  1:14:28    2  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA (HBLen=1024 about same)
   30,10    16:50  1:13:51    8  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA
   ----------------------------
   28,12      24h      40h    1  O2100-MIPS-250MHz-CC v1.4
   28,12  3:42:27 10:32:52    2  O2100-MIPS-250MHz-CC v2.17 -64 -Ofast -IPA
   28,12     171m       7h    1  Pentium-1.7GHz-gcc   v2.15
   28,12       5h      10h    1  GS160-Alpha-731MHz-cxx v2.15
   28,12  3:03:00 10:04:03    1  GS160-Alpha-731MHz-cxx v2.17pre -fast
   28,12  1:13:27  5:45:12    3  GS160-Alpha-731MHz-cxx v2.17 -fast -pthread 16
   28,12      81m     5.7h    9  GS160-Alpha-731MHz-cxx v2.15
** 28,12    57:39  5:29:57    16 GS160-Alpha-731MHz-cxx v2.15 (16  threads)
   28,12    59:22  2:51:54    16 GS160-Alpha-731MHz-cxx v2.15 (128 threads) .
   28,12    59:42  3:44:56    4  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast -lz (sun4u) lt=24:16  4 threads 4*250ns/access
   28,12    53:29  3:17:55    4  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast     (sun4u) lt=24:03  4 threads
   28,12    53:54  3:18:20    4  SunFire-880-SparcIII-750MHz-CC-5.3  v2.17 -fast -O4 (sun4u) lt=24:06  4 threads
   ----------------------------
   27,13       9h        -    1  GS160-Alpha-731MHz-cxx v2.15
   27,13      53h      96h    1  O2100-MIPS-250MHz-CC v1.4
   26,14     107h     212h    1  O2100-MIPS-250MHz-CC v1.4
   20,20      21h     515h    1  O2100-MIPS-250MHz-CC v1.4

Next figure shows the computing time for different older program versions and computers (I update it as soon as I can). The computing time depends nearly linearly from the matrix size n1 (time is proportional to n1^1.07, n1 is named n in the figure).

4kB png image of computing time

memory and disk usage

Memory usage depends from the matrix dimension n1. For the N=40 sample two double vectors and one 5-byte vector is stored in the memory, so we need n1*21 Bytes, where n1 is approximatly (N!/(nu!*nd!))/(4N). Disk usage is mainly the number of nonzero matrix elements hnz times 5 (disk size for tmp_l1.dat is 5*n1 and is not included here). The number of nonzero matrix elements hnz depends from n1 by hnz=10.4*x^1.07, which was found empirically. Here are some examples:

  nu,nd     n1   memory     hnz    disk  (zip)  (n1*21=memory, hnz*5=disk)
  -----+---------------+----------------------
  34,6    24e3    432kB   526e3   2.6MB 1.3MB 
  32,8   482e3     11MB    13e6    66MB  34MB
  30,10  5.3e6    113MB   168e6   840MB 444MB   small speed test
  28,12   35e6    735MB   1.2e9     6GB 3.6GB   big speed test
  27,13   75e6    1.4GB   2.8e9    14GB 
  26,14  145e6    2.6GB   5.5e9    28GB 
  20,20  431e6    7.8GB    18e9    90GB 
  
   

CPU load

A typical cpu load for a N=40 site system looks like this:

4kB png image of cpu-load

Data are generated using the following tiny script:

   #!/bin/sh
   while ps -o pid,pcpu,time,etime,cpu,user,args -p 115877;\
     do sleep 30; done | grep -v CPU
   

115877 is the PID of the process. You have to replace it. Alternativly you can activate a script activated by daten.i (edit it). The machine was used by 5 users, therefore peak load is only about 12CPUs. 735MB memory and 6GB diskspace were used. You can see the initialization process (20min), the matrix generation (57min) and the first 4 iterations (4x8min). The matrix generation is most dependend from CPU power. The iteration time mainly depends from the disk speed (try: time cat exe/tmp/ht* >/dev/null) and the speed of random memory access. You can improve disk speed using striped disks or files (AdvFS). The maximum number of threads was limited to 16, but this can be changed (see src/config.h).

Why multi-processor scaling is so bad?

During iterations the multi-processor scaling is so bad on most machines -- why? I guess, this is because of random read access to the vector a (see picture below). I thought a shared memory computer should not have such problems with scaling here, but probably I am wrong. In future I try to solve the problem.

6kB png image of dataflow Figure shows dataflow during iterations for 2 CPUs.