<p style="margin-bottom: 0cm;">Hello,</p><p style="margin-bottom: 0cm;">I have a performance problem when using mvapich2 compiled with BLCR support <br></p>on infiniband cluster with following parameters:<br><p style="margin-bottom: 0cm;">
Node: 2xQuad Core Intel Xeon 2.33 GHz<br>O/S: RHEL4.5<br>File System: GPFS<br>We are using MVAPICH2-1.0.2p1 with BLCR-0.6.5.</p><br>I've done 3 test runs of my program using 8 MPI processes:<br>1) All of 8 processes on one node<br>
2) by 4 processes on two nodes<br>3) by 2 processes on 4 nodes<br><br><pre style="margin-bottom: 0.5cm;"><i>Results MVAPICH2 configured for BLCR support:</i><br><i>[ccs-dev@n5304]$ mpiexec -machinefile ./mf1 -np 8 ./test</i><br>
<br><i>Calc time: 341.3279, send/recv time = 297.817</i><br><i>[ccs-dev@n5304]$ mpiexec -machinefile ./mf2 -np 8 ./test</i><br><i>Calc time: 85.7075, send/recv time = 42.2270</i><br><i>[ccs-dev@n5304]$ mpiexec -machinefile ./mf3 -np 8 ./test</i><br>
<br><i>Calc time: 84.6182, send/recv time = 40.3554</i><br><br><i>Results MVAPICH2 configured without BLCR support:</i><br><i>[ccs-dev@n5304]$ mpiexec -machinefile ./mf1 -np 8 ./test</i><br><i>Calc time: 51.5888, send/recv time = 8.0186</i><br>
<br><i>[ccs-dev@n5304]$ mpiexec -machinefile ./mf2 -np 8 ./test</i><br><i>Calc time: 53.6679, send/recv time = 10.1187</i><br><i>[ccs-dev@n5304]$ mpiexec -machinefile ./mf3 -np 8 ./test</i><br><i>Calc time: 63.6611, send/recv time = 20.0127<br>
<br></i></pre>So when using MVAPICH2 configured with BLCR support I have much<br>time which is spent on communication between processes.<br>Is it concerned with the fact of shared-memory support automatic disabling in such build?<br>
If it is so, do you plan to include support of both BLCR & shared-memory communications in future releases?<br>And maybe there are another ways to improve performance of MPI program running on multi-core node?<br><br>
<br>Thanks.<br><span class="sg"><br>Maya</span>