<p style="font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);" class="MsoNormal">Hi all,</p>
<p class="MsoNormal" style="margin-bottom: 12pt; font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);"> <br><b><span style="font-size: 10pt;">I got some problem, can someone
help me on this issue.</span></b></p>
<p class="MsoNormal" style="margin-bottom: 12pt; font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);"><b><span style="font-size: 10pt;">The scenario is : We have a
Rocks(4.2) cluster with 12 nodes. We installed Infiniband cards newly in 5
nodes(Masternode doesn't have IB card). Installation of OFED is successful and
IP got assigned.</span></b></p>
<p class="MsoNormal" style="margin-bottom: 12pt; font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);"><b><span style="font-size: 10pt;">I installed Mvapich2 in that and
created password free environment from computenode-0-8 to 12(the nodes which
have IB card). So far everything is fine, And the MPD is booting up also.
</span></b></p>
<p class="MsoNormal" style="margin-bottom: 12pt; font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);"><b><span style="font-size: 10pt;">I've compiled a sample MPI
program and tried to execute it and I got the following kind of results:</span></b></p>
<p class="MsoNormal" style="margin-bottom: 12pt; font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);">Scenario
1: Using root to execute Hellow.o (compiled with mvapich2-mpicc)<br>
<br>
[root@compute-0-8 test]# /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o<br>
Hello world from process 0 of 2<br>
Hello world from process 1 of 2<br>
rank 1 in job 8 compute-0-8.local_34399 caused collective
abort of all ranks<br>
exit status of rank 1: killed by signal 9<br>
rank 0 in job 8 compute-0-8.local_34399 caused collective
abort of all ranks<br>
exit status of rank 0: killed by signal 9<span style="font-size: 12pt;"></span></p>
<p style="font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);" class="MsoNormal">Scenario 2: Using user id (srinu) to
execute the same file.<br>
<br>
[srinu@compute-0-8 test]$ /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o<br>
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.<br>
This will severely limit memory registrations.<br>
libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.<br>
This will severely limit memory registrations.<br>
Fatal error in MPI_Init:<br>
Other MPI error, error stack:<br>
MPIR_Init_thread(259)....: Initialization failed<br>
MPID_Init(102)...........: channel initialization failed<br>
MPIDI_CH3_Init(178)......:<br>
MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type<br>
rdma_iba_hca_init(645)...: cannot create cq<br>
Fatal error in MPI_Init:<br>
Other MPI error, error stack:<br>
MPIR_Init_thread(259)....: Initialization failed<br>
MPID_Init(102)...........: channel initialization failed<br>
MPIDI_CH3_Init(178)......:<br>
MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type<br>
rdma_iba_hca_init(645)...: cannot create cq<br>
rank 1 in job 9 compute-0-8.local_34399 caused collective
abort of all ranks<br>
exit status of rank 1: return code 1</p>
<p style="font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);" class="MsoNormal">For 2<sup>nd</sup> scenario, I found solution from net
such as ulimit –l unlimited. But further, this also produced same error as of 1<sup>st</sup>
scenario.</p><span style="font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);">Can someone solve this error?<br><br>Thanks in advance,</span><p class="MsoNormal"><span style="font-family: trebuchet ms,sans-serif; color: rgb(51, 51, 255);">Sangamesh</span><br>
</p><br><p class="MsoNormal"><br></p><p class="MsoNormal"><br></p>