Hi,<BR> <BR>MVAPICH2 is supposed to detect the active port automatically for you. Could you try the following options:<BR> <BR>$ mpiexec -n 2 -env MV2_IBA_HCA mthca1 -env MV2_DEFAULT_PORT 1 ./a.out<BR> <BR>and see if it works for you?<BR> <BR>Lei<BR><BR><BR>----- Original Message -----<BR>From: biswajit@crlindia.com<BR>Date: Tuesday, June 17, 2008 6:55 am<BR>Subject: [mvapich-discuss] problem with running mvapich<BR>To: mvapich-discuss@cse.ohio-state.edu<BR><BR><BR><FONT face=sans-serif size=2><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>When I ran a simple MPI application with mvapich2-1.0.2, I got the following error messages:</FONT> <BR><BR><FONT face=sans-serif size=2><I> </I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>Unknown Mellanox PCI-Express HCA best guess as Mellanox PCI-Express SDR</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>[3] Abort: Not enough ports are in active stateneeded active ports 1</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> at line 424 in file rdma_iba_priv.c</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>rank 3 in job 1 n23_32790 caused collective abort of all ranks</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> exit status of rank 3: return code 252</I></FONT> <BR><BR><FONT face=sans-serif size=2><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>But there is a active port in each node. See the below <I>'ibstat' </I>output.</FONT> <BR><BR><BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>CA 'mthca0'</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> CA type: MT25204</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Number of ports: 1</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Firmware version: 1.1.0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Hardware version: a0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Node GUID: 0x0019bbfffff70cb8</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> System image GUID: 0x0019bbfffff70cbb</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Port 1:</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> State: Down</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Physical state: Polling</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Rate: 10</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Base lid: 0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> LMC: 0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> SM lid: 0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Capability mask: 0x02510a68</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Port GUID: 0x0019bbfffff70cb9</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>CA 'mthca1'</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> CA type: MT25204</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Number of ports: 1</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Firmware version: 1.1.0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Hardware version: a0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Node GUID: 0x0019bbfffff7fbe8</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> System image GUID: 0x0019bbfffff7fbeb</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Port 1:</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> State: Active</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Physical state: LinkUp</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Rate: 20</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Base lid: 226</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> LMC: 0</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> SM lid: 117</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Capability mask: 0x02510a68</I></FONT> <BR><FONT face=sans-serif size=2><I><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> Port GUID: 0x0019bbfffff7fbe9</I></FONT> <BR><BR><FONT face=sans-serif size=2><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT> And, whenever I run same job in nodes with IB port 1 active, it works properly.</FONT> <BR><FONT face=sans-serif size=2><FONT style="FONT-WEIGHT: normal; FONT-SIZE: 14px; FONT-STYLE: normal; BACKGROUND-COLOR: #f5f8f0">> </FONT>Is there any option in MVAPICH to select the IB port which should be used ?</FONT> <BR><BR>> _______________________________________________<BR>> mvapich-discuss mailing list<BR>> mvapich-discuss@cse.ohio-state.edu<BR>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss