From koop at cse.ohio-state.edu Fri May 2 16:30:15 2008 From: koop at cse.ohio-state.edu (Matthew Koop) Date: Fri May 2 16:30:26 2008 Subject: [mvapich-discuss] race in mvapich-0.9.9 cm_create_rc_qp() with viadev.connections==NULL In-Reply-To: Message-ID: John, We've now fixed this issue in the MVAPICH SVN (trunk and 1.0 branches). It will be in the upcoming MVAPICH minor release as well. For 0.9.9 your suggested fix should work there as well. Thanks again for pointing this issue out. Matt On Wed, 23 Apr 2008, Matthew Koop wrote: > John, > > Thanks for reporting this problem and looking into a possible solution. > This does appear to be a race condition in the initialization of > viadev.connections. We'll add this as a bug report and fix this in the > very near future. > > Thanks again, > > Matt > > On Tue, 22 Apr 2008, John Hawkes wrote: > > > I've encountered a race condition in mvapich-0.9.9 (also exists in > > mvapich-1.0) in cm_create_rc_qp() (mpid/ch_gen2/cm.c). On occasion, > > under conditions of dozens of threads starting up, cm_create_rc_qp() > > encounters viadev.connections==NULL. > > > > I believe the problem stems from the ordering of initialization. The > > main viainit.c calls: > > if (MPICM_Connect_UD(viadev.ud_qpn_table, viadev.lid_table)) { > > error_abort_all(GEN_EXIT_ERR, "MPICM_Connect_UD"); > > } > > and soon thereafter it initializes viadev.connections. Meanwhile, > > MPICM_Connect_UD() has done a pthread_create() of cm_completion_handler > > (). That concurrently executing thread handles incoming messages, one > > of which may get to cm_accept(), which then calls cm_create_rc_qp(), > > which may dereference viadev.connections before the main thread has > > initialized it. > > > > I seem to be able to avoid this race condition by moving the call to > > MPICM_Connect_UD() to follow the initialization of viadev.connections. > > Does this fix create other problems that my current testing has not yet > > encountered? > > > > John Hawkes > > jhawkes@PenguinComputing.com > > > > _______________________________________________ > > mvapich-discuss mailing list > > mvapich-discuss@cse.ohio-state.edu > > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > > > > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From sridharj at cse.ohio-state.edu Fri May 2 16:36:13 2008 From: sridharj at cse.ohio-state.edu (Jaidev Sridhar) Date: Fri May 2 16:36:25 2008 Subject: [mvapich-discuss] Re: rsh problem (fwd) In-Reply-To: References: Message-ID: <20080502203613.GA29743@mu.cse.ohio-state.edu> Hi Tai, | Hi, | I have a problem when running mvapich-0.9.9, I built mvapich using the TCP interface. When compiling using mpif90 I have no problem but when I try to run it with 4 processors I encounter an error message. | er@AI03170801 <145> $MPIRUN -np 4 imslmpi | Trying krb4 rsh... | krb_sendauth failed: You have no tickets cached | trying normal rsh (/usr/bin/rsh) | localhost.localdomain: Connection refused This looks like a problem with rsh setup. Can you check if password-less rsh works? If it doesn't, you might want to try the steps mentioned at http://www.beowulf.org/archive/2005-July/013409.html -Jaidev | p0_24371: p4_error: Child process exited while making connection to remote proc | ess on AI03170801: 0 | I'm running on redhat5 enterprise using pgi compiler. How do I resolve this issue? | | Thank you | Tai From keshetti85-student at yahoo.co.in Mon May 5 07:01:07 2008 From: keshetti85-student at yahoo.co.in (Keshetti Mahesh) Date: Mon May 5 08:17:45 2008 Subject: [mvapich-discuss] Using RDMA CM with MVAPICH2 Message-ID: <829ded920805050401v16961f10y4cae507d58c74cfe@mail.gmail.com> Hi all, I want to use the RDMA CM option of MVAPICH2. The procedure described in the user guide is not much informative. Can anyone here give me the detailed procedure for using the RDMA CM option. Also, I'll be glad if some one can give me a document describing how it works in detail. Actually I have some doubts like, how the IP addresses (???) are resolved into IB addresses and what happens in the case of nodes two HCAs (or 1 HCA with two ports) ? In the MVAPICH2 user guide it is mentioned that "RDMA CM device needs to be setup, configured with an IP address and connected to the network". Is this same as configuring IPoIB device ? -Mahesh From keshetti85-student at yahoo.co.in Mon May 5 08:24:07 2008 From: keshetti85-student at yahoo.co.in (Keshetti Mahesh) Date: Mon May 5 08:24:19 2008 Subject: [mvapich-discuss] Using RDMA CM with MVAPICH2 Message-ID: <829ded920805050524p545cd1fcie583c732c8dd92ca@mail.gmail.com> Hi all, I want to use the RDMA CM option of MVAPICH2. The procedure described in the user guide is not much informative. Can anyone here give me the detailed procedure for using the RDMA CM option. Also, I'll be glad if some one can give me a document describing how it works in detail. Actually I have some doubts like, how the IP addresses (???) are resolved into IB addresses and what happens in the case of nodes two HCAs (or 1 HCA with two ports) ? In the MVAPICH2 user guide it is mentioned that "RDMA CM device needs to be setup, configured with an IP address and connected to the network". Is this same as configuring IPoIB device ? -Mahesh From panda at cse.ohio-state.edu Mon May 5 08:24:20 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Mon May 5 08:24:33 2008 Subject: [mvapich-discuss] Using RDMA CM with MVAPICH2 In-Reply-To: <829ded920805050401v16961f10y4cae507d58c74cfe@mail.gmail.com> Message-ID: RDMA CM is designed by OpenFabrics. MVAPICH2 uses this. If you post your questions to OpenFabrics General mailing list, the designers of RDMA CM component will be able to provide detailed answers regarding how it works. DK On Mon, 5 May 2008, Keshetti Mahesh wrote: > Hi all, > > I want to use the RDMA CM option of MVAPICH2. The procedure described > in the user guide is not much informative. Can anyone here give me the detailed > procedure for using the RDMA CM option. Also, I'll be glad if some one can give > me a document describing how it works in detail. > Actually I have some doubts like, how the IP addresses (???) are > resolved into IB > addresses and what happens in the case of nodes two HCAs (or 1 HCA > with two ports) ? > > In the MVAPICH2 user guide it is mentioned that "RDMA CM device needs > to be setup, > configured with an IP address and connected to the network". Is this > same as configuring > IPoIB device ? > > -Mahesh > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From keshetti85-student at yahoo.co.in Tue May 6 05:21:08 2008 From: keshetti85-student at yahoo.co.in (Keshetti Mahesh) Date: Tue May 6 05:21:25 2008 Subject: [mvapich-discuss] Using RDMA CM with MVAPICH2 In-Reply-To: <829ded920805050529k19dee08o3ad19ce9d07bd2ca@mail.gmail.com> References: <829ded920805050401v16961f10y4cae507d58c74cfe@mail.gmail.com> <829ded920805050529k19dee08o3ad19ce9d07bd2ca@mail.gmail.com> Message-ID: <829ded920805060221lc3b5f77safacf8a8ad299b33@mail.gmail.com> I'vecouple of more questions to ask you. Below are the steps mentioned in MVAPICH user guide for running MPI application with RDMA CM support. *? Setup the RDMA CM device: RDMA CM device needs to be setup, configured with an IP address and connected to the network.* I have two machines (n0 and n1)connected with one ethernet interface and 2 IB interfaces in each. And /etc/hosts on both machines is like below. 192.168.3.1 n0 192.168.3.2 n1 172.131.15.1 n0_ib0 172.131.15.2 n0_ib1 172.131.15.3 n1_ib0 172.131.15.4 n1_ib0 Now If I want to run an MPI job on both of the nodes what should I mention in the *'hostfile'* given to MPI ("n0, n1" or "n0_ib0, n1_ib0 ... " ) ? *? Setup the Local Address File: Create the file (/etc/mv2.conf) with the local IP address to be used by RDMA CM. $ echo 10.1.1.1 >> /etc/mv2.conf* Why is this file (/etc/mv2.conf) required ? Is it required to be present on all nodes? -Mahesh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080506/d7e8f6bd/attachment.html From forum.san at gmail.com Tue May 6 00:27:49 2008 From: forum.san at gmail.com (Sangamesh B) Date: Tue May 6 08:05:45 2008 Subject: [mvapich-discuss] caused collective abort of all ranks + signal 9 Message-ID: Hi all, *I got some problem, can someone help me on this issue.* *The scenario is : We have a Rocks(4.2) cluster with 12 nodes. We installed Infiniband cards newly in 5 nodes(Masternode doesn't have IB card). Installation of OFED is successful and IP got assigned.* *I installed Mvapich2 in that and created password free environment from computenode-0-8 to 12(the nodes which have IB card). So far everything is fine, And the MPD is booting up also. * *I've compiled a sample MPI program and tried to execute it and I got the following kind of results:* Scenario 1: Using root to execute Hellow.o (compiled with mvapich2-mpicc) [root@compute-0-8 test]# /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o Hello world from process 0 of 2 Hello world from process 1 of 2 rank 1 in job 8 compute-0-8.local_34399 caused collective abort of all ranks exit status of rank 1: killed by signal 9 rank 0 in job 8 compute-0-8.local_34399 caused collective abort of all ranks exit status of rank 0: killed by signal 9 Scenario 2: Using user id (srinu) to execute the same file. [srinu@compute-0-8 test]$ /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This will severely limit memory registrations. Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(259)....: Initialization failed MPID_Init(102)...........: channel initialization failed MPIDI_CH3_Init(178)......: MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type rdma_iba_hca_init(645)...: cannot create cq Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(259)....: Initialization failed MPID_Init(102)...........: channel initialization failed MPIDI_CH3_Init(178)......: MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type rdma_iba_hca_init(645)...: cannot create cq rank 1 in job 9 compute-0-8.local_34399 caused collective abort of all ranks exit status of rank 1: return code 1 For 2nd scenario, I found solution from net such as ulimit ?l unlimited. But further, this also produced same error as of 1st scenario. Can someone solve this error? Thanks in advance, Sangamesh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080506/a68ccbda/attachment.html From narravul at cse.ohio-state.edu Tue May 6 09:50:48 2008 From: narravul at cse.ohio-state.edu (Sundeep Narravula) Date: Tue May 6 09:51:00 2008 Subject: [mvapich-discuss] Using RDMA CM with MVAPICH2 In-Reply-To: <829ded920805060221lc3b5f77safacf8a8ad299b33@mail.gmail.com> Message-ID: Hi Mahesh, Thanks for trying out our RDMA CM support. My answers are inline. > I'vecouple of more questions to ask you. > > Below are the steps mentioned in MVAPICH user guide for > running MPI application with RDMA CM support. > > *• Setup the RDMA CM device: RDMA CM device needs to be > setup, configured with an IP address and connected to the network.* > > I have two machines (n0 and n1)connected with one ethernet interface > and 2 IB interfaces in each. And /etc/hosts on both machines is like > below. > > 192.168.3.1 n0 > 192.168.3.2 n1 > 172.131.15.1 n0_ib0 > 172.131.15.2 n0_ib1 > 172.131.15.3 n1_ib0 > 172.131.15.4 n1_ib0 > > Now If I want to run an MPI job on both of the nodes what should I > mention in the *'hostfile'* given to MPI ("n0, n1" or "n0_ib0, n1_ib0 ... " > ) ? You can use any one of these pairs in your hostfile. i.e. using n0 and n1 should work fine. > *• Setup the Local Address File: Create the file (/etc/mv2.conf) with the > local IP address to be used by RDMA CM. > $ echo 10.1.1.1 >> /etc/mv2.conf* > > Why is this file (/etc/mv2.conf) required ? Is it required to be present on > all nodes? The local rdma-cm device that the mpi library needs to use is specified in the /etc/mv2.conf file. The file needs to be on all the machines. --Sundeep. > > -Mahesh > From koop at cse.ohio-state.edu Tue May 6 11:45:11 2008 From: koop at cse.ohio-state.edu (Matthew Koop) Date: Tue May 6 11:45:23 2008 Subject: [mvapich-discuss] caused collective abort of all ranks + signal 9 In-Reply-To: Message-ID: Sangamesh, Can you run any of the included benchmarks with the OFED package? Try running the ibv_rc_pingpoing test between nodes in your system to first make sure the fabric is healthy. Also, can you give us some additional information on your setup? What type of cards are these? Also, how did you set the 'ulimit -l unlimited'. We suggest placing it in /etc/init.d/sshd on all nodes and restarting sshd (and mpd). This will ensure that the processes started will inherit the modified ulimit settings. Thanks, Matt On Tue, 6 May 2008, Sangamesh B wrote: > Hi all, > > > *I got some problem, can someone help me on this issue.* > > *The scenario is : We have a Rocks(4.2) cluster with 12 nodes. We installed > Infiniband cards newly in 5 nodes(Masternode doesn't have IB card). > Installation of OFED is successful and IP got assigned.* > > *I installed Mvapich2 in that and created password free environment from > computenode-0-8 to 12(the nodes which have IB card). So far everything is > fine, And the MPD is booting up also. * > > *I've compiled a sample MPI program and tried to execute it and I got the > following kind of results:* > > Scenario 1: Using root to execute Hellow.o (compiled with mvapich2-mpicc) > > [root@compute-0-8 test]# /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o > Hello world from process 0 of 2 > Hello world from process 1 of 2 > rank 1 in job 8 compute-0-8.local_34399 caused collective abort of all > ranks > exit status of rank 1: killed by signal 9 > rank 0 in job 8 compute-0-8.local_34399 caused collective abort of all > ranks > exit status of rank 0: killed by signal 9 > > Scenario 2: Using user id (srinu) to execute the same file. > > [srinu@compute-0-8 test]$ /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. > This will severely limit memory registrations. > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. > This will severely limit memory registrations. > Fatal error in MPI_Init: > Other MPI error, error stack: > MPIR_Init_thread(259)....: Initialization failed > MPID_Init(102)...........: channel initialization failed > MPIDI_CH3_Init(178)......: > MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type > rdma_iba_hca_init(645)...: cannot create cq > Fatal error in MPI_Init: > Other MPI error, error stack: > MPIR_Init_thread(259)....: Initialization failed > MPID_Init(102)...........: channel initialization failed > MPIDI_CH3_Init(178)......: > MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type > rdma_iba_hca_init(645)...: cannot create cq > rank 1 in job 9 compute-0-8.local_34399 caused collective abort of all > ranks > exit status of rank 1: return code 1 > > For 2nd scenario, I found solution from net such as ulimit –l unlimited. > But further, this also produced same error as of 1st scenario. > Can someone solve this error? > > Thanks in advance, > > Sangamesh > From srikanth.gumma at Locuz.com Tue May 6 22:42:50 2008 From: srikanth.gumma at Locuz.com (Srikanth Gumma) Date: Tue May 6 22:43:32 2008 Subject: [mvapich-discuss] Execution problem Message-ID: <004a01c8afec$0bd63730$2382a590$@gumma@locuz.com> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/png Size: 47069 bytes Desc: not available Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080507/f9eac213/attachment-0001.png -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 3194 bytes Desc: not available Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080507/f9eac213/attachment-0001.jpe From srikanth.gumma at Locuz.com Tue May 6 22:49:25 2008 From: srikanth.gumma at Locuz.com (Srikanth Gumma) Date: Tue May 6 22:49:56 2008 Subject: [mvapich-discuss] Execution problem Message-ID: <005101c8afec$f769d530$e63d7f90$@gumma@locuz.com> Hello, I got some problem, can somebody help on this issue. The scenario is : We have a Rocks(4.2) cluster with 12 nodes. We installed PCI-Ex Infiniband cards newly in 5 nodes(Masternode doesn?t have IB card). Installation of OFED is successful and IP got assigned. I installed Mvapich2 in that and created password free environment from computenode-0-8 to 12(the nodes which have IB card).? So far everything is fine, And the MPD is booting up also. I?ve compiled a sample MPI program and tried to execute it and I got the following kind of results: Scenario 1: Using root to execute Hellow.o (compiled with mvapich2-mpicc) [root@compute-0-8 test]# /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o Hello world from process 0 of 2 Hello world from process 1 of 2 rank 1 in job 8? compute-0-8.local_34399?? caused collective abort of all ranks ? exit status of rank 1: killed by signal 9 rank 0 in job 8? compute-0-8.local_34399?? caused collective abort of all ranks ? exit status of rank 0: killed by signal 9 Scenario 2: Using user id (srinu) to execute the same file. [srinu@compute-0-8 test]$ /opt/mvapich2_ps/bin/mpiexec -np 2 /test/Hellow.o libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. ??? This will severely limit memory registrations. libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. ??? This will severely limit memory registrations. Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(259)....: Initialization failed MPID_Init(102)...........: channel initialization failed MPIDI_CH3_Init(178)......: MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type rdma_iba_hca_init(645)...: cannot create cq Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(259)....: Initialization failed MPID_Init(102)...........: channel initialization failed MPIDI_CH3_Init(178)......: MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type rdma_iba_hca_init(645)...: cannot create cq rank 1 in job 9? compute-0-8.local_34399?? caused collective abort of all ranks ? exit status of rank 1: return code 1 I believe there should be some kind of permissions or configuration to be changed such that the problem will get resolved. So far I?m unable to resolve the issue If some of you knows and fix/workaround for the above situation please suggest me the same. Any Help is appreciated. Srikanth Gumma Locuz Enterprise Solutions Pvt Ltd #20, Alfa Centere, VS Layout Intermediate Ring Road Bangalore 560047 Ph: +91-80-41314747 From jbernstein at penguincomputing.com Tue May 6 23:11:04 2008 From: jbernstein at penguincomputing.com (Joshua Bernstein) Date: Tue May 6 23:12:18 2008 Subject: [mvapich-discuss] Execution problem In-Reply-To: <005101c8afec$f769d530$e63d7f90$@gumma@locuz.com> References: <005101c8afec$f769d530$e63d7f90$@gumma@locuz.com> Message-ID: Hi Srikanth, On May 6, 2008, at 7:49 PM, Srikanth Gumma wrote: > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. > This will severely limit memory registrations. > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This tells me that perhaps you, as a user aren't able to lock as much memory as required to startup the job. Can you give it a shot as root? In BASH you, as root, can adjust your ulimit to unlimited to see if the problem goes away: # ulimit -l unlimited If this works, then you need to adjust /etc/security/limits.conf and relogin to enable the change. -Joshua Bernstein Software Engineer Penguin Computing From Bill.Riner at comcast.net Tue May 6 20:15:12 2008 From: Bill.Riner at comcast.net (Bill Riner) Date: Tue May 6 23:20:18 2008 Subject: [mvapich-discuss] IBV_EVENT_QP_LAST_WQE_REACHED error Message-ID: <20080507001459.A899B2A0065@barracuda2.cse.ohio-state.edu> We upgraded from IBGold to OFED 1.2.5.5 and we also get the "Got FATAL event IBV_EVENT_QP_LAST_WQE_REACHED" error that's discussed in this thread: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2007-October/001139 .html [0:compute-0-23.local] Abort: [0] Got FATAL event IBV_EVENT_QP_LAST_WQE_REACHED, code=16 at line 2555 in file viacheck.c The particular code worked fine before the upgrade. The error doesn't occur when using just one processor. MPICH works fine on any number of processors. The OSU and Intel MPI benchmarks work fine. Other MPI codes work fine also. We're using MVAPICH 0.9.9 that came with OFED. Has this been resolved? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080506/6bda7a16/attachment.html From keshetti85-student at yahoo.co.in Wed May 7 00:30:18 2008 From: keshetti85-student at yahoo.co.in (Keshetti Mahesh) Date: Wed May 7 00:30:31 2008 Subject: [mvapich-discuss] Using RDMA CM with MVAPICH2 In-Reply-To: References: <829ded920805060221lc3b5f77safacf8a8ad299b33@mail.gmail.com> Message-ID: <829ded920805062130s89f7e1eyaee3b4f6a5ab55ac@mail.gmail.com> > Hi Mahesh, > > Thanks for trying out our RDMA CM support. My answers are inline. > Thanks Sundeep for replying. I've successfully tested RDMA CM support of MVAPICH2 and it worked fine. As far as I know, not many people are using the RDMA CM support of MVAPICH2. So I want to ask you whether using RDMA CM support have any impact on the performance of the MPI application AND How stable is the RDMA CM support code of MVAPICH2 ? -Mahesh From srikanth.gumma at Locuz.com Wed May 7 01:33:55 2008 From: srikanth.gumma at Locuz.com (Srikanth Gumma) Date: Wed May 7 01:34:23 2008 Subject: [mvapich-discuss] Execution problem In-Reply-To: References: <005101c8afec$f769d530$e63d7f90$@gumma@locuz.com> Message-ID: <005601c8b003$f25a8c30$d70fa490$@gumma@locuz.com> Hi Joshua, I tried with ulimit -l unlimited, But it didn't work Please let me know any other options Regards Srikanth -----Original Message----- From: Joshua Bernstein [mailto:jbernstein@penguincomputing.com] Sent: Wednesday, May 07, 2008 8:41 AM To: Srikanth Gumma Cc: mvapich-discuss@cse.ohio-state.edu Subject: Re: [mvapich-discuss] Execution problem Hi Srikanth, On May 6, 2008, at 7:49 PM, Srikanth Gumma wrote: > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. > This will severely limit memory registrations. > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. This tells me that perhaps you, as a user aren't able to lock as much memory as required to startup the job. Can you give it a shot as root? In BASH you, as root, can adjust your ulimit to unlimited to see if the problem goes away: # ulimit -l unlimited If this works, then you need to adjust /etc/security/limits.conf and relogin to enable the change. -Joshua Bernstein Software Engineer Penguin Computing From Craig.Tierney at noaa.gov Wed May 7 11:09:04 2008 From: Craig.Tierney at noaa.gov (Craig Tierney) Date: Wed May 7 11:09:18 2008 Subject: [mvapich-discuss] Execution problem In-Reply-To: <005601c8b003$f25a8c30$d70fa490$%gumma@locuz.com> References: <005101c8afec$f769d530$e63d7f90$%gumma@locuz.com> <005601c8b003$f25a8c30$d70fa490$%gumma@locuz.com> Message-ID: <4821C610.3080503@noaa.gov> Srikanth Gumma wrote: > Hi Joshua, > > I tried with ulimit -l unlimited, But it didn't work > > Please let me know any other options > Where did you put the ulimit command? If you put it right before the mpirun command, that won't help. It needs to be in your environment on the remote nodes. You should put it in your .cshrc/.bashrc. Are you using a batch system (like SGE)? Craig > Regards > Srikanth > > -----Original Message----- > From: Joshua Bernstein [mailto:jbernstein@penguincomputing.com] > Sent: Wednesday, May 07, 2008 8:41 AM > To: Srikanth Gumma > Cc: mvapich-discuss@cse.ohio-state.edu > Subject: Re: [mvapich-discuss] Execution problem > > Hi Srikanth, > > On May 6, 2008, at 7:49 PM, Srikanth Gumma wrote: > >> libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. >> This will severely limit memory registrations. >> libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. > > This tells me that perhaps you, as a user aren't able to lock as much > memory as required to startup the job. Can you give it a shot as root? > > In BASH you, as root, can adjust your ulimit to unlimited to see if > the problem goes away: > > # ulimit -l unlimited > > If this works, then you need to adjust /etc/security/limits.conf and > relogin to enable the change. > > -Joshua Bernstein > Software Engineer > Penguin Computing > > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > -- Craig Tierney (craig.tierney@noaa.gov) From koop at cse.ohio-state.edu Wed May 7 11:48:31 2008 From: koop at cse.ohio-state.edu (Matthew Koop) Date: Wed May 7 11:48:42 2008 Subject: [mvapich-discuss] IBV_EVENT_QP_LAST_WQE_REACHED error In-Reply-To: <20080507001459.A899B2A0065@barracuda2.cse.ohio-state.edu> Message-ID: Bill, The error described in the link you gave was the result of some 'fork's or system calls in the code (that did a fork). Does your code contain any system calls or forks? If so, please look at section 7.1.2 of the user guide: https://mvapich.cse.ohio-state.edu/support/mvapich_user_guide.html Also, you may want to consider upgrading to OFED 1.3 and MVAPICH 1.0, which has additional scalability and stability enhancements. Let us know how things go. Thanks, Matt On Tue, 6 May 2008, Bill Riner wrote: > We upgraded from IBGold to OFED 1.2.5.5 and we also get the "Got FATAL event > IBV_EVENT_QP_LAST_WQE_REACHED" error that's discussed in this thread: > > > > http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2007-October/001139 > .html > > > > [0:compute-0-23.local] Abort: [0] Got FATAL event > IBV_EVENT_QP_LAST_WQE_REACHED, code=16 > > at line 2555 in file viacheck.c > > > > The particular code worked fine before the upgrade. The error doesn't occur > when using just one processor. MPICH works fine on any number of > processors. The OSU and Intel MPI benchmarks work fine. Other MPI codes > work fine also. We're using MVAPICH 0.9.9 that came with OFED. > > > > Has this been resolved? > > > > > > From rcbord at wm.edu Wed May 7 12:47:51 2008 From: rcbord at wm.edu (rcbord@wm.edu) Date: Wed May 7 13:33:40 2008 Subject: [mvapich-discuss] MPI_Cart_Create Message-ID: Hi, I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with ofed 1.2 for infinaband. We have be able to use it for 6 months without any problem. I tried to port a users fortran code that runs under solaris-sparc without any problems. The code is fairly vanilla fotran-90 with mpi I know because I wrote it. When I tried to port it to the Linux Cluster it hangs a mpisendrecv function. I included a simple test code that shows that the new communicator (comm2d) returned by the MPI_Cart_create function has multiple values. Same code run on the sun sparc cluster returns a single value. I am guessing it could be how I compiled MVAPICH, but have tried a few more flags without any success. I tried this with MVAPICH2 also and got the same error, but I compiled it in a similar fashion. Could it be a PGI problem? Has anyone else seen this? Thanks for any help! Program mpicart ! With input files ! Conversion of efit2d.f90 to 3d JPB 11-2007 Implicit none include "mpif.h" Interface Subroutine read_model(Pdim0,Pdim1) Integer, Intent(out) ::Pdim0,Pdim1 end Subroutine read_model end Interface Double Precision :: t1,t2 ! ! define MPI variables ! Integer :: Pid, N_proc, ierr Integer :: comm2d = 0 Integer :: status Integer :: period(2),Pdim_size(2),coords(2) Integer :: nbrleft,nbrright,nbrtop,nbrbottom Integer :: Pdim0, Pdim1 Integer :: i, t, numt call MPI_INIT(ierr) call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) if (Pid .EQ. 0) then call read_model(Pdim0,Pdim1) end if ! ! Broad cast input parameters values to all processors ! call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & & MPI_COMM_WORLD,ierr) call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & & MPI_COMM_WORLD,ierr) call MPI_BARRIER(MPI_COMM_WORLD,ierr) Pdim_size(1) = Pdim0 Pdim_size(2) = Pdim1 period(1) = 0 period(2) = 0 call MPI_Dims_create(N_proc,2,Pdim_size,ierr) call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) do i = 0,N_proc-1 if (Pid == i) then write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d end if end do call MPI_BARRIER(MPI_COMM_WORLD,ierr) call MPI_Comm_free(comm2d,ierr) call MPI_FINALIZE(ierr) end program mpicart ###################### OUTPUT ######################### Model parameters for mpicart test Processor Topology is ( 3 by 4 ) pid 0 mpi_comm_2d 140 pid 11 mpi_comm_2d 138 pid 2 mpi_comm_2d 140 pid 4 mpi_comm_2d 140 pid 6 mpi_comm_2d 140 pid 8 mpi_comm_2d 140 pid 3 mpi_comm_2d 138 pid 10 mpi_comm_2d 140 pid 1 mpi_comm_2d 138 pid 5 mpi_comm_2d 138 pid 7 mpi_comm_2d 138 pid 9 mpi_comm_2d 138 Chris Bording Application Analyst High Performance Computing Group Information Technology The College of William and Mary (757)-221-3488 rcbord@wm.edu From Frank.Leers at Sun.COM Wed May 7 13:34:56 2008 From: Frank.Leers at Sun.COM (Frank Leers) Date: Wed May 7 13:35:40 2008 Subject: [mvapich-discuss] Sun Studio build mvapich1 Message-ID: Hi, I am attempting to build mvapich 1 with Studio12 and am not having much success. Could someone have a look and point out the err of my ways? A couple of things jump out at me - checking for cc... no checking whether the Fortran 77 compiler (/opt/apps/sun/sunstudio12/ bin/f77 -L/opt/ofed/lib64 ) works... no OFED 1.2.5-5 was built with gnu, is that a problem? I've modified the make_mvapich.gen2 as follows: 18,25c18,24 < STUDIOHOME=/opt/apps/sun/sunstudio12 < IBHOME=${IBHOME:-/opt/ofed} < IBHOME_LIB=${IBHOME_LIB:-/opt/ofed/lib64} < PREFIX=${PREFIX:-/tmp/mvapich-1.0-2008-02-28} < export CC=${CC:-$STUDIOHOME/bin/cc} < export CXX=${CXX:-$STUDIOHOME/bin/CC} < export F77=${F77:-$STUDIOHOME/bin/f77} < export F90=${F90:-$STUDIOHOME/bin/f90} --- > IBHOME=${IBHOME:-/usr/local/ofed} > IBHOME_LIB=${IBHOME_LIB:-/usr/local/ofed/lib64} > PREFIX=${PREFIX:-/usr/local/mvapich} > export CC=${CC:-gcc} > export CXX=${CXX:-g++} > export F77=${F77:-g77} > export F90=${F90:-} thanks, -frank [mvapich-1.0-2008-02-28]# ./make.mvapich.gen2.studio12 Configuring MVAPICH... Configuring with args --with-device=ch_gen2 --with-arch=LINUX -prefix=/ tmp/mvapich-1.0-2008-02-28 --with-romio --without-mpe -lib=-L/opt/ofed/ lib64 -Wl,-rpath=/opt/ofed/lib64 -libverbs -libumad -lpthread Configuring MPICH Version 1.2.7 (release) of : 2005/06/22 16:33:49 Make will not build MPE routines checking whether filesystem respects case in file names... yes checking for current directory name... /tmp/mvapich1/ mvapich-1.0-2008-02-28 checking for install checking for ranlib checking for gnumake... yes using --no-print-directory checking whether make supports include... yes checking for OSF V3 make... no checking for virtual path format... VPATH checking whether /opt/apps/sun/sunstudio12/bin/CC returns correct error code... yes checking whether selected C++ compiler can compile iostream.h... yes Include C++ bindings for MPI from http://www.osl.iu.edu/research/mpi2c++ Send bug reports about the C++ to mpi2cpp-devel@osl.iu.edu checking for cc... no checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts ANSI prototypes... yes checking for /opt/apps/sun/sunstudio12/bin/f77... found /opt/apps/sun/ sunstudio12/bin/f77 (1) checking for ar... found /usr/bin/ar (ar) checking for whether compiler works... yes checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts ## for concatenation in cpp... yes checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts const modifier... yes checking for extension on executables... none checking how to run the C preprocessor checking for AIX checking for minix/config.h checking for volatile... yes Running device-specific setup program setup_ch_gen2 starts... -D_X86_64_ -D_AMD_QUAD_CORE_ - DEARLY_SEND_COMPLETION -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ - D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/opt/ofed/include -O3 About to do ch_gen2 device Configuring ch_gen2 subsystem... No addition setup since mpd is not enabled Use ${exec_prefix}/bin to set up variables MPIBOOT and MPIUNBOOT checking whether cross-compiling... no checking for gethrtime... no checking for clock_gettime... no checking for clock_getres... no checking for how many arguments gettimeofday takes... two - whew checking for signal.h... yes checking for sigaction... yes checking for for struct sigaction... yes checking for weak symbol support... pragma weak checking whether weak symbols are visible to other files... yes checking for remsh... no checking for rsh... found /usr/kerberos/bin/rsh (rsh) checking for perl5... no checking for perl... found /usr/bin/perl (perl) checking for byte ordering... little endian checking for uname... yes checking for netdb.h... yes checking for gethostbyname... yes checking whether C++ compiler also has gethostbyname... yes checking whether Fortran programs can link with needed C functions... yes checking whether C++ programs can link with needed C functions... yes checking for catopen... yes checking for catclose... yes checking for catgets... yes checking for gencat... found /usr/bin/gencat (1) checking for nl_types.h... yes checking for ANSI C header files checking for stdlib.h... yes checking for string.h... yes checking for unistd.h... yes checking for stdarg.h... yes checking whether stdarg is oldstyle... hmm, warnings from compiler. Trying newstyle checking whether stdarg is correct... yes checking whether programs using stdarg link... yes checking for malloc return type... void checking for system... yes checking for nice... yes checking for strdup... yes checking for memory.h... yes checking for sys/ioctl.h... yes checking for sys/sockio.h... no checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc runs... yes checking whether the compiler /opt/apps/sun/sunstudio12/bin/f77 runs... yes checking for correct handling of conditionals...... yes checking for correct handling of conditionals part 2 ...... yes checking for correct handling of #elif... yes checking that constant folding in arrays works... yes checking that structs are handled properly... yes checking that compiler truncates unsigned char correctly ...... yes checking that compiler/hardware has commutative arithmetic ...... yes checking that compiler assigns to char correctly ...... yes checking that compiler does not call unknown routines for long doubles... yes checking that compiler accepts args correctly ...... yes checking whether ranlib works... yes checking for broken handling of common symbols... no checking for ar... found /usr/bin/ar (1) checking for size of void *... 8 checking for pointers greater than 32 bits... yes checking for size of int... 4 checking for int large enough for pointers... no checking for C integer type for MPI_Aint... checking for size of long... 8 C integer type for MPI_Aint is... long checking for long long... yes checking whether compiler can handle loops with long long... yes checking for long double... yes checking for size of double... 8 checking for size of long double... 16 checking for structure alignment... largest checking for sync... found /bin/sync (1) checking whether mpich can be used as the library name... yes Configuring Fortran subsystem Configure in /tmp/mvapich1/mvapich-1.0-2008-02-28/src/fortran creating cache ./config.cache checking for cc... /opt/apps/sun/sunstudio12/bin/cc checking whether the C compiler (/opt/apps/sun/sunstudio12/bin/cc - D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H ) works... yes checking whether the C compiler (/opt/apps/sun/sunstudio12/bin/cc - D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H ) is a cross-compiler... no checking whether the C compiler sets its return status correctly... yes checking whether we are using GNU C... no checking for working const... yes checking whether the Fortran 77 compiler (/opt/apps/sun/sunstudio12/ bin/f77 -L/opt/ofed/lib64 ) works... no configure: error: installation or configuration problem: Fortran 77 compiler cannot create executables. Error configuring the Fortran subsystem! Turning off Fortran support Configuring ROMIO, for support of MPI-IO checking whether compiler allows define a(b) a(b,__LINE__)... yes Configuring with args --with-mpi=mpich --cache-file=/dev/null checking for architecture... LINUX ROMIO home directory is /tmp/mvapich1/mvapich-1.0-2008-02-28/romio checking for ranlib... ranlib checking for gcc... /opt/apps/sun/sunstudio12/bin/cc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... no checking whether /opt/apps/sun/sunstudio12/bin/cc accepts -g... yes checking for /opt/apps/sun/sunstudio12/bin/cc option to accept ANSI C... none needed checking whether we are using the GNU Fortran 77 compiler... no checking whether /opt/apps/sun/sunstudio12/bin/f77 accepts -g... yes checking how to run the C preprocessor... /opt/apps/sun/sunstudio12/ bin/cc -E checking for memalign... yes checking Fortran external names Fortran externals have a trailing underscore and are lowercase checking for inline... inline checking how to run the C preprocessor... /opt/apps/sun/sunstudio12/ bin/cc -E checking for egrep... grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for unistd.h... (cached) yes checking fcntl.h usability... yes checking fcntl.h presence... yes checking for fcntl.h... yes checking malloc.h usability... yes checking malloc.h presence... yes checking for malloc.h... yes checking stddef.h usability... yes checking stddef.h presence... yes checking for stddef.h... yes checking for int... yes checking size of int... 4 checking for void*... yes checking size of void*... 8 checking for int large enough for pointers... no checking for long long... yes checking size of long long... 8 checking for extension for Fortran 90 programs... f90 checking for Fortran 90 KIND parameter for 8-byte integers... 8 checking for weak symbol support... pragma weak checking that weak symbols are visible to other files... yes checking whether struct flock compatible with MPI_Offset... yes checking configured file systems... testfs ufs nfs configure: WARNING: File locks may not work with NFS. See the Installation and users manual for instructions on testing and if necessary fixing this checking for library containing pthread_create... -lpthread checking for library containing aio_write... -lrt checking signal.h usability... yes checking signal.h presence... yes checking for signal.h... yes checking if aio.h exists and can be compiled... yes checking if sys/aio.h exists and can be compiled... no checking whether aio routines can be used... yes checking for obsolete two-argument aio_write... no checking for obsolete two-argument aio_suspend... no checking for aio_fildes member of aiocb structure... yes checking for aio_whence member of aiocb structure... no checking for aio_handle member of aiocb structure... no checking for aio_reqprio member of aiocb structure... yes checking for aio_sigevent member of aiocb structure... yes checking sys/vfs.h usability... yes checking sys/vfs.h presence... yes checking for sys/vfs.h... yes checking sys/param.h usability... yes checking sys/param.h presence... yes checking for sys/param.h... yes checking sys/mount.h usability... yes checking sys/mount.h presence... yes checking for sys/mount.h... yes checking sys/statvfs.h usability... yes checking sys/statvfs.h presence... yes checking for sys/statvfs.h... yes checking whether struct statfs properly defined... yes checking for f_fstypename member of statfs structure... no checking for sys/stat.h... (cached) yes checking for sys/types.h... (cached) yes checking for unistd.h... (cached) yes checking for stat... yes checking for st_fstype member of stat structure... no checking for sys/types.h... (cached) yes checking for sys/statvfs.h... (cached) yes checking for sys/vfs.h... (cached) yes checking for statvfs... yes checking for f_basetype member of statvfs structure... no checking for unistd.h... (cached) yes checking for large file defines... yes checking whether off64_t is an scalar type... yes checking for strerror... yes checking for doctext... no checking for strdup... no checking for lstat... no checking for readlink... no checking for fsync... no checking for ftruncate... no setting CC to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc setting F77 to /opt/apps/sun/sunstudio12/bin/f77 setting TEST_CC to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc setting TEST_F77 to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpif77 setting CFLAGS to -D_X86_64_ -D_AMD_QUAD_CORE_ - DEARLY_SEND_COMPLETION -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ - D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/opt/ofed/include -O3 - DHAVE_MPICHCONF_H -D_X86_64_ -D_AMD_QUAD_CORE_ - DEARLY_SEND_COMPLETION -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ - D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/opt/ofed/include -O3 - DHAVE_MPICHCONF_H -DFORTRANUNDERSCORE -D_LARGEFILE64_SOURCE - D_FILE_OFFSET_BITS=64 -DHAVE_ROMIOCONF_H setting USER_CFLAGS to setting USER_FFLAGS to configure: creating ./config.status config.status: creating Makefile config.status: creating localdefs config.status: creating mpi-io/Makefile config.status: creating mpi2-other/info/Makefile config.status: creating mpi2-other/array/Makefile config.status: creating adio/common/Makefile config.status: creating test/Makefile config.status: creating test/misc.c config.status: creating test/large_file.c config.status: creating test/runtests config.status: creating util/romioinstall config.status: creating include/mpio.h config.status: creating include/mpiof.h config.status: creating adio/ad_nfs/Makefile config.status: creating adio/ad_ufs/Makefile config.status: creating adio/ad_panfs/Makefile config.status: creating adio/ad_xfs/Makefile config.status: creating adio/ad_sfs/Makefile config.status: creating adio/ad_pfs/Makefile config.status: creating adio/ad_testfs/Makefile config.status: creating adio/ad_pvfs/Makefile config.status: creating adio/ad_pvfs2/Makefile config.status: creating adio/ad_gridftp/Makefile config.status: creating adio/ad_lustre/Makefile config.status: creating mpi-io/fortran/Makefile config.status: creating mpi2-other/info/fortran/Makefile config.status: creating mpi2-other/array/fortran/Makefile config.status: creating test/fmisc.f config.status: creating test/fcoll_test.f config.status: creating test/pfcoll_test.f config.status: creating test/fperf.f config.status: creating mpi-io/glue/mpich2/Makefile config.status: creating mpi-io/glue/mpich1/Makefile config.status: creating mpi-io/glue/default/Makefile config.status: creating adio/include/romioconf.h config.status: executing default-1 commands Finished configuring ROMIO checking whether head command is broken... no creating config.status creating Makefile creating util/Makefile creating examples/Makefile creating src/pt2pt/Makefile creating src/coll/Makefile creating src/context/Makefile creating src/dmpi/Makefile creating src/env/Makefile creating src/topol/Makefile creating src/profile/Makefile creating src/coll2/Makefile creating src/util/Makefile creating src/infoexport/Makefile creating examples/basic/Makefile creating src/misc2/Makefile creating src/external/Makefile creating mpichconf.h creating util/mpicc creating util/mpicxx creating util/mpiinstall creating util/mpirun creating util/mpichlib.conf creating util/mpiman creating util/chkserv creating util/chkmachine creating util/tstmachines creating include/mpidefs.h creating include/mpif.f90 Configuring the test suite... Configuring with args -basecc=/opt/apps/sun/sunstudio12/bin/cc -cc=/ tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc -make=make --no-print- directory -device=ch_gen2 -mpichpath=/tmp/mvapich1/ mvapich-1.0-2008-02-28/bin --enable-boot -cflags=-D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -nof77 --enable-io Don't build the tests that need Fortran checking whether cross-compiling... no checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc runs... yes checking for gnumake... no checking whether make supports include... yes checking for OSF V3 make... no checking for virtual path format... VPATH checking for stdlib.h... yes checking for unistd.h... yes checking for string.h... yes checking for stdarg is correct... yes checking for long double... yes checking for long long... yes checking whether compiler can handle loops with long long... yes checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts ANSI prototypes... yes checking for signal.h... yes checking for sigaction... yes checking for sleep... yes checking for sysconf... yes creating config.status creating Makefile creating pt2pt/Makefile creating coll/Makefile creating topol/Makefile creating context/Makefile creating env/Makefile creating profile/Makefile creating io/Mfile creating pt2pt/runtests creating coll/runtests creating topol/runtests creating context/runtests creating command/runtests creating env/runtests creating profile/runtests creating io/rtest creating io/misc.c creating io/fmisc.f creating io/fcoll_test.f creating io/pfcoll_test.f creating io/large_file.c creating io/fperf.f Configuring the performance tests ... Configure in /tmp/mvapich1/mvapich-1.0-2008-02-28/examples/perftest creating cache ./config.cache checking for a BSD compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking whether make --no-print-directory sets ${MAKE}... yes checking for working aclocal... found checking for working autoconf... found checking for working automake... found checking for working autoheader... found checking for working makeinfo... found checking whether clock skew breaks make... no checking whether make supports include... yes checking whether make allows comments in actions... yes checking for virtual path format... VPATH checking whether make sets CFLAGS... yes checking for cit... no checking for gnuplot... gnuplot checking how to run the C preprocessor... /opt/apps/sun/sunstudio12/ bin/cc -E checking whether C compiler accepts option -O4 -Ofast... no checking whether C compiler accepts option -Ofast... no checking whether C compiler accepts option -fast... yes checking whether routines compiled with -fast can be linked with ones compiled without -fast... yes checking for library containing fabs... -lm checking for ANSI C header files... yes checking for sys/time.h... yes checking for stdlib.h... yes checking for string.h... yes checking whether time.h and sys/time.h may both be included... yes checking whether struct tm is in sys/time.h or time.h... time.h checking for working const... yes checking for restrict... restrict checking for long long... yes updating cache ./config.cache creating ./config.status creating Makefile creating mpptestconf.h creating mpid/ch_gen2/Makefile creating mpid/tests/Makefile creating mpichconf.h mpichconf.h is unchanged *# ---> *# You should register your copy of MPICH with us by sending mail *# to majordomo@mcs.anl.gov containing the message *# subscribe mpi-users *# This will allow us to notify you of new releases of MPICH. *# *# You can also check the MPICH home page at *# http://www.mcs.anl.gov/mpi/mpich *# ---< Configuration completed. Building MVAPICH... cleaning directory mpi-io cleaning directory adio/common cleaning directory mpi-io/glue/mpich1 cleaning directory adio/ad_testfs cleaning directory adio/ad_ufs cleaning directory adio/ad_nfs cleaning directory mpi-io/fortran cleaning directory test rm -f .P* PI* *.o rm -f simple perf async coll_test coll_perf misc file_info excl large_array atomicity noncontig i_noncontig noncontig_coll split_coll shared_fp large_file psimple error status noncontig_coll2 fcoll_test fperf fmisc pfcoll_test cleaning src/pt2pt rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/pt2pt/*.o cleaning src/env rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/env/*.o cleaning src/dmpi rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/dmpi/*.o cleaning src/util rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/util/*.o cleaning src/context rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/context/*.o cleaning src/coll rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/coll/*.o cleaning src/topol rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/topol/*.o cleaning src/profile rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/profile/*.o cleaning src/misc2 rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/misc2/*.o cleaning src/external rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/external/*.o cleaning mpid/ch_gen2 /bin/rm -f *.o *.d *~ viainit.o viasend.o viarecv.o viacheck.o viapriv.o viaparam.o viutil.o vbuf.o cm_user.o mpid_init.o mpid_send.o mpid_recv.o cm.o mpid_hsend.o mpid_hrecv.o mpid_pack.o cmnargs.o mpid_misc.o dreg.o mpid_smpi.o collutils.o intra_rdma_barrier.o mpid_mcast.o mcst_grp.o ibmcgrp.o crc32h.o avl.o mem_hooks.o viacoalesce.o shmem_coll.o async_progress.o \ queue.o sbcnst2.o tr2.o queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h core ibmcgrp/*.o ibmcgrp/ibmcgrp \ /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/ibmcgrp make --no-print-directory -C process clean rm -f *.o mpirun_rsh mpispawn minidaemon_client pmgr_collective_client.o pmgr_collective_common.o minidaemon.o mpirun_util.o cleaning examples rm -f sendchar *.o Making clean in directory test Making clean in directory pt2pt Making clean in directory coll Making clean in directory topol Making clean in directory env Making clean in directory context Making clean in directory profile Making clean in directory io Making clean in directory perftest test -z "mpptest goptest buflimit " || rm -f mpptest goptest buflimit rm -f *.o core *.core rm -f tunepkt pktuse copytest vectest stress ctest cluster tcomm rm -f work.pc work.pcl cleaning examples/basic rm -f work.pc work.pcl rm -f *.o *~ PI* cpi systest srtest fpi cpilog cpi_autolog hello++ iotest pi3f90 upshot rdb.* startup.* core rm -f hello++.ti hello++.ii cleaning examples/test/pt2pt cleaning examples/test/coll cleaning examples/test/topol cleaning examples/test/context cleaning examples/test/env cleaning examples/test/profile cleaning examples/test Making clean in directory pt2pt Making clean in directory coll Making clean in directory topol Making clean in directory env Making clean in directory context Making clean in directory profile Making clean in directory io cleaning examples/perftest test -z "mpptest goptest buflimit " || rm -f mpptest goptest buflimit rm -f *.o core *.core rm -f tunepkt pktuse copytest vectest stress ctest cluster tcomm rm -f work.pc work.pcl cleaning src/infoexport rm -f *.o rm -f *.i rm -f *.s rm -f *.cxx.log rm -f *.cxx.errors rm -f libtvmpich.so.1.0 rm -f libtvmpich.so* rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/src/infoexport/*.o rm -f *~ *.o aditest1 aditest2 aditest3 aditest4 aditest5 aditest6 aditest7 aditest8 aditest9 aditest10 aditest11 aditest12 aditest13 timers trunc rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/lib*.a rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/shared/lib*.so* make --no-print-directory mpi-modules make --no-print-directory mpilib for file in queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h ; do \ if [ ! -s $file ] ; then \ ln -s ../util/$file; \ fi; \ done making mpir in directory mpid/ch_gen2 for file in queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h ; do \ if [ ! -s $file ] ; then \ ln -s ../util/$file; \ fi; \ done /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viainit.c "ib_init.h", line 12: warning: initializer does not fit or is out of range: -1 "ib_init.h", line 34: warning: improper pointer/integer combination: op "<" "viainit.c", line 149: warning: argument #3 is incompatible with prototype: prototype: pointer to const struct {array[16] of unsigned long __bits} : "/usr/include/sched.h", line 76 argument : pointer to unsigned long "viainit.c", line 162: warning: argument #3 is incompatible with prototype: prototype: pointer to const struct {array[16] of unsigned long __bits} : "/usr/include/sched.h", line 76 argument : pointer to unsigned long "viainit.c", line 556: warning: argument #3 is incompatible with prototype: prototype: pointer to function(pointer to void) returning pointer to void : "/usr/include/pthread.h", line 163 argument : pointer to void "viainit.c", line 792: warning: implicit function declaration: init_apm_lock "viainit.c", line 990: warning: argument mismatch "viainit.c", line 1158: warning: implicit function declaration: reload_alternate_path "viainit.c", line 1221: warning: argument #3 is incompatible with prototype: prototype: pointer to function(pointer to void) returning pointer to void : "/usr/include/pthread.h", line 163 argument : pointer to void "viainit.c", line 1345: warning: argument mismatch /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viasend.c "viasend.c", line 145: warning: implicit function declaration: viadev_post_send_async_qp "viasend.c", line 401: warning: argument mismatch "viasend.c", line 703: warning: argument mismatch /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viarecv.c "viarecv.c", line 263: warning: argument mismatch /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viacheck.c "viacheck.c", line 571: warning: argument mismatch "viacheck.c", line 1824: warning: argument mismatch "viacheck.c", line 1846: warning: argument mismatch "viacheck.c", line 1861: warning: argument mismatch "viacheck.c", line 2260: warning: implicit function declaration: perform_manual_apm "viacheck.c", line 2856: warning: implicit function declaration: reload_alternate_path /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viapriv.c "viapriv.c", line 649: warning: implicit function declaration: perform_manual_apm /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viaparam.c "/opt/ofed/include/infiniband/common.h", line 60: warning: implicit function declaration: bswap_64 "/opt/ofed/include/infiniband/umad.h", line 72: warning: zero or negative subscript /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viutil.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c vbuf.c "vbuf.c", line 125: warning: argument mismatch "vbuf.c", line 283: warning: argument mismatch "vbuf.c", line 442: warning: argument mismatch "vbuf.c", line 488: warning: argument mismatch "vbuf.c", line 491: warning: argument mismatch "vbuf.c", line 492: warning: argument mismatch /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c cm_user.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_init.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_send.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_recv.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c cm.c "ib_init.h", line 12: warning: initializer does not fit or is out of range: -1 "ib_init.h", line 34: warning: improper pointer/integer combination: op "<" "cm.c", line 176: warning: pointer to void or function used in arithmetic "cm.c", line 265: warning: implicit function declaration: reload_alternate_path "cm.c", line 677: warning: statement not reached "cm.c", line 735: warning: pointer to void or function used in arithmetic "cm.c", line 735: warning: pointer to void or function used in arithmetic "cm.c", line 744: warning: pointer to void or function used in arithmetic "cm.c", line 757: warning: statement not reached "cm.c", line 830: warning: pointer to void or function used in arithmetic "cm.c", line 830: warning: pointer to void or function used in arithmetic "cm.c", line 925: warning: pointer to void or function used in arithmetic "cm.c", line 1006: warning: pointer to void or function used in arithmetic /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_hsend.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_hrecv.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_pack.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c cmnargs.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_misc.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c dreg.c "dreg.c", line 132: warning: initializer does not fit or is out of range: -1 "dreg.c", line 780: warning: argument mismatch /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_smpi.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c collutils.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c intra_rdma_barrier.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mpid_mcast.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mcst_grp.c "mcst_grp.c", line 338: warning: empty translation unit /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c ibmcgrp.c "ibmcgrp.c", line 480: warning: empty translation unit /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c crc32h.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c avl.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c mem_hooks.c "mem_hooks.c", line 34: warning: assignment type mismatch: pointer to function(pointer to void, unsigned long) returning int "=" pointer to void /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c viacoalesce.c "viacoalesce.c", line 70: warning: argument mismatch "viacoalesce.c", line 97: warning: argument mismatch "viacoalesce.c", line 167: warning: argument mismatch "viacoalesce.c", line 231: warning: argument mismatch "viacoalesce.c", line 235: warning: argument mismatch /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c shmem_coll.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c async_progress.c "ib_init.h", line 12: warning: initializer does not fit or is out of range: -1 "ib_init.h", line 34: warning: improper pointer/integer combination: op "<" "async_progress.c", line 102: warning: function expects to return value: atomic_send_or_queue "async_progress.c", line 112: warning: implicit function declaration: perform_manual_apm "async_progress.c", line 686: warning: argument #3 is incompatible with prototype: prototype: pointer to function(pointer to void) returning pointer to void : "/usr/include/pthread.h", line 163 argument : pointer to void "async_progress.c", line 841: warning: implicit function declaration: reload_alternate_path /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c queue.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c sbcnst2.c /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/mvapich1/ mvapich-1.0-2008-02-28/include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/ include -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I/tmp/ mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE - DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 - DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ - D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE - DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/ opt/ofed/include -O3 -DHAVE_MPICHCONF_H -I/tmp/mvapich1/ mvapich-1.0-2008-02-28 -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 -I. -c tr2.c ar crl /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/libmpich.a viainit.o viasend.o viarecv.o viacheck.o viapriv.o viaparam.o viutil.o vbuf.o cm_user.o mpid_init.o mpid_send.o mpid_recv.o cm.o mpid_hsend.o mpid_hrecv.o mpid_pack.o cmnargs.o mpid_misc.o dreg.o mpid_smpi.o collutils.o intra_rdma_barrier.o mpid_mcast.o mcst_grp.o ibmcgrp.o crc32h.o avl.o mem_hooks.o viacoalesce.o shmem_coll.o async_progress.o queue.o sbcnst2.o tr2.o cd process && make --no-print-directory &&\ install -m 755 mpirun_rsh mpispawn minidaemon_client /tmp/mvapich1/ mvapich-1.0-2008-02-28/bin /opt/apps/sun/sunstudio12/bin/cc -DLD_LIBRARY_PATH_MPI=\"/tmp/ mvapich-1.0-2008-02-28/lib/shared\" -DMPI_PREFIX=\"/tmp/ mvapich-1.0-2008-02-28/\" -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/ etc/mvapich.conf\" -DVAPI -D_SMP_ -DUSE_SSH -c -o pmgr_collective_common.o pmgr_collective_common.c /opt/apps/sun/sunstudio12/bin/cc -DLD_LIBRARY_PATH_MPI=\"/tmp/ mvapich-1.0-2008-02-28/lib/shared\" -DMPI_PREFIX=\"/tmp/ mvapich-1.0-2008-02-28/\" -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/ etc/mvapich.conf\" -DVAPI -D_SMP_ -DUSE_SSH -c -o pmgr_collective_mpirun.o pmgr_collective_mpirun.c "pmgr_collective_mpirun.c", line 75: warning: pointer to void or function used in arithmetic "pmgr_collective_mpirun.c", line 97: warning: pointer to void or function used in arithmetic "pmgr_collective_mpirun.c", line 98: warning: pointer to void or function used in arithmetic "pmgr_collective_mpirun.c", line 250: warning: pointer to void or function used in arithmetic "pmgr_collective_mpirun.c", line 263: warning: pointer to void or function used in arithmetic "pmgr_collective_mpirun.c", line 269: warning: pointer to void or function used in arithmetic /opt/apps/sun/sunstudio12/bin/cc -DLD_LIBRARY_PATH_MPI=\"/tmp/ mvapich-1.0-2008-02-28/lib/shared\" -DMPI_PREFIX=\"/tmp/ mvapich-1.0-2008-02-28/\" -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/ etc/mvapich.conf\" -DVAPI -D_SMP_ -DUSE_SSH -c -o minidaemon.o minidaemon.c "minidaemon.c", line 267: warning: argument mismatch "minidaemon.c", line 300: warning: argument mismatch "minidaemon.c", line 407: warning: argument mismatch "minidaemon.c", line 430: warning: argument mismatch "minidaemon.c", line 455: warning: argument mismatch "minidaemon.c", line 542: warning: argument mismatch "minidaemon.c", line 552: warning: argument mismatch "minidaemon.c", line 560: warning: argument mismatch "minidaemon.c", line 591: warning: pointer to void or function used in arithmetic "minidaemon.c", line 679: warning: argument mismatch "minidaemon.c", line 781: warning: enum type mismatch: op "=" "minidaemon.c", line 812: warning: argument mismatch "minidaemon.c", line 842: warning: enum type mismatch: op "=" "minidaemon.c", line 854: warning: enum type mismatch: op "=" "minidaemon.c", line 1072: warning: argument mismatch "minidaemon.c", line 1114: warning: argument mismatch "minidaemon.c", line 1130: warning: argument mismatch "minidaemon.c", line 1213: warning: argument mismatch "minidaemon.c", line 1214: warning: enum type mismatch: op "=" "minidaemon.c", line 1222: warning: argument mismatch "minidaemon.c", line 1244: warning: argument mismatch "minidaemon.c", line 1247: warning: argument mismatch "minidaemon.c", line 1278: warning: argument mismatch "minidaemon.c", line 1281: warning: argument mismatch "minidaemon.c", line 1288: warning: argument mismatch "minidaemon.c", line 1309: warning: argument mismatch "minidaemon.c", line 1369: warning: argument mismatch "minidaemon.c", line 1417: warning: argument mismatch "minidaemon.c", line 1443: warning: argument mismatch "minidaemon.c", line 1492: warning: argument mismatch "minidaemon.c", line 1533: warning: argument mismatch /opt/apps/sun/sunstudio12/bin/cc -DLD_LIBRARY_PATH_MPI=\"/tmp/ mvapich-1.0-2008-02-28/lib/shared\" -DMPI_PREFIX=\"/tmp/ mvapich-1.0-2008-02-28/\" -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/ etc/mvapich.conf\" -DVAPI -D_SMP_ -DUSE_SSH -c -o mpirun_util.o mpirun_util.c "mpirun_util.c", line 25: warning: implicit function declaration: vsnprintf "mpirun_util.c", line 33: warning: implicit function declaration: perror "mpirun_util.c", line 77: warning: implicit function declaration: strlen "mpirun_util.c", line 84: warning: implicit function declaration: strcat /opt/apps/sun/sunstudio12/bin/cc -DLD_LIBRARY_PATH_MPI=\"/tmp/ mvapich-1.0-2008-02-28/lib/shared\" -DMPI_PREFIX=\"/tmp/ mvapich-1.0-2008-02-28/\" -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/ etc/mvapich.conf\" -DVAPI -D_SMP_ -DUSE_SSH mpirun_rsh.c pmgr_collective_common.o pmgr_collective_mpirun.o minidaemon.o mpirun_util.o -lm -o mpirun_rsh "mpirun_rsh.c", line 579: warning: implicit function declaration: pmgr_processops mpirun_rsh.o(.text+0xf1b): In function `main': : undefined reference to `env2int' mpirun_rsh.o(.text+0x70cc): In function `mpispawn_checkin': : undefined reference to `env2int' make[4]: *** [mpirun_rsh] Error 1 make[3]: *** [mpirun_rsh] Error 2 Exit status from make was 2 make[2]: *** [mpilib] Error 1 make[1]: *** [mpi-modules] Error 2 make: *** [mpi] Error 2 Failure in building MVAPICH. Please file an error report to mvapich-discuss@cse.ohio-state.edu with all your log files. From mamidala at cse.ohio-state.edu Wed May 7 13:51:20 2008 From: mamidala at cse.ohio-state.edu (amith rajith mamidala) Date: Wed May 7 13:51:32 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: Message-ID: Hi, Can you try these two options? 1. use MVAPICH-1.0 and see if you are seeing the issue. 2.Can you run the code by passing the environment variable: VIADEV_USE_SHMEM_COLL=0 with mvapich Thanks, Amith On Wed, 7 May 2008 rcbord@wm.edu wrote: > Hi, > I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with > ofed 1.2 for infinaband. We have be able to use it for 6 months without > any problem. I tried to port a users fortran code that runs under > solaris-sparc without any problems. The code is fairly vanilla fotran-90 > with mpi I know because I wrote it. When I tried to port it to the Linux > Cluster it hangs a mpisendrecv function. I included a simple test code > that shows that the new communicator (comm2d) returned by the > MPI_Cart_create function has multiple values. Same code run on the > sun sparc cluster returns a single value. I am guessing it could be how > I compiled MVAPICH, but have tried a few more flags without any success. > I tried this with MVAPICH2 also and got the same error, but I compiled it > in a similar fashion. Could it be a PGI problem? > > > Has anyone else seen this? Thanks for any help! > > Program mpicart > ! With input files > ! Conversion of efit2d.f90 to 3d JPB 11-2007 > > Implicit none > > include "mpif.h" > > Interface > Subroutine read_model(Pdim0,Pdim1) > Integer, Intent(out) ::Pdim0,Pdim1 > end Subroutine read_model > end Interface > > Double Precision :: t1,t2 > ! > ! define MPI variables > ! > Integer :: Pid, N_proc, ierr > Integer :: comm2d = 0 > Integer :: status > Integer :: period(2),Pdim_size(2),coords(2) > Integer :: nbrleft,nbrright,nbrtop,nbrbottom > Integer :: Pdim0, Pdim1 > Integer :: i, t, numt > > call MPI_INIT(ierr) > call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) > call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) > > if (Pid .EQ. 0) then > call read_model(Pdim0,Pdim1) > > end if > ! > ! Broad cast input parameters values to all processors > ! > call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & > & MPI_COMM_WORLD,ierr) > call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & > & MPI_COMM_WORLD,ierr) > > call MPI_BARRIER(MPI_COMM_WORLD,ierr) > > Pdim_size(1) = Pdim0 > Pdim_size(2) = Pdim1 > > period(1) = 0 > period(2) = 0 > > call MPI_Dims_create(N_proc,2,Pdim_size,ierr) > > call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) > do i = 0,N_proc-1 > if (Pid == i) then > write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d > end if > end do > > call MPI_BARRIER(MPI_COMM_WORLD,ierr) > call MPI_Comm_free(comm2d,ierr) > call MPI_FINALIZE(ierr) > > end program mpicart > > ###################### OUTPUT ######################### > > Model parameters for mpicart test > > Processor Topology is ( 3 by 4 ) > > pid 0 mpi_comm_2d 140 > pid 11 mpi_comm_2d 138 > pid 2 mpi_comm_2d 140 > pid 4 mpi_comm_2d 140 > pid 6 mpi_comm_2d 140 > pid 8 mpi_comm_2d 140 > pid 3 mpi_comm_2d 138 > pid 10 mpi_comm_2d 140 > pid 1 mpi_comm_2d 138 > pid 5 mpi_comm_2d 138 > pid 7 mpi_comm_2d 138 > pid 9 mpi_comm_2d 138 > > > Chris Bording > Application Analyst > High Performance Computing Group > Information Technology > The College of William and Mary > (757)-221-3488 > rcbord@wm.edu > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From narravul at cse.ohio-state.edu Wed May 7 14:52:55 2008 From: narravul at cse.ohio-state.edu (Sundeep Narravula) Date: Wed May 7 14:53:05 2008 Subject: [mvapich-discuss] Using RDMA CM with MVAPICH2 In-Reply-To: <829ded920805062130s89f7e1eyaee3b4f6a5ab55ac@mail.gmail.com> Message-ID: Hi Mahesh, > Thanks Sundeep for replying. I've successfully tested RDMA CM support of > MVAPICH2 and it worked fine. Good to know that your installation went fine. > As far as I know, not many people are using the RDMA CM support of MVAPICH2. > So I want to ask you whether using RDMA CM support have any impact > on the performance of the MPI application AND How stable is the RDMA CM > support code of MVAPICH2 ? You should not see any performance changes while using RDMA CM. RDMA CM is used in connection setup only and does not impact the rest of the application execution. We have fully tested RDMA CM code at our end. This has been in our release for some time now and should be quite stable. Also, RDMA CM-based iWARP support provided by MVAPICH2 is being used/tested by others as well. Hope that helps. --Sundeep. > > -Mahesh > From rcbord at wm.edu Wed May 7 15:18:38 2008 From: rcbord at wm.edu (rcbord@wm.edu) Date: Wed May 7 15:18:52 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: References: Message-ID: Hi Amith, I addedd the VIADEV_USE_SHMEM_COLL=0 to the mvapich environment module and that worked as far as the new communicators all have the same value but there is still something wrong. Tried the full code and still hanging at the same mpi_sendrecv. What does the VIADEV_USE_SHMEM_COLL do? Did a little digging so now MPI_sendrecv is changing the values of destination and source values. Tried mvapich-1.0 but am getting an error with mpiexec... mpiexec: Warning: ib.c: protocol version 8 not known, but might still work. mpiexec: Error: ib.c: mixed version executables (6 and 8), no hope. Chris On Wed, 7 May 2008, amith rajith mamidala wrote: > Hi, > > Can you try these two options? > > 1. use MVAPICH-1.0 and see if you are seeing the issue. > > 2.Can you run the code by passing the environment variable: > VIADEV_USE_SHMEM_COLL=0 with mvapich > > > Thanks, > Amith > > > On Wed, 7 May 2008 rcbord@wm.edu wrote: >>> Hi, >> I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with >> ofed 1.2 for infinaband. We have be able to use it for 6 months without >> any problem. I tried to port a users fortran code that runs under >> solaris-sparc without any problems. The code is fairly vanilla fotran-90 >> with mpi I know because I wrote it. When I tried to port it to the Linux >> Cluster it hangs a mpisendrecv function. I included a simple test code >> that shows that the new communicator (comm2d) returned by the >> MPI_Cart_create function has multiple values. Same code run on the >> sun sparc cluster returns a single value. I am guessing it could be how >> I compiled MVAPICH, but have tried a few more flags without any success. >> I tried this with MVAPICH2 also and got the same error, but I compiled it >> in a similar fashion. Could it be a PGI problem? >> >> >> Has anyone else seen this? Thanks for any help! >> >> Program mpicart >> ! With input files >> ! Conversion of efit2d.f90 to 3d JPB 11-2007 >> >> Implicit none >> >> include "mpif.h" >> >> Interface >> Subroutine read_model(Pdim0,Pdim1) >> Integer, Intent(out) ::Pdim0,Pdim1 >> end Subroutine read_model >> end Interface >> >> Double Precision :: t1,t2 >> ! >> ! define MPI variables >> ! >> Integer :: Pid, N_proc, ierr >> Integer :: comm2d = 0 >> Integer :: status >> Integer :: period(2),Pdim_size(2),coords(2) >> Integer :: nbrleft,nbrright,nbrtop,nbrbottom >> Integer :: Pdim0, Pdim1 >> Integer :: i, t, numt >> >> call MPI_INIT(ierr) >> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) >> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) >> >> if (Pid .EQ. 0) then >> call read_model(Pdim0,Pdim1) >> >> end if >> ! >> ! Broad cast input parameters values to all processors >> ! >> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & >> & MPI_COMM_WORLD,ierr) >> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & >> & MPI_COMM_WORLD,ierr) >> >> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >> >> Pdim_size(1) = Pdim0 >> Pdim_size(2) = Pdim1 >> >> period(1) = 0 >> period(2) = 0 >> >> call MPI_Dims_create(N_proc,2,Pdim_size,ierr) >> >> call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) >> do i = 0,N_proc-1 >> if (Pid == i) then >> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d >> end if >> end do >> >> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >> call MPI_Comm_free(comm2d,ierr) >> call MPI_FINALIZE(ierr) >> >> end program mpicart >> >> ###################### OUTPUT ######################### >> >> Model parameters for mpicart test >> >> Processor Topology is ( 3 by 4 ) >> >> pid 0 mpi_comm_2d 140 >> pid 11 mpi_comm_2d 138 >> pid 2 mpi_comm_2d 140 >> pid 4 mpi_comm_2d 140 >> pid 6 mpi_comm_2d 140 >> pid 8 mpi_comm_2d 140 >> pid 3 mpi_comm_2d 138 >> pid 10 mpi_comm_2d 140 >> pid 1 mpi_comm_2d 138 >> pid 5 mpi_comm_2d 138 >> pid 7 mpi_comm_2d 138 >> pid 9 mpi_comm_2d 138 >> >> >> Chris Bording >> Application Analyst >> High Performance Computing Group >> Information Technology >> The College of William and Mary >> (757)-221-3488 >> rcbord@wm.edu >> _______________________________________________ >> mvapich-discuss mailing list >> mvapich-discuss@cse.ohio-state.edu >> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss >> > > > From Shuichi.Ihara at Sun.COM Thu May 8 00:24:31 2008 From: Shuichi.Ihara at Sun.COM (Shuichi Ihara) Date: Thu May 8 00:24:56 2008 Subject: [mvapich-discuss] Sun Studio build mvapich1 In-Reply-To: References: Message-ID: <4822807F.2010100@Sun.COM> Hi Frank, I have been successful to build mvapich-1.0 with SunStudio12 on the CentOS5.1 I did to pick up a mvapich spec file from the ofed-1.3 and same modifications for the SunStudio, then built it. Please see the attached diffs of specfile. Thanks, -Ihara Frank Leers wrote: > Hi, > > I am attempting to build mvapich 1 with Studio12 and am not having much > success. Could someone have a look and point out the err of my ways? > > A couple of things jump out at me - > > checking for cc... no > checking whether the Fortran 77 compiler > (/opt/apps/sun/sunstudio12/bin/f77 -L/opt/ofed/lib64 ) works... no > > OFED 1.2.5-5 was built with gnu, is that a problem? > > I've modified the make_mvapich.gen2 as follows: > > 18,25c18,24 > < STUDIOHOME=/opt/apps/sun/sunstudio12 > < IBHOME=${IBHOME:-/opt/ofed} > < IBHOME_LIB=${IBHOME_LIB:-/opt/ofed/lib64} > < PREFIX=${PREFIX:-/tmp/mvapich-1.0-2008-02-28} > < export CC=${CC:-$STUDIOHOME/bin/cc} > < export CXX=${CXX:-$STUDIOHOME/bin/CC} > < export F77=${F77:-$STUDIOHOME/bin/f77} > < export F90=${F90:-$STUDIOHOME/bin/f90} > --- > > IBHOME=${IBHOME:-/usr/local/ofed} > > IBHOME_LIB=${IBHOME_LIB:-/usr/local/ofed/lib64} > > PREFIX=${PREFIX:-/usr/local/mvapich} > > export CC=${CC:-gcc} > > export CXX=${CXX:-g++} > > export F77=${F77:-g77} > > export F90=${F90:-} > > > thanks, > > -frank > > > [mvapich-1.0-2008-02-28]# ./make.mvapich.gen2.studio12 > Configuring MVAPICH... > Configuring with args --with-device=ch_gen2 --with-arch=LINUX > -prefix=/tmp/mvapich-1.0-2008-02-28 --with-romio --without-mpe > -lib=-L/opt/ofed/lib64 -Wl,-rpath=/opt/ofed/lib64 -libverbs -libumad > -lpthread > Configuring MPICH Version 1.2.7 (release) of : 2005/06/22 16:33:49 > Make will not build MPE routines > checking whether filesystem respects case in file names... yes > checking for current directory name... /tmp/mvapich1/mvapich-1.0-2008-02-28 > checking for install > checking for ranlib > checking for gnumake... yes using --no-print-directory > checking whether make supports include... yes > checking for OSF V3 make... no > checking for virtual path format... VPATH > checking whether /opt/apps/sun/sunstudio12/bin/CC returns correct error > code... yes > checking whether selected C++ compiler can compile iostream.h... yes > > Include C++ bindings for MPI from http://www.osl.iu.edu/research/mpi2c++ > Send bug reports about the C++ to mpi2cpp-devel@osl.iu.edu > > checking for cc... no > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts > ANSI prototypes... yes > checking for /opt/apps/sun/sunstudio12/bin/f77... found > /opt/apps/sun/sunstudio12/bin/f77 (1) > checking for ar... found /usr/bin/ar (ar) > checking for whether compiler works... yes > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts > ## for concatenation in cpp... yes > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts > const modifier... yes > checking for extension on executables... none > checking how to run the C preprocessor > checking for AIX > checking for minix/config.h > checking for volatile... yes > Running device-specific setup program > setup_ch_gen2 starts... -D_X86_64_ -D_AMD_QUAD_CORE_ > -DEARLY_SEND_COMPLETION -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ > -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/opt/ofed/include -O3 > About to do ch_gen2 device > Configuring ch_gen2 subsystem... > No addition setup since mpd is not enabled > Use ${exec_prefix}/bin to set up variables MPIBOOT and MPIUNBOOT > checking whether cross-compiling... no > checking for gethrtime... no > checking for clock_gettime... no > checking for clock_getres... no > checking for how many arguments gettimeofday takes... two - whew > checking for signal.h... yes > checking for sigaction... yes > checking for for struct sigaction... yes > checking for weak symbol support... pragma weak > checking whether weak symbols are visible to other files... yes > checking for remsh... no > checking for rsh... found /usr/kerberos/bin/rsh (rsh) > checking for perl5... no > checking for perl... found /usr/bin/perl (perl) > checking for byte ordering... little endian > checking for uname... yes > checking for netdb.h... yes > checking for gethostbyname... yes > checking whether C++ compiler also has gethostbyname... yes > checking whether Fortran programs can link with needed C functions... yes > checking whether C++ programs can link with needed C functions... yes > checking for catopen... yes > checking for catclose... yes > checking for catgets... yes > checking for gencat... found /usr/bin/gencat (1) > checking for nl_types.h... yes > checking for ANSI C header files > checking for stdlib.h... yes > checking for string.h... yes > checking for unistd.h... yes > checking for stdarg.h... yes > checking whether stdarg is oldstyle... hmm, warnings from compiler. > Trying newstyle > checking whether stdarg is correct... yes > checking whether programs using stdarg link... yes > checking for malloc return type... void > checking for system... yes > checking for nice... yes > checking for strdup... yes > checking for memory.h... yes > checking for sys/ioctl.h... yes > checking for sys/sockio.h... no > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc runs... yes > checking whether the compiler /opt/apps/sun/sunstudio12/bin/f77 runs... yes > checking for correct handling of conditionals...... yes > checking for correct handling of conditionals part 2 ...... yes > checking for correct handling of #elif... yes > checking that constant folding in arrays works... yes > checking that structs are handled properly... yes > checking that compiler truncates unsigned char correctly ...... yes > checking that compiler/hardware has commutative arithmetic ...... yes > checking that compiler assigns to char correctly ...... yes > checking that compiler does not call unknown routines for long > doubles... yes > checking that compiler accepts args correctly ...... yes > checking whether ranlib works... yes > checking for broken handling of common symbols... no > checking for ar... found /usr/bin/ar (1) > checking for size of void *... 8 > checking for pointers greater than 32 bits... yes > checking for size of int... 4 > checking for int large enough for pointers... no > checking for C integer type for MPI_Aint... > checking for size of long... 8 > C integer type for MPI_Aint is... long > checking for long long... yes > checking whether compiler can handle loops with long long... yes > checking for long double... yes > checking for size of double... 8 > checking for size of long double... 16 > checking for structure alignment... largest > checking for sync... found /bin/sync (1) > checking whether mpich can be used as the library name... yes > Configuring Fortran subsystem > Configure in /tmp/mvapich1/mvapich-1.0-2008-02-28/src/fortran creating > cache ./config.cache > checking for cc... /opt/apps/sun/sunstudio12/bin/cc > checking whether the C compiler (/opt/apps/sun/sunstudio12/bin/cc > -D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H ) works... yes > checking whether the C compiler (/opt/apps/sun/sunstudio12/bin/cc > -D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H ) is a cross-compiler... no > checking whether the C compiler sets its return status correctly... yes > checking whether we are using GNU C... no > checking for working const... yes > checking whether the Fortran 77 compiler > (/opt/apps/sun/sunstudio12/bin/f77 -L/opt/ofed/lib64 ) works... no > configure: error: installation or configuration problem: Fortran 77 > compiler cannot create executables. > Error configuring the Fortran subsystem! > Turning off Fortran support > Configuring ROMIO, for support of MPI-IO > checking whether compiler allows define a(b) a(b,__LINE__)... yes > Configuring with args --with-mpi=mpich --cache-file=/dev/null > checking for architecture... LINUX > ROMIO home directory is /tmp/mvapich1/mvapich-1.0-2008-02-28/romio > checking for ranlib... ranlib > checking for gcc... /opt/apps/sun/sunstudio12/bin/cc > checking for C compiler default output file name... a.out > checking whether the C compiler works... yes > checking whether we are cross compiling... no > checking for suffix of executables... > checking for suffix of object files... o > checking whether we are using the GNU C compiler... no > checking whether /opt/apps/sun/sunstudio12/bin/cc accepts -g... yes > checking for /opt/apps/sun/sunstudio12/bin/cc option to accept ANSI C... > none needed > checking whether we are using the GNU Fortran 77 compiler... no > checking whether /opt/apps/sun/sunstudio12/bin/f77 accepts -g... yes > checking how to run the C preprocessor... > /opt/apps/sun/sunstudio12/bin/cc -E > checking for memalign... yes > checking Fortran external names > Fortran externals have a trailing underscore and are lowercase > checking for inline... inline > checking how to run the C preprocessor... > /opt/apps/sun/sunstudio12/bin/cc -E > checking for egrep... grep -E > checking for ANSI C header files... yes > checking for sys/types.h... yes > checking for sys/stat.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking for memory.h... yes > checking for strings.h... yes > checking for inttypes.h... yes > checking for stdint.h... yes > checking for unistd.h... yes > checking for unistd.h... (cached) yes > checking fcntl.h usability... yes > checking fcntl.h presence... yes > checking for fcntl.h... yes > checking malloc.h usability... yes > checking malloc.h presence... yes > checking for malloc.h... yes > checking stddef.h usability... yes > checking stddef.h presence... yes > checking for stddef.h... yes > checking for int... yes > checking size of int... 4 > checking for void*... yes > checking size of void*... 8 > checking for int large enough for pointers... no > checking for long long... yes > checking size of long long... 8 > checking for extension for Fortran 90 programs... f90 > checking for Fortran 90 KIND parameter for 8-byte integers... 8 > checking for weak symbol support... pragma weak > checking that weak symbols are visible to other files... yes > checking whether struct flock compatible with MPI_Offset... yes > checking configured file systems... testfs ufs nfs > configure: WARNING: File locks may not work with NFS. See the > Installation and > users manual for instructions on testing and if necessary fixing this > checking for library containing pthread_create... -lpthread > checking for library containing aio_write... -lrt > checking signal.h usability... yes > checking signal.h presence... yes > checking for signal.h... yes > checking if aio.h exists and can be compiled... yes > checking if sys/aio.h exists and can be compiled... no > checking whether aio routines can be used... yes > checking for obsolete two-argument aio_write... no > checking for obsolete two-argument aio_suspend... no > checking for aio_fildes member of aiocb structure... yes > checking for aio_whence member of aiocb structure... no > checking for aio_handle member of aiocb structure... no > checking for aio_reqprio member of aiocb structure... yes > checking for aio_sigevent member of aiocb structure... yes > checking sys/vfs.h usability... yes > checking sys/vfs.h presence... yes > checking for sys/vfs.h... yes > checking sys/param.h usability... yes > checking sys/param.h presence... yes > checking for sys/param.h... yes > checking sys/mount.h usability... yes > checking sys/mount.h presence... yes > checking for sys/mount.h... yes > checking sys/statvfs.h usability... yes > checking sys/statvfs.h presence... yes > checking for sys/statvfs.h... yes > checking whether struct statfs properly defined... yes > checking for f_fstypename member of statfs structure... no > checking for sys/stat.h... (cached) yes > checking for sys/types.h... (cached) yes > checking for unistd.h... (cached) yes > checking for stat... yes > checking for st_fstype member of stat structure... no > checking for sys/types.h... (cached) yes > checking for sys/statvfs.h... (cached) yes > checking for sys/vfs.h... (cached) yes > checking for statvfs... yes > checking for f_basetype member of statvfs structure... no > checking for unistd.h... (cached) yes > checking for large file defines... yes > checking whether off64_t is an scalar type... yes > checking for strerror... yes > checking for doctext... no > checking for strdup... no > checking for lstat... no > checking for readlink... no > checking for fsync... no > checking for ftruncate... no > setting CC to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc > setting F77 to /opt/apps/sun/sunstudio12/bin/f77 > setting TEST_CC to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc > setting TEST_F77 to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpif77 > setting CFLAGS to -D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION > -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 > -D_GNU_SOURCE -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H -DFORTRANUNDERSCORE > -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -DHAVE_ROMIOCONF_H > setting USER_CFLAGS to > setting USER_FFLAGS to > configure: creating ./config.status > config.status: creating Makefile > config.status: creating localdefs > config.status: creating mpi-io/Makefile > config.status: creating mpi2-other/info/Makefile > config.status: creating mpi2-other/array/Makefile > config.status: creating adio/common/Makefile > config.status: creating test/Makefile > config.status: creating test/misc.c > config.status: creating test/large_file.c > config.status: creating test/runtests > config.status: creating util/romioinstall > config.status: creating include/mpio.h > config.status: creating include/mpiof.h > config.status: creating adio/ad_nfs/Makefile > config.status: creating adio/ad_ufs/Makefile > config.status: creating adio/ad_panfs/Makefile > config.status: creating adio/ad_xfs/Makefile > config.status: creating adio/ad_sfs/Makefile > config.status: creating adio/ad_pfs/Makefile > config.status: creating adio/ad_testfs/Makefile > config.status: creating adio/ad_pvfs/Makefile > config.status: creating adio/ad_pvfs2/Makefile > config.status: creating adio/ad_gridftp/Makefile > config.status: creating adio/ad_lustre/Makefile > config.status: creating mpi-io/fortran/Makefile > config.status: creating mpi2-other/info/fortran/Makefile > config.status: creating mpi2-other/array/fortran/Makefile > config.status: creating test/fmisc.f > config.status: creating test/fcoll_test.f > config.status: creating test/pfcoll_test.f > config.status: creating test/fperf.f > config.status: creating mpi-io/glue/mpich2/Makefile > config.status: creating mpi-io/glue/mpich1/Makefile > config.status: creating mpi-io/glue/default/Makefile > config.status: creating adio/include/romioconf.h > config.status: executing default-1 commands > Finished configuring ROMIO > checking whether head command is broken... no > creating config.status > creating Makefile > creating util/Makefile > creating examples/Makefile > creating src/pt2pt/Makefile > creating src/coll/Makefile > creating src/context/Makefile > creating src/dmpi/Makefile > creating src/env/Makefile > creating src/topol/Makefile > creating src/profile/Makefile > creating src/coll2/Makefile > creating src/util/Makefile > creating src/infoexport/Makefile > creating examples/basic/Makefile > creating src/misc2/Makefile > creating src/external/Makefile > creating mpichconf.h > creating util/mpicc > creating util/mpicxx > creating util/mpiinstall > creating util/mpirun > creating util/mpichlib.conf > creating util/mpiman > creating util/chkserv > creating util/chkmachine > creating util/tstmachines > creating include/mpidefs.h > creating include/mpif.f90 > Configuring the test suite... > Configuring with args -basecc=/opt/apps/sun/sunstudio12/bin/cc > -cc=/tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc -make=make > --no-print-directory -device=ch_gen2 > -mpichpath=/tmp/mvapich1/mvapich-1.0-2008-02-28/bin --enable-boot > -cflags=-D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION > -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 > -D_GNU_SOURCE -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H -nof77 --enable-io > Don't build the tests that need Fortran > checking whether cross-compiling... no > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc runs... yes > checking for gnumake... no > checking whether make supports include... yes > checking for OSF V3 make... no > checking for virtual path format... VPATH > checking for stdlib.h... yes > checking for unistd.h... yes > checking for string.h... yes > checking for stdarg is correct... yes > checking for long double... yes > checking for long long... yes > checking whether compiler can handle loops with long long... yes > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc accepts > ANSI prototypes... yes > checking for signal.h... yes > checking for sigaction... yes > checking for sleep... yes > checking for sysconf... yes > creating config.status > creating Makefile > creating pt2pt/Makefile > creating coll/Makefile > creating topol/Makefile > creating context/Makefile > creating env/Makefile > creating profile/Makefile > creating io/Mfile > creating pt2pt/runtests > creating coll/runtests > creating topol/runtests > creating context/runtests > creating command/runtests > creating env/runtests > creating profile/runtests > creating io/rtest > creating io/misc.c > creating io/fmisc.f > creating io/fcoll_test.f > creating io/pfcoll_test.f > creating io/large_file.c > creating io/fperf.f > Configuring the performance tests ... > Configure in /tmp/mvapich1/mvapich-1.0-2008-02-28/examples/perftest > creating cache ./config.cache > checking for a BSD compatible install... /usr/bin/install -c > checking whether build environment is sane... yes > checking whether make --no-print-directory sets ${MAKE}... yes > checking for working aclocal... found > checking for working autoconf... found > checking for working automake... found > checking for working autoheader... found > checking for working makeinfo... found > checking whether clock skew breaks make... no > checking whether make supports include... yes > checking whether make allows comments in actions... yes > checking for virtual path format... VPATH > checking whether make sets CFLAGS... yes > checking for cit... no > checking for gnuplot... gnuplot > checking how to run the C preprocessor... > /opt/apps/sun/sunstudio12/bin/cc -E > checking whether C compiler accepts option -O4 -Ofast... no > checking whether C compiler accepts option -Ofast... no > checking whether C compiler accepts option -fast... yes > checking whether routines compiled with -fast can be linked with ones > compiled without -fast... yes > checking for library containing fabs... -lm > checking for ANSI C header files... yes > checking for sys/time.h... yes > checking for stdlib.h... yes > checking for string.h... yes > checking whether time.h and sys/time.h may both be included... yes > checking whether struct tm is in sys/time.h or time.h... time.h > checking for working const... yes > checking for restrict... restrict > checking for long long... yes > updating cache ./config.cache > creating ./config.status > creating Makefile > creating mpptestconf.h > creating mpid/ch_gen2/Makefile > creating mpid/tests/Makefile > creating mpichconf.h > mpichconf.h is unchanged > *# ---> > *# You should register your copy of MPICH with us by sending mail > *# to majordomo@mcs.anl.gov containing the message > *# subscribe mpi-users > *# This will allow us to notify you of new releases of MPICH. > *# > *# You can also check the MPICH home page at > *# http://www.mcs.anl.gov/mpi/mpich > *# ---< > Configuration completed. > Building MVAPICH... > > cleaning directory mpi-io > > cleaning directory adio/common > > cleaning directory mpi-io/glue/mpich1 > > cleaning directory adio/ad_testfs > > cleaning directory adio/ad_ufs > > cleaning directory adio/ad_nfs > > cleaning directory mpi-io/fortran > > cleaning directory test > rm -f .P* PI* *.o > rm -f simple perf async coll_test coll_perf misc file_info excl > large_array atomicity noncontig i_noncontig noncontig_coll split_coll > shared_fp large_file psimple error status noncontig_coll2 fcoll_test > fperf fmisc pfcoll_test > > cleaning src/pt2pt > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/pt2pt/*.o > cleaning src/env > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/env/*.o > cleaning src/dmpi > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/dmpi/*.o > cleaning src/util > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/util/*.o > cleaning src/context > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/context/*.o > cleaning src/coll > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/coll/*.o > cleaning src/topol > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/topol/*.o > cleaning src/profile > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/profile/*.o > cleaning src/misc2 > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/misc2/*.o > cleaning src/external > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/external/*.o > cleaning mpid/ch_gen2 > /bin/rm -f *.o *.d *~ viainit.o viasend.o viarecv.o viacheck.o viapriv.o > viaparam.o viutil.o vbuf.o cm_user.o mpid_init.o mpid_send.o mpid_recv.o > cm.o mpid_hsend.o mpid_hrecv.o mpid_pack.o cmnargs.o mpid_misc.o dreg.o > mpid_smpi.o collutils.o intra_rdma_barrier.o mpid_mcast.o mcst_grp.o > ibmcgrp.o crc32h.o avl.o mem_hooks.o viacoalesce.o shmem_coll.o > async_progress.o \ > queue.o sbcnst2.o tr2.o queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h core > ibmcgrp/*.o ibmcgrp/ibmcgrp \ > /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/ibmcgrp > make --no-print-directory -C process clean > rm -f *.o mpirun_rsh mpispawn minidaemon_client pmgr_collective_client.o > pmgr_collective_common.o minidaemon.o mpirun_util.o > cleaning examples > rm -f sendchar *.o > > Making clean in directory test > > Making clean in directory pt2pt > > Making clean in directory coll > > Making clean in directory topol > > Making clean in directory env > > Making clean in directory context > > Making clean in directory profile > > Making clean in directory io > > Making clean in directory perftest > test -z "mpptest goptest buflimit " || rm -f mpptest goptest buflimit > rm -f *.o core *.core > rm -f tunepkt pktuse copytest vectest stress ctest cluster tcomm > rm -f work.pc work.pcl > cleaning examples/basic > rm -f work.pc work.pcl > rm -f *.o *~ PI* cpi systest srtest fpi cpilog cpi_autolog hello++ > iotest pi3f90 upshot rdb.* startup.* core > rm -f hello++.ti hello++.ii > cleaning examples/test/pt2pt > cleaning examples/test/coll > cleaning examples/test/topol > cleaning examples/test/context > cleaning examples/test/env > cleaning examples/test/profile > cleaning examples/test > > Making clean in directory pt2pt > > Making clean in directory coll > > Making clean in directory topol > > Making clean in directory env > > Making clean in directory context > > Making clean in directory profile > > Making clean in directory io > cleaning examples/perftest > test -z "mpptest goptest buflimit " || rm -f mpptest goptest buflimit > rm -f *.o core *.core > rm -f tunepkt pktuse copytest vectest stress ctest cluster tcomm > rm -f work.pc work.pcl > cleaning src/infoexport > rm -f *.o > rm -f *.i > rm -f *.s > rm -f *.cxx.log > rm -f *.cxx.errors > rm -f libtvmpich.so.1.0 > rm -f libtvmpich.so* > rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/src/infoexport/*.o > rm -f *~ *.o aditest1 aditest2 aditest3 aditest4 aditest5 aditest6 > aditest7 aditest8 aditest9 aditest10 aditest11 aditest12 aditest13 > timers trunc > rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/lib*.a > rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/shared/lib*.so* > make --no-print-directory mpi-modules > make --no-print-directory mpilib > for file in queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h ; do \ > if [ ! -s $file ] ; then \ > ln -s ../util/$file; \ > fi; \ > done > > making mpir in directory mpid/ch_gen2 > for file in queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h ; do \ > if [ ! -s $file ] ; then \ > ln -s ../util/$file; \ > fi; \ > done > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viainit.c > "ib_init.h", line 12: warning: initializer does not fit or is out of > range: -1 > "ib_init.h", line 34: warning: improper pointer/integer combination: op "<" > "viainit.c", line 149: warning: argument #3 is incompatible with prototype: > prototype: pointer to const struct {array[16] of unsigned long > __bits} : "/usr/include/sched.h", line 76 > argument : pointer to unsigned long > "viainit.c", line 162: warning: argument #3 is incompatible with prototype: > prototype: pointer to const struct {array[16] of unsigned long > __bits} : "/usr/include/sched.h", line 76 > argument : pointer to unsigned long > "viainit.c", line 556: warning: argument #3 is incompatible with prototype: > prototype: pointer to function(pointer to void) returning pointer to > void : "/usr/include/pthread.h", line 163 > argument : pointer to void > "viainit.c", line 792: warning: implicit function declaration: > init_apm_lock > "viainit.c", line 990: warning: argument mismatch > "viainit.c", line 1158: warning: implicit function declaration: > reload_alternate_path > "viainit.c", line 1221: warning: argument #3 is incompatible with > prototype: > prototype: pointer to function(pointer to void) returning pointer to > void : "/usr/include/pthread.h", line 163 > argument : pointer to void > "viainit.c", line 1345: warning: argument mismatch > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viasend.c > "viasend.c", line 145: warning: implicit function declaration: > viadev_post_send_async_qp > "viasend.c", line 401: warning: argument mismatch > "viasend.c", line 703: warning: argument mismatch > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viarecv.c > "viarecv.c", line 263: warning: argument mismatch > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viacheck.c > "viacheck.c", line 571: warning: argument mismatch > "viacheck.c", line 1824: warning: argument mismatch > "viacheck.c", line 1846: warning: argument mismatch > "viacheck.c", line 1861: warning: argument mismatch > "viacheck.c", line 2260: warning: implicit function declaration: > perform_manual_apm > "viacheck.c", line 2856: warning: implicit function declaration: > reload_alternate_path > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viapriv.c > "viapriv.c", line 649: warning: implicit function declaration: > perform_manual_apm > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viaparam.c > "/opt/ofed/include/infiniband/common.h", line 60: warning: implicit > function declaration: bswap_64 > "/opt/ofed/include/infiniband/umad.h", line 72: warning: zero or > negative subscript > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viutil.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c vbuf.c > "vbuf.c", line 125: warning: argument mismatch > "vbuf.c", line 283: warning: argument mismatch > "vbuf.c", line 442: warning: argument mismatch > "vbuf.c", line 488: warning: argument mismatch > "vbuf.c", line 491: warning: argument mismatch > "vbuf.c", line 492: warning: argument mismatch > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c cm_user.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_init.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_send.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_recv.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c cm.c > "ib_init.h", line 12: warning: initializer does not fit or is out of > range: -1 > "ib_init.h", line 34: warning: improper pointer/integer combination: op "<" > "cm.c", line 176: warning: pointer to void or function used in arithmetic > "cm.c", line 265: warning: implicit function declaration: > reload_alternate_path > "cm.c", line 677: warning: statement not reached > "cm.c", line 735: warning: pointer to void or function used in arithmetic > "cm.c", line 735: warning: pointer to void or function used in arithmetic > "cm.c", line 744: warning: pointer to void or function used in arithmetic > "cm.c", line 757: warning: statement not reached > "cm.c", line 830: warning: pointer to void or function used in arithmetic > "cm.c", line 830: warning: pointer to void or function used in arithmetic > "cm.c", line 925: warning: pointer to void or function used in arithmetic > "cm.c", line 1006: warning: pointer to void or function used in arithmetic > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_hsend.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_hrecv.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_pack.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c cmnargs.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_misc.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c dreg.c > "dreg.c", line 132: warning: initializer does not fit or is out of > range: -1 > "dreg.c", line 780: warning: argument mismatch > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_smpi.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c collutils.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c > intra_rdma_barrier.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mpid_mcast.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mcst_grp.c > "mcst_grp.c", line 338: warning: empty translation unit > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c ibmcgrp.c > "ibmcgrp.c", line 480: warning: empty translation unit > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c crc32h.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c avl.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c mem_hooks.c > "mem_hooks.c", line 34: warning: assignment type mismatch: > pointer to function(pointer to void, unsigned long) returning int > "=" pointer to void > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c viacoalesce.c > "viacoalesce.c", line 70: warning: argument mismatch > "viacoalesce.c", line 97: warning: argument mismatch > "viacoalesce.c", line 167: warning: argument mismatch > "viacoalesce.c", line 231: warning: argument mismatch > "viacoalesce.c", line 235: warning: argument mismatch > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c shmem_coll.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c > async_progress.c > "ib_init.h", line 12: warning: initializer does not fit or is out of > range: -1 > "ib_init.h", line 34: warning: improper pointer/integer combination: op "<" > "async_progress.c", line 102: warning: function expects to return value: > atomic_send_or_queue > "async_progress.c", line 112: warning: implicit function declaration: > perform_manual_apm > "async_progress.c", line 686: warning: argument #3 is incompatible with > prototype: > prototype: pointer to function(pointer to void) returning pointer to > void : "/usr/include/pthread.h", line 163 > argument : pointer to void > "async_progress.c", line 841: warning: implicit function declaration: > reload_alternate_path > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c queue.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c sbcnst2.c > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util -DMPID_DEVICE_CODE > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE -D_X86_64_ > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 -I. -c tr2.c > ar crl /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/libmpich.a viainit.o > viasend.o viarecv.o viacheck.o viapriv.o viaparam.o viutil.o vbuf.o > cm_user.o mpid_init.o mpid_send.o mpid_recv.o cm.o mpid_hsend.o > mpid_hrecv.o mpid_pack.o cmnargs.o mpid_misc.o dreg.o mpid_smpi.o > collutils.o intra_rdma_barrier.o mpid_mcast.o mcst_grp.o ibmcgrp.o > crc32h.o avl.o mem_hooks.o viacoalesce.o shmem_coll.o async_progress.o > queue.o sbcnst2.o tr2.o > cd process && make --no-print-directory &&\ > install -m 755 mpirun_rsh mpispawn minidaemon_client > /tmp/mvapich1/mvapich-1.0-2008-02-28/bin > /opt/apps/sun/sunstudio12/bin/cc > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02-28/lib/shared\" > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/etc/mvapich.conf\" -DVAPI > -D_SMP_ -DUSE_SSH -c -o pmgr_collective_common.o > pmgr_collective_common.c > /opt/apps/sun/sunstudio12/bin/cc > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02-28/lib/shared\" > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/etc/mvapich.conf\" -DVAPI > -D_SMP_ -DUSE_SSH -c -o pmgr_collective_mpirun.o > pmgr_collective_mpirun.c > "pmgr_collective_mpirun.c", line 75: warning: pointer to void or > function used in arithmetic > "pmgr_collective_mpirun.c", line 97: warning: pointer to void or > function used in arithmetic > "pmgr_collective_mpirun.c", line 98: warning: pointer to void or > function used in arithmetic > "pmgr_collective_mpirun.c", line 250: warning: pointer to void or > function used in arithmetic > "pmgr_collective_mpirun.c", line 263: warning: pointer to void or > function used in arithmetic > "pmgr_collective_mpirun.c", line 269: warning: pointer to void or > function used in arithmetic > /opt/apps/sun/sunstudio12/bin/cc > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02-28/lib/shared\" > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/etc/mvapich.conf\" -DVAPI > -D_SMP_ -DUSE_SSH -c -o minidaemon.o minidaemon.c > "minidaemon.c", line 267: warning: argument mismatch > "minidaemon.c", line 300: warning: argument mismatch > "minidaemon.c", line 407: warning: argument mismatch > "minidaemon.c", line 430: warning: argument mismatch > "minidaemon.c", line 455: warning: argument mismatch > "minidaemon.c", line 542: warning: argument mismatch > "minidaemon.c", line 552: warning: argument mismatch > "minidaemon.c", line 560: warning: argument mismatch > "minidaemon.c", line 591: warning: pointer to void or function used in > arithmetic > "minidaemon.c", line 679: warning: argument mismatch > "minidaemon.c", line 781: warning: enum type mismatch: op "=" > "minidaemon.c", line 812: warning: argument mismatch > "minidaemon.c", line 842: warning: enum type mismatch: op "=" > "minidaemon.c", line 854: warning: enum type mismatch: op "=" > "minidaemon.c", line 1072: warning: argument mismatch > "minidaemon.c", line 1114: warning: argument mismatch > "minidaemon.c", line 1130: warning: argument mismatch > "minidaemon.c", line 1213: warning: argument mismatch > "minidaemon.c", line 1214: warning: enum type mismatch: op "=" > "minidaemon.c", line 1222: warning: argument mismatch > "minidaemon.c", line 1244: warning: argument mismatch > "minidaemon.c", line 1247: warning: argument mismatch > "minidaemon.c", line 1278: warning: argument mismatch > "minidaemon.c", line 1281: warning: argument mismatch > "minidaemon.c", line 1288: warning: argument mismatch > "minidaemon.c", line 1309: warning: argument mismatch > "minidaemon.c", line 1369: warning: argument mismatch > "minidaemon.c", line 1417: warning: argument mismatch > "minidaemon.c", line 1443: warning: argument mismatch > "minidaemon.c", line 1492: warning: argument mismatch > "minidaemon.c", line 1533: warning: argument mismatch > /opt/apps/sun/sunstudio12/bin/cc > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02-28/lib/shared\" > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/etc/mvapich.conf\" -DVAPI > -D_SMP_ -DUSE_SSH -c -o mpirun_util.o mpirun_util.c > "mpirun_util.c", line 25: warning: implicit function declaration: vsnprintf > "mpirun_util.c", line 33: warning: implicit function declaration: perror > "mpirun_util.c", line 77: warning: implicit function declaration: strlen > "mpirun_util.c", line 84: warning: implicit function declaration: strcat > /opt/apps/sun/sunstudio12/bin/cc > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02-28/lib/shared\" > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02-28/etc/mvapich.conf\" -DVAPI > -D_SMP_ -DUSE_SSH mpirun_rsh.c pmgr_collective_common.o > pmgr_collective_mpirun.o minidaemon.o mpirun_util.o -lm -o mpirun_rsh > "mpirun_rsh.c", line 579: warning: implicit function declaration: > pmgr_processops > mpirun_rsh.o(.text+0xf1b): In function `main': > : undefined reference to `env2int' > mpirun_rsh.o(.text+0x70cc): In function `mpispawn_checkin': > : undefined reference to `env2int' > make[4]: *** [mpirun_rsh] Error 1 > make[3]: *** [mpirun_rsh] Error 2 > Exit status from make was 2 > make[2]: *** [mpilib] Error 1 > make[1]: *** [mpi-modules] Error 2 > make: *** [mpi] Error 2 > Failure in building MVAPICH. > Please file an error report to mvapich-discuss@cse.ohio-state.edu with > all your log files. > > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss -------------- next part -------------- --- /usr/src/redhat/SPECS/mvapich.spec 2008-02-24 21:21:23.000000000 +0900 +++ mvapich_sun.spec 2008-05-07 12:41:49.000000000 +0900 @@ -187,6 +187,21 @@ export CONFIG_FLAGS="" export OPTIMIZATION_FLAG="" %endif +# +# Sun Studio compiler +# +%if %(test "%{compiler}" = "sun" && echo 1 || echo 0) + export CC=suncc + export CXX=sunCC + export F77=sunf77 + export F90=sunf90 + export CFLAGS= + export FFLAGS= + export CXXFLAGS="" + export F90FLAGS=$FFLAGS + export CONFIG_FLAGS="" + export OPTIMIZATION_FLAG="-O3" +%endif ############################################################################# EXTRA_CFLAG=$CFLAGS @@ -259,7 +274,11 @@ CFLAGS="-DCOMPAT_MODE $CFLAGS" %endif -export CFLAGS="$OPTIMIZATION_FLAG -g -D_GNU_SOURCE -DCH_GEN2 -DMEMORY_SCALE -D_AFFINITY_ $CFLAGS -D_SMP_ -D_SMP_RNDV_ -DVIADEV_RPUT_SUPPORT -DEARLY_SEND_COMPLETION -DLAZY_MEM_UNREGISTER $ARCH_NAME -I$IB_INCLUDE" +#export CFLAGS="$OPTIMIZATION_FLAG -g -D_GNU_SOURCE -DCH_GEN2 -DMEMORY_SCALE -D_AFFINITY_ $CFLAGS -D_SMP_ -D_SMP_RNDV_ -DVIADEV_RPUT_SUPPORT -DEARLY_SEND_COMPLETION -DLAZY_MEM_UNREGISTER $ARCH_NAME -I$IB_INCLUDE" +# +# for Sun Studio compiler +# +export CFLAGS="$OPTIMIZATION_FLAG -DRDMA_FAST_PATH -DEARLY_SEND_COMPLETION -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2_MRAIL -DLAZY_MEM_UNREGISTER ${COMPILER_FLAG} $ARCH_NAME -I${IB_INCLUDE}" export USER_CFLAGS export MPE_OPTS export MPE_CFLAGS @@ -269,7 +288,11 @@ export F90FLAGS="$F90FLAGS $EXTRA_CFLAG" export CONFIG_FLAGS export MPIRUN_CFLAGS="$MPIRUN_CFLAGS -DPARAM_GLOBAL=\\\"%{_prefix}/etc/$conffile\\\" -DLD_LIBRARY_PATH_MPI=\\\"%{_prefix}/lib/shared\\\" -DMPI_PREFIX=\\\"%{_prefix}/\\\" $DEF_BUILDID" -./configure --enable-sharedlib --with-device=%{mpi_device} --with-arch=LINUX --prefix=%{build_root}%{_prefix} $CONFIG_ENABLE_F77 $CONFIG_ENABLE_F90 $COMPILER_CONFIG -lib="-L$IB_LIB -libverbs -libumad -libcommon -lpthread $EXTRA_CFLAG" $MPE_FLAGS $CONFIG_FLAGS +#./configure --enable-sharedlib --with-device=%{mpi_device} --with-arch=LINUX --prefix=%{build_root}%{_prefix} $CONFIG_ENABLE_F77 $CONFIG_ENABLE_F90 $COMPILER_CONFIG -lib="-L$IB_LIB -libverbs -libumad -libcommon -lpthread $EXTRA_CFLAG" $MPE_FLAGS $CONFIG_FLAGS +# +# for Sun Studio compiler +# +./configure --with-device=%{mpi_device} --with-arch=LINUX --prefix=%{build_root}%{_prefix} $CONFIG_ENABLE_F77 $CONFIG_ENABLE_F90 $COMPILER_CONFIG -lib="-L$IB_LIB -libverbs -libumad -libcommon -lpthread $EXTRA_CFLAG" $MPE_FLAGS $CONFIG_FLAGS --without-mpe %{__make} ############################################################################# From koop at cse.ohio-state.edu Thu May 8 12:38:57 2008 From: koop at cse.ohio-state.edu (Matthew Koop) Date: Thu May 8 12:39:09 2008 Subject: [mvapich-discuss] IBV_EVENT_QP_LAST_WQE_REACHED error In-Reply-To: Message-ID: Just to update the list -- this has been tracked down to system calls in the application and an old kernel that does not have the fork support for OpenFabrics. Matt On Wed, 7 May 2008, Matthew Koop wrote: > Bill, > > The error described in the link you gave was the result of some 'fork's or > system calls in the code (that did a fork). Does your code contain any > system calls or forks? If so, please look at section 7.1.2 of the user > guide: > > https://mvapich.cse.ohio-state.edu/support/mvapich_user_guide.html > > Also, you may want to consider upgrading to OFED 1.3 and MVAPICH 1.0, > which has additional scalability and stability enhancements. > > Let us know how things go. Thanks, > > Matt > > > > On Tue, 6 May 2008, Bill Riner wrote: > > > We upgraded from IBGold to OFED 1.2.5.5 and we also get the "Got FATAL event > > IBV_EVENT_QP_LAST_WQE_REACHED" error that's discussed in this thread: > > > > > > > > http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2007-October/001139 > > .html > > > > > > > > [0:compute-0-23.local] Abort: [0] Got FATAL event > > IBV_EVENT_QP_LAST_WQE_REACHED, code=16 > > > > at line 2555 in file viacheck.c > > > > > > > > The particular code worked fine before the upgrade. The error doesn't occur > > when using just one processor. MPICH works fine on any number of > > processors. The OSU and Intel MPI benchmarks work fine. Other MPI codes > > work fine also. We're using MVAPICH 0.9.9 that came with OFED. > > > > > > > > Has this been resolved? > > > > > > > > > > > > > > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From chai.15 at osu.edu Thu May 8 13:50:17 2008 From: chai.15 at osu.edu (LEI CHAI) Date: Thu May 8 13:51:21 2008 Subject: [mvapich-discuss] Sun Studio build mvapich1 In-Reply-To: <4822807F.2010100@Sun.COM> References: <4822807F.2010100@Sun.COM> Message-ID: An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080508/29d6738e/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: mvapich.sun.diff Type: text/x-patch Size: 1020 bytes Desc: not available Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080508/29d6738e/mvapich.sun-0001.bin From Frank.Leers at Sun.COM Thu May 8 16:05:25 2008 From: Frank.Leers at Sun.COM (Frank Leers) Date: Thu May 8 16:05:53 2008 Subject: [mvapich-discuss] Sun Studio build mvapich1 In-Reply-To: References: <4822807F.2010100@Sun.COM> Message-ID: <2E3E775F-8BF5-4069-8140-CB7B014B88CF@Sun.COM> Thank you both - Lei and Ihara! The patch and change below allowed me to build for gen2 device. I'll pursue multi-rail in the near future. -frank On May 8, 2008, at 10:50 AM, LEI CHAI wrote: > Hi Frank, > > The solution Ihara suggested below will enable mvapich to be > compiled with the gen2_multirail device with sun studio compiler. If > you want to use the gen2 device, please apply the patch I'm > attaching in this email. You also need to make a minor change to > make.mvapich.gen2 to enable f77, change > > export LIBS=${LIBS:--L${IBHOME_LIB} -Wl,-rpah=${IBHOME_LIB} - > libverbs -libumad -lpthread} > > to > > export LIBS=${LIBS:--L${IBHOME_LIB} -R${IBHOME_LIB} -libverbs - > libumad -lpthread} > > MVAPICH should compile successfully with these two changes. The way > you specified compilers etc below looks correct. > > Lei > > ----- Original Message ----- > From: Shuichi Ihara > Date: Thursday, May 8, 2008 12:25 am > Subject: Re: [mvapich-discuss] Sun Studio build mvapich1 > To: Frank Leers > Cc: mvapich-discuss@cse.ohio-state.edu > > > > > Hi Frank, > > > > I have been successful to build mvapich-1.0 with SunStudio12 on > > the CentOS5.1 > > I did to pick up a mvapich spec file from the ofed-1.3 and same > > modificationsfor the SunStudio, then built it. Please see the > > attached diffs of specfile. > > > > Thanks, > > > > -Ihara > > > > Frank Leers wrote: > > > Hi, > > > > > > I am attempting to build mvapich 1 with Studio12 and am not > > having much > > > success. Could someone have a look and point out the err > > of my ways? > > > > > > A couple of things jump out at me - > > > > > > checking for cc... no > > > checking whether the Fortran 77 compiler > > > (/opt/apps/sun/sunstudio12/bin/f77 -L/opt/ofed/lib64 ) > > works... no > > > > > > OFED 1.2.5-5 was built with gnu, is that a problem? > > > > > > I've modified the make_mvapich.gen2 as follows: > > > > > > 18,25c18,24 > > > < STUDIOHOME=/opt/apps/sun/sunstudio12 > > > < IBHOME=${IBHOME:-/opt/ofed} > > > < IBHOME_LIB=${IBHOME_LIB:-/opt/ofed/lib64} > > > < PREFIX=${PREFIX:-/tmp/mvapich-1.0-2008-02-28} > > > < export CC=${CC:-$STUDIOHOME/bin/cc} > > > < export CXX=${CXX:-$STUDIOHOME/bin/CC} > > > < export F77=${F77:-$STUDIOHOME/bin/f77} > > > < export F90=${F90:-$STUDIOHOME/bin/f90} > > > --- > > > > IBHOME=${IBHOME:-/usr/local/ofed} > > > > IBHOME_LIB=${IBHOME_LIB:-/usr/local/ofed/lib64} > > > > PREFIX=${PREFIX:-/usr/local/mvapich} > > > > export CC=${CC:-gcc} > > > > export CXX=${CXX:-g++} > > > > export F77=${F77:-g77} > > > > export F90=${F90:-} > > > > > > > > > thanks, > > > > > > -frank > > > > > > > > > [mvapich-1.0-2008-02-28]# ./make.mvapich.gen2.studio12 > > > Configuring MVAPICH... > > > Configuring with args --with-device=ch_gen2 --with-arch=LINUX > > > -prefix=/tmp/mvapich-1.0-2008-02-28 --with-romio --without-mpe > > > -lib=-L/opt/ofed/lib64 -Wl,-rpath=/opt/ofed/lib64 -libverbs - > > libumad > > > -lpthread > > > Configuring MPICH Version 1.2.7 (release) of : 2005/06/22 16:33:49 > > > Make will not build MPE routines > > > checking whether filesystem respects case in file names... yes > > > checking for current directory name... /tmp/mvapich1/mvapich- > > 1.0-2008-02-28 > > > checking for install > > > checking for ranlib > > > checking for gnumake... yes using --no-print-directory > > > checking whether make supports include... yes > > > checking for OSF V3 make... no > > > checking for virtual path format... VPATH > > > checking whether /opt/apps/sun/sunstudio12/bin/CC returns > > correct error > > > code... yes > > > checking whether selected C++ compiler can compile > > iostream.h... yes > > > > > > Include C++ bindings for MPI from > > http://www.osl.iu.edu/research/mpi2c++> Send bug reports about > > the C++ to mpi2cpp-devel@osl.iu.edu > > > > > > checking for cc... no > > > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc > > accepts > > > ANSI prototypes... yes > > > checking for /opt/apps/sun/sunstudio12/bin/f77... found > > > /opt/apps/sun/sunstudio12/bin/f77 (1) > > > checking for ar... found /usr/bin/ar (ar) > > > checking for whether compiler works... yes > > > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc > > accepts > > > ## for concatenation in cpp... yes > > > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc > > accepts > > > const modifier... yes > > > checking for extension on executables... none > > > checking how to run the C preprocessor > > > checking for AIX > > > checking for minix/config.h > > > checking for volatile... yes > > > Running device-specific setup program > > > setup_ch_gen2 starts... -D_X86_64_ -D_AMD_QUAD_CORE_ > > > -DEARLY_SEND_COMPLETION -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT - > > D_SMP_ > > > -D_SMP_RNDV_ -DCH_GEN2 -D_GNU_SOURCE -I/opt/ofed/include -O3 > > > About to do ch_gen2 device > > > Configuring ch_gen2 subsystem... > > > No addition setup since mpd is not enabled > > > Use ${exec_prefix}/bin to set up variables MPIBOOT and MPIUNBOOT > > > checking whether cross-compiling... no > > > checking for gethrtime... no > > > checking for clock_gettime... no > > > checking for clock_getres... no > > > checking for how many arguments gettimeofday takes... two - whew > > > checking for signal.h... yes > > > checking for sigaction... yes > > > checking for for struct sigaction... yes > > > checking for weak symbol support... pragma weak > > > checking whether weak symbols are visible to other files... yes > > > checking for remsh... no > > > checking for rsh... found /usr/kerberos/bin/rsh (rsh) > > > checking for perl5... no > > > checking for perl... found /usr/bin/perl (perl) > > > checking for byte ordering... little endian > > > checking for uname... yes > > > checking for netdb.h... yes > > > checking for gethostbyname... yes > > > checking whether C++ compiler also has gethostbyname... yes > > > checking whether Fortran programs can link with needed C > > functions... yes > > > checking whether C++ programs can link with needed C > > functions... yes > > > checking for catopen... yes > > > checking for catclose... yes > > > checking for catgets... yes > > > checking for gencat... found /usr/bin/gencat (1) > > > checking for nl_types.h... yes > > > checking for ANSI C header files > > > checking for stdlib.h... yes > > > checking for string.h... yes > > > checking for unistd.h... yes > > > checking for stdarg.h... yes > > > checking whether stdarg is oldstyle... hmm, warnings from > > compiler. > > > Trying newstyle > > > checking whether stdarg is correct... yes > > > checking whether programs using stdarg link... yes > > > checking for malloc return type... void > > > checking for system... yes > > > checking for nice... yes > > > checking for strdup... yes > > > checking for memory.h... yes > > > checking for sys/ioctl.h... yes > > > checking for sys/sockio.h... no > > > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc > > runs... yes > > > checking whether the compiler > > /opt/apps/sun/sunstudio12/bin/f77 runs... yes > > > checking for correct handling of conditionals...... yes > > > checking for correct handling of conditionals part 2 ...... yes > > > checking for correct handling of #elif... yes > > > checking that constant folding in arrays works... yes > > > checking that structs are handled properly... yes > > > checking that compiler truncates unsigned char correctly > > ...... yes > > > checking that compiler/hardware has commutative arithmetic > > ...... yes > > > checking that compiler assigns to char correctly ...... yes > > > checking that compiler does not call unknown routines for long > > > doubles... yes > > > checking that compiler accepts args correctly ...... yes > > > checking whether ranlib works... yes > > > checking for broken handling of common symbols... no > > > checking for ar... found /usr/bin/ar (1) > > > checking for size of void *... 8 > > > checking for pointers greater than 32 bits... yes > > > checking for size of int... 4 > > > checking for int large enough for pointers... no > > > checking for C integer type for MPI_Aint... > > > checking for size of long... 8 > > > C integer type for MPI_Aint is... long > > > checking for long long... yes > > > checking whether compiler can handle loops with long long... yes > > > checking for long double... yes > > > checking for size of double... 8 > > > checking for size of long double... 16 > > > checking for structure alignment... largest > > > checking for sync... found /bin/sync (1) > > > checking whether mpich can be used as the library name... yes > > > Configuring Fortran subsystem > > > Configure in /tmp/mvapich1/mvapich-1.0-2008-02-28/src/fortran > > creating > > > cache ./config.cache > > > checking for cc... /opt/apps/sun/sunstudio12/bin/cc > > > checking whether the C compiler > > (/opt/apps/sun/sunstudio12/bin/cc > > > -D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION - > > DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H ) works... yes > > > checking whether the C compiler > > (/opt/apps/sun/sunstudio12/bin/cc > > > -D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION - > > DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H ) is a cross- > > compiler... no > > > checking whether the C compiler sets its return status > > correctly... yes > > > checking whether we are using GNU C... no > > > checking for working const... yes > > > checking whether the Fortran 77 compiler > > > (/opt/apps/sun/sunstudio12/bin/f77 -L/opt/ofed/lib64 ) > > works... no > > > configure: error: installation or configuration problem: > > Fortran 77 > > > compiler cannot create executables. > > > Error configuring the Fortran subsystem! > > > Turning off Fortran support > > > Configuring ROMIO, for support of MPI-IO > > > checking whether compiler allows define a(b) a(b,__LINE__)... yes > > > Configuring with args --with-mpi=mpich --cache-file=/dev/null > > > checking for architecture... LINUX > > > ROMIO home directory is /tmp/mvapich1/mvapich-1.0-2008-02-28/romio > > > checking for ranlib... ranlib > > > checking for gcc... /opt/apps/sun/sunstudio12/bin/cc > > > checking for C compiler default output file name... a.out > > > checking whether the C compiler works... yes > > > checking whether we are cross compiling... no > > > checking for suffix of executables... > > > checking for suffix of object files... o > > > checking whether we are using the GNU C compiler... no > > > checking whether /opt/apps/sun/sunstudio12/bin/cc accepts - > > g... yes > > > checking for /opt/apps/sun/sunstudio12/bin/cc option to accept > > ANSI C... > > > none needed > > > checking whether we are using the GNU Fortran 77 compiler... no > > > checking whether /opt/apps/sun/sunstudio12/bin/f77 accepts - > > g... yes > > > checking how to run the C preprocessor... > > > /opt/apps/sun/sunstudio12/bin/cc -E > > > checking for memalign... yes > > > checking Fortran external names > > > Fortran externals have a trailing underscore and are lowercase > > > checking for inline... inline > > > checking how to run the C preprocessor... > > > /opt/apps/sun/sunstudio12/bin/cc -E > > > checking for egrep... grep -E > > > checking for ANSI C header files... yes > > > checking for sys/types.h... yes > > > checking for sys/stat.h... yes > > > checking for stdlib.h... yes > > > checking for string.h... yes > > > checking for memory.h... yes > > > checking for strings.h... yes > > > checking for inttypes.h... yes > > > checking for stdint.h... yes > > > checking for unistd.h... yes > > > checking for unistd.h... (cached) yes > > > checking fcntl.h usability... yes > > > checking fcntl.h presence... yes > > > checking for fcntl.h... yes > > > checking malloc.h usability... yes > > > checking malloc.h presence... yes > > > checking for malloc.h... yes > > > checking stddef.h usability... yes > > > checking stddef.h presence... yes > > > checking for stddef.h... yes > > > checking for int... yes > > > checking size of int... 4 > > > checking for void*... yes > > > checking size of void*... 8 > > > checking for int large enough for pointers... no > > > checking for long long... yes > > > checking size of long long... 8 > > > checking for extension for Fortran 90 programs... f90 > > > checking for Fortran 90 KIND parameter for 8-byte > > integers... 8 > > > checking for weak symbol support... pragma weak > > > checking that weak symbols are visible to other files... yes > > > checking whether struct flock compatible with MPI_Offset... yes > > > checking configured file systems... testfs ufs nfs > > > configure: WARNING: File locks may not work with NFS. > > See the > > > Installation and > > > users manual for instructions on testing and if necessary > > fixing this > > > checking for library containing pthread_create... -lpthread > > > checking for library containing aio_write... -lrt > > > checking signal.h usability... yes > > > checking signal.h presence... yes > > > checking for signal.h... yes > > > checking if aio.h exists and can be compiled... yes > > > checking if sys/aio.h exists and can be compiled... no > > > checking whether aio routines can be used... yes > > > checking for obsolete two-argument aio_write... no > > > checking for obsolete two-argument aio_suspend... no > > > checking for aio_fildes member of aiocb structure... yes > > > checking for aio_whence member of aiocb structure... no > > > checking for aio_handle member of aiocb structure... no > > > checking for aio_reqprio member of aiocb structure... yes > > > checking for aio_sigevent member of aiocb structure... yes > > > checking sys/vfs.h usability... yes > > > checking sys/vfs.h presence... yes > > > checking for sys/vfs.h... yes > > > checking sys/param.h usability... yes > > > checking sys/param.h presence... yes > > > checking for sys/param.h... yes > > > checking sys/mount.h usability... yes > > > checking sys/mount.h presence... yes > > > checking for sys/mount.h... yes > > > checking sys/statvfs.h usability... yes > > > checking sys/statvfs.h presence... yes > > > checking for sys/statvfs.h... yes > > > checking whether struct statfs properly defined... yes > > > checking for f_fstypename member of statfs structure... no > > > checking for sys/stat.h... (cached) yes > > > checking for sys/types.h... (cached) yes > > > checking for unistd.h... (cached) yes > > > checking for stat... yes > > > checking for st_fstype member of stat structure... no > > > checking for sys/types.h... (cached) yes > > > checking for sys/statvfs.h... (cached) yes > > > checking for sys/vfs.h... (cached) yes > > > checking for statvfs... yes > > > checking for f_basetype member of statvfs structure... no > > > checking for unistd.h... (cached) yes > > > checking for large file defines... yes > > > checking whether off64_t is an scalar type... yes > > > checking for strerror... yes > > > checking for doctext... no > > > checking for strdup... no > > > checking for lstat... no > > > checking for readlink... no > > > checking for fsync... no > > > checking for ftruncate... no > > > setting CC to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc > > > setting F77 to /opt/apps/sun/sunstudio12/bin/f77 > > > setting TEST_CC to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc > > > setting TEST_F77 to /tmp/mvapich1/mvapich-1.0-2008-02-28/bin/ > mpif77 > > > setting CFLAGS to -D_X86_64_ -D_AMD_QUAD_CORE_ - > > DEARLY_SEND_COMPLETION > > > -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ - > > DCH_GEN2 > > > -D_GNU_SOURCE -I/opt/ofed/include -O3 - > > DHAVE_MPICHCONF_H -D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H -DFORTRANUNDERSCORE > > > -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -DHAVE_ROMIOCONF_H > > > setting USER_CFLAGS to > > > setting USER_FFLAGS to > > > configure: creating ./config.status > > > config.status: creating Makefile > > > config.status: creating localdefs > > > config.status: creating mpi-io/Makefile > > > config.status: creating mpi2-other/info/Makefile > > > config.status: creating mpi2-other/array/Makefile > > > config.status: creating adio/common/Makefile > > > config.status: creating test/Makefile > > > config.status: creating test/misc.c > > > config.status: creating test/large_file.c > > > config.status: creating test/runtests > > > config.status: creating util/romioinstall > > > config.status: creating include/mpio.h > > > config.status: creating include/mpiof.h > > > config.status: creating adio/ad_nfs/Makefile > > > config.status: creating adio/ad_ufs/Makefile > > > config.status: creating adio/ad_panfs/Makefile > > > config.status: creating adio/ad_xfs/Makefile > > > config.status: creating adio/ad_sfs/Makefile > > > config.status: creating adio/ad_pfs/Makefile > > > config.status: creating adio/ad_testfs/Makefile > > > config.status: creating adio/ad_pvfs/Makefile > > > config.status: creating adio/ad_pvfs2/Makefile > > > config.status: creating adio/ad_gridftp/Makefile > > > config.status: creating adio/ad_lustre/Makefile > > > config.status: creating mpi-io/fortran/Makefile > > > config.status: creating mpi2-other/info/fortran/Makefile > > > config.status: creating mpi2-other/array/fortran/Makefile > > > config.status: creating test/fmisc.f > > > config.status: creating test/fcoll_test.f > > > config.status: creating test/pfcoll_test.f > > > config.status: creating test/fperf.f > > > config.status: creating mpi-io/glue/mpich2/Makefile > > > config.status: creating mpi-io/glue/mpich1/Makefile > > > config.status: creating mpi-io/glue/default/Makefile > > > config.status: creating adio/include/romioconf.h > > > config.status: executing default-1 commands > > > Finished configuring ROMIO > > > checking whether head command is broken... no > > > creating config.status > > > creating Makefile > > > creating util/Makefile > > > creating examples/Makefile > > > creating src/pt2pt/Makefile > > > creating src/coll/Makefile > > > creating src/context/Makefile > > > creating src/dmpi/Makefile > > > creating src/env/Makefile > > > creating src/topol/Makefile > > > creating src/profile/Makefile > > > creating src/coll2/Makefile > > > creating src/util/Makefile > > > creating src/infoexport/Makefile > > > creating examples/basic/Makefile > > > creating src/misc2/Makefile > > > creating src/external/Makefile > > > creating mpichconf.h > > > creating util/mpicc > > > creating util/mpicxx > > > creating util/mpiinstall > > > creating util/mpirun > > > creating util/mpichlib.conf > > > creating util/mpiman > > > creating util/chkserv > > > creating util/chkmachine > > > creating util/tstmachines > > > creating include/mpidefs.h > > > creating include/mpif.f90 > > > Configuring the test suite... > > > Configuring with args -basecc=/opt/apps/sun/sunstudio12/bin/cc > > > -cc=/tmp/mvapich1/mvapich-1.0-2008-02-28/bin/mpicc -make=make > > > --no-print-directory -device=ch_gen2 > > > -mpichpath=/tmp/mvapich1/mvapich-1.0-2008-02-28/bin --enable- > > boot > > > -cflags=-D_X86_64_ -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION > > > -DMEMORY_SCALE -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ - > > DCH_GEN2 > > > -D_GNU_SOURCE -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H - > > nof77 --enable-io > > > Don't build the tests that need Fortran > > > checking whether cross-compiling... no > > > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc > > runs... yes > > > checking for gnumake... no > > > checking whether make supports include... yes > > > checking for OSF V3 make... no > > > checking for virtual path format... VPATH > > > checking for stdlib.h... yes > > > checking for unistd.h... yes > > > checking for string.h... yes > > > checking for stdarg is correct... yes > > > checking for long double... yes > > > checking for long long... yes > > > checking whether compiler can handle loops with long long... yes > > > checking whether the compiler /opt/apps/sun/sunstudio12/bin/cc > > accepts > > > ANSI prototypes... yes > > > checking for signal.h... yes > > > checking for sigaction... yes > > > checking for sleep... yes > > > checking for sysconf... yes > > > creating config.status > > > creating Makefile > > > creating pt2pt/Makefile > > > creating coll/Makefile > > > creating topol/Makefile > > > creating context/Makefile > > > creating env/Makefile > > > creating profile/Makefile > > > creating io/Mfile > > > creating pt2pt/runtests > > > creating coll/runtests > > > creating topol/runtests > > > creating context/runtests > > > creating command/runtests > > > creating env/runtests > > > creating profile/runtests > > > creating io/rtest > > > creating io/misc.c > > > creating io/fmisc.f > > > creating io/fcoll_test.f > > > creating io/pfcoll_test.f > > > creating io/large_file.c > > > creating io/fperf.f > > > Configuring the performance tests ... > > > Configure in /tmp/mvapich1/mvapich-1.0-2008-02- > > 28/examples/perftest > > > creating cache ./config.cache > > > checking for a BSD compatible install... /usr/bin/install -c > > > checking whether build environment is sane... yes > > > checking whether make --no-print-directory sets ${MAKE}... yes > > > checking for working aclocal... found > > > checking for working autoconf... found > > > checking for working automake... found > > > checking for working autoheader... found > > > checking for working makeinfo... found > > > checking whether clock skew breaks make... no > > > checking whether make supports include... yes > > > checking whether make allows comments in actions... yes > > > checking for virtual path format... VPATH > > > checking whether make sets CFLAGS... yes > > > checking for cit... no > > > checking for gnuplot... gnuplot > > > checking how to run the C preprocessor... > > > /opt/apps/sun/sunstudio12/bin/cc -E > > > checking whether C compiler accepts option -O4 -Ofast... no > > > checking whether C compiler accepts option -Ofast... no > > > checking whether C compiler accepts option -fast... yes > > > checking whether routines compiled with -fast can be linked > > with ones > > > compiled without -fast... yes > > > checking for library containing fabs... -lm > > > checking for ANSI C header files... yes > > > checking for sys/time.h... yes > > > checking for stdlib.h... yes > > > checking for string.h... yes > > > checking whether time.h and sys/time.h may both be included... yes > > > checking whether struct tm is in sys/time.h or time.h... time.h > > > checking for working const... yes > > > checking for restrict... restrict > > > checking for long long... yes > > > updating cache ./config.cache > > > creating ./config.status > > > creating Makefile > > > creating mpptestconf.h > > > creating mpid/ch_gen2/Makefile > > > creating mpid/tests/Makefile > > > creating mpichconf.h > > > mpichconf.h is unchanged > > > *# ---> > > > *# You should register your copy of MPICH with us by sending mail > > > *# to majordomo@mcs.anl.gov containing the message > > > *# subscribe mpi-users > > > *# This will allow us to notify you of new releases of MPICH. > > > *# > > > *# You can also check the MPICH home page at > > > *# http://www.mcs.anl.gov/mpi/mpich > > > *# ---< > > > Configuration completed. > > > Building MVAPICH... > > > > > > cleaning directory mpi-io > > > > > > cleaning directory adio/common > > > > > > cleaning directory mpi-io/glue/mpich1 > > > > > > cleaning directory adio/ad_testfs > > > > > > cleaning directory adio/ad_ufs > > > > > > cleaning directory adio/ad_nfs > > > > > > cleaning directory mpi-io/fortran > > > > > > cleaning directory test > > > rm -f .P* PI* *.o > > > rm -f simple perf async coll_test coll_perf misc file_info > > excl > > > large_array atomicity noncontig i_noncontig noncontig_coll > > split_coll > > > shared_fp large_file psimple error status noncontig_coll2 > > fcoll_test > > > fperf fmisc pfcoll_test > > > > > > cleaning src/pt2pt > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/pt2pt/*.o > > > cleaning src/env > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/env/*.o > > > cleaning src/dmpi > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/dmpi/*.o > > > cleaning src/util > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/util/*.o > > > cleaning src/context > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/context/*.o > > > cleaning src/coll > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/coll/*.o > > > cleaning src/topol > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/topol/*.o > > > cleaning src/profile > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/profile/*.o > > > cleaning src/misc2 > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/misc2/*.o > > > cleaning src/external > > > rm -f *.o *~ /tmp/mvapich1/mvapich-1.0-2008-02-28/src/external/*.o > > > cleaning mpid/ch_gen2 > > > /bin/rm -f *.o *.d *~ viainit.o viasend.o viarecv.o viacheck.o > > viapriv.o > > > viaparam.o viutil.o vbuf.o cm_user.o mpid_init.o mpid_send.o > > mpid_recv.o > > > cm.o mpid_hsend.o mpid_hrecv.o mpid_pack.o cmnargs.o > > mpid_misc.o dreg.o > > > mpid_smpi.o collutils.o intra_rdma_barrier.o mpid_mcast.o > > mcst_grp.o > > > ibmcgrp.o crc32h.o avl.o mem_hooks.o viacoalesce.o > > shmem_coll.o > > > async_progress.o \ > > > queue.o sbcnst2.o tr2.o queue.c > > sbcnst2.c tr2.c sbcnst2.h tr2.h core > > > ibmcgrp/*.o ibmcgrp/ibmcgrp \ > > > /tmp/mvapich1/mvapich-1.0-2008-02- > > 28/bin/ibmcgrp> make --no-print-directory -C process clean > > > rm -f *.o mpirun_rsh mpispawn minidaemon_client > > pmgr_collective_client.o > > > pmgr_collective_common.o minidaemon.o mpirun_util.o > > > cleaning examples > > > rm -f sendchar *.o > > > > > > Making clean in directory test > > > > > > Making clean in directory pt2pt > > > > > > Making clean in directory coll > > > > > > Making clean in directory topol > > > > > > Making clean in directory env > > > > > > Making clean in directory context > > > > > > Making clean in directory profile > > > > > > Making clean in directory io > > > > > > Making clean in directory perftest > > > test -z "mpptest goptest buflimit " || rm -f mpptest goptest > > buflimit> rm -f *.o core *.core > > > rm -f tunepkt pktuse copytest vectest stress ctest cluster tcomm > > > rm -f work.pc work.pcl > > > cleaning examples/basic > > > rm -f work.pc work.pcl > > > rm -f *.o *~ PI* cpi systest srtest fpi cpilog > > cpi_autolog hello++ > > > iotest pi3f90 upshot rdb.* startup.* core > > > rm -f hello++.ti hello++.ii > > > cleaning examples/test/pt2pt > > > cleaning examples/test/coll > > > cleaning examples/test/topol > > > cleaning examples/test/context > > > cleaning examples/test/env > > > cleaning examples/test/profile > > > cleaning examples/test > > > > > > Making clean in directory pt2pt > > > > > > Making clean in directory coll > > > > > > Making clean in directory topol > > > > > > Making clean in directory env > > > > > > Making clean in directory context > > > > > > Making clean in directory profile > > > > > > Making clean in directory io > > > cleaning examples/perftest > > > test -z "mpptest goptest buflimit " || rm -f mpptest goptest > > buflimit> rm -f *.o core *.core > > > rm -f tunepkt pktuse copytest vectest stress ctest cluster tcomm > > > rm -f work.pc work.pcl > > > cleaning src/infoexport > > > rm -f *.o > > > rm -f *.i > > > rm -f *.s > > > rm -f *.cxx.log > > > rm -f *.cxx.errors > > > rm -f libtvmpich.so.1.0 > > > rm -f libtvmpich.so* > > > rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/src/infoexport/*.o > > > rm -f *~ *.o aditest1 aditest2 aditest3 aditest4 aditest5 > > aditest6 > > > aditest7 aditest8 aditest9 aditest10 aditest11 aditest12 > > aditest13 > > > timers trunc > > > rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/lib*.a > > > rm -f /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/shared/lib*.so* > > > make --no-print-directory mpi-modules > > > make --no-print-directory mpilib > > > for file in queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h ; do \ > > > if [ ! -s $file ] ; then \ > > > ln -s > > ../util/$file; \ > > > > > fi; \ > > > done > > > > > > making mpir in directory mpid/ch_gen2 > > > for file in queue.c sbcnst2.c tr2.c sbcnst2.h tr2.h ; do \ > > > if [ ! -s $file ] ; then \ > > > ln -s > > ../util/$file; \ > > > > > fi; \ > > > done > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viainit.c > > > "ib_init.h", line 12: warning: initializer does not fit or is > > out of > > > range: -1 > > > "ib_init.h", line 34: warning: improper pointer/integer > > combination: op "<" > > > "viainit.c", line 149: warning: argument #3 is incompatible > > with prototype: > > > prototype: pointer to const > > struct {array[16] of unsigned long > > > __bits} : "/usr/include/sched.h", line 76 > > > argument : pointer to unsigned long > > > "viainit.c", line 162: warning: argument #3 is incompatible > > with prototype: > > > prototype: pointer to const > > struct {array[16] of unsigned long > > > __bits} : "/usr/include/sched.h", line 76 > > > argument : pointer to unsigned long > > > "viainit.c", line 556: warning: argument #3 is incompatible > > with prototype: > > > prototype: pointer to function(pointer > > to void) returning pointer to > > > void : "/usr/include/pthread.h", line 163 > > > argument : pointer to void > > > "viainit.c", line 792: warning: implicit function declaration: > > > init_apm_lock > > > "viainit.c", line 990: warning: argument mismatch > > > "viainit.c", line 1158: warning: implicit function > > declaration: > > > reload_alternate_path > > > "viainit.c", line 1221: warning: argument #3 is incompatible > > with > > > prototype: > > > prototype: pointer to function(pointer > > to void) returning pointer to > > > void : "/usr/include/pthread.h", line 163 > > > argument : pointer to void > > > "viainit.c", line 1345: warning: argument mismatch > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viasend.c > > > "viasend.c", line 145: warning: implicit function declaration: > > > viadev_post_send_async_qp > > > "viasend.c", line 401: warning: argument mismatch > > > "viasend.c", line 703: warning: argument mismatch > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viarecv.c > > > "viarecv.c", line 263: warning: argument mismatch > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viacheck.c > > > "viacheck.c", line 571: warning: argument mismatch > > > "viacheck.c", line 1824: warning: argument mismatch > > > "viacheck.c", line 1846: warning: argument mismatch > > > "viacheck.c", line 1861: warning: argument mismatch > > > "viacheck.c", line 2260: warning: implicit function > > declaration: > > > perform_manual_apm > > > "viacheck.c", line 2856: warning: implicit function > > declaration: > > > reload_alternate_path > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viapriv.c > > > "viapriv.c", line 649: warning: implicit function declaration: > > > perform_manual_apm > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viaparam.c > > > "/opt/ofed/include/infiniband/common.h", line 60: warning: > > implicit > > > function declaration: bswap_64 > > > "/opt/ofed/include/infiniband/umad.h", line 72: warning: zero > > or > > > negative subscript > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viutil.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c vbuf.c > > > "vbuf.c", line 125: warning: argument mismatch > > > "vbuf.c", line 283: warning: argument mismatch > > > "vbuf.c", line 442: warning: argument mismatch > > > "vbuf.c", line 488: warning: argument mismatch > > > "vbuf.c", line 491: warning: argument mismatch > > > "vbuf.c", line 492: warning: argument mismatch > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c cm_user.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_init.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_send.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_recv.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c cm.c > > > "ib_init.h", line 12: warning: initializer does not fit or is > > out of > > > range: -1 > > > "ib_init.h", line 34: warning: improper pointer/integer > > combination: op "<" > > > "cm.c", line 176: warning: pointer to void or function used in > > arithmetic> "cm.c", line 265: warning: implicit function > > declaration: > > > reload_alternate_path > > > "cm.c", line 677: warning: statement not reached > > > "cm.c", line 735: warning: pointer to void or function used in > > arithmetic> "cm.c", line 735: warning: pointer to void or > > function used in arithmetic > > > "cm.c", line 744: warning: pointer to void or function used in > > arithmetic> "cm.c", line 757: warning: statement not reached > > > "cm.c", line 830: warning: pointer to void or function used in > > arithmetic> "cm.c", line 830: warning: pointer to void or > > function used in arithmetic > > > "cm.c", line 925: warning: pointer to void or function used in > > arithmetic> "cm.c", line 1006: warning: pointer to void or > > function used in arithmetic > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_hsend.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_hrecv.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_pack.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c cmnargs.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_misc.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c dreg.c > > > "dreg.c", line 132: warning: initializer does not fit or is > > out of > > > range: -1 > > > "dreg.c", line 780: warning: argument mismatch > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_smpi.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c collutils.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c > > > intra_rdma_barrier.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mpid_mcast.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mcst_grp.c > > > "mcst_grp.c", line 338: warning: empty translation unit > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c ibmcgrp.c > > > "ibmcgrp.c", line 480: warning: empty translation unit > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c crc32h.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c avl.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c mem_hooks.c > > > "mem_hooks.c", line 34: warning: assignment type mismatch: > > > pointer to function(pointer to void, > > unsigned long) returning int > > > "=" pointer to void > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c viacoalesce.c > > > "viacoalesce.c", line 70: warning: argument mismatch > > > "viacoalesce.c", line 97: warning: argument mismatch > > > "viacoalesce.c", line 167: warning: argument mismatch > > > "viacoalesce.c", line 231: warning: argument mismatch > > > "viacoalesce.c", line 235: warning: argument mismatch > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c shmem_coll.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c > > > async_progress.c > > > "ib_init.h", line 12: warning: initializer does not fit or is > > out of > > > range: -1 > > > "ib_init.h", line 34: warning: improper pointer/integer > > combination: op "<" > > > "async_progress.c", line 102: warning: function expects to > > return value: > > > atomic_send_or_queue > > > "async_progress.c", line 112: warning: implicit function > > declaration: > > > perform_manual_apm > > > "async_progress.c", line 686: warning: argument #3 is > > incompatible with > > > prototype: > > > prototype: pointer to function(pointer > > to void) returning pointer to > > > void : "/usr/include/pthread.h", line 163 > > > argument : pointer to void > > > "async_progress.c", line 841: warning: implicit function > > declaration: > > > reload_alternate_path > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c queue.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c sbcnst2.c > > > /opt/apps/sun/sunstudio12/bin/cc -DHAVE_CONFIG_H -I. > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/include > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/util - > > DMPID_DEVICE_CODE > > > -DHAVE_UNAME=1 -DHAVE_NETDB_H=1 -DHAVE_GETHOSTBYNAME=1 > > > -DMPID_DEBUG_NONE -DMPID_STAT_NONE -D_GNU_SOURCE - > > D_X86_64_ > > > -D_AMD_QUAD_CORE_ -DEARLY_SEND_COMPLETION -DMEMORY_SCALE > > > -DVIADEV_RPUT_SUPPORT -D_SMP_ -D_SMP_RNDV_ -DCH_GEN2 - > > D_GNU_SOURCE > > > -I/opt/ofed/include -O3 -DHAVE_MPICHCONF_H > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28 > > > -I/tmp/mvapich1/mvapich-1.0-2008-02-28/mpid/ch_gen2 - > > I. -c tr2.c > > > ar crl /tmp/mvapich1/mvapich-1.0-2008-02-28/lib/libmpich.a > > viainit.o > > > viasend.o viarecv.o viacheck.o viapriv.o viaparam.o viutil.o > > vbuf.o > > > cm_user.o mpid_init.o mpid_send.o mpid_recv.o cm.o > > mpid_hsend.o > > > mpid_hrecv.o mpid_pack.o cmnargs.o mpid_misc.o dreg.o > > mpid_smpi.o > > > collutils.o intra_rdma_barrier.o mpid_mcast.o mcst_grp.o > > ibmcgrp.o > > > crc32h.o avl.o mem_hooks.o viacoalesce.o shmem_coll.o > > async_progress.o > > > queue.o sbcnst2.o tr2.o > > > cd process && make --no-print-directory &&\ > > > install -m 755 mpirun_rsh mpispawn > > minidaemon_client > > > /tmp/mvapich1/mvapich-1.0-2008-02-28/bin > > > /opt/apps/sun/sunstudio12/bin/cc > > > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02- > > 28/lib/shared\" > > > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > > > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02- > > 28/etc/mvapich.conf\" -DVAPI > > > -D_SMP_ -DUSE_SSH -c -o > > pmgr_collective_common.o > > > pmgr_collective_common.c > > > /opt/apps/sun/sunstudio12/bin/cc > > > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02- > > 28/lib/shared\" > > > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > > > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02- > > 28/etc/mvapich.conf\" -DVAPI > > > -D_SMP_ -DUSE_SSH -c -o > > pmgr_collective_mpirun.o > > > pmgr_collective_mpirun.c > > > "pmgr_collective_mpirun.c", line 75: warning: pointer to void > > or > > > function used in arithmetic > > > "pmgr_collective_mpirun.c", line 97: warning: pointer to void > > or > > > function used in arithmetic > > > "pmgr_collective_mpirun.c", line 98: warning: pointer to void > > or > > > function used in arithmetic > > > "pmgr_collective_mpirun.c", line 250: warning: pointer to void > > or > > > function used in arithmetic > > > "pmgr_collective_mpirun.c", line 263: warning: pointer to void > > or > > > function used in arithmetic > > > "pmgr_collective_mpirun.c", line 269: warning: pointer to void > > or > > > function used in arithmetic > > > /opt/apps/sun/sunstudio12/bin/cc > > > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02- > > 28/lib/shared\" > > > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > > > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02- > > 28/etc/mvapich.conf\" -DVAPI > > > -D_SMP_ -DUSE_SSH -c -o minidaemon.o minidaemon.c > > > "minidaemon.c", line 267: warning: argument mismatch > > > "minidaemon.c", line 300: warning: argument mismatch > > > "minidaemon.c", line 407: warning: argument mismatch > > > "minidaemon.c", line 430: warning: argument mismatch > > > "minidaemon.c", line 455: warning: argument mismatch > > > "minidaemon.c", line 542: warning: argument mismatch > > > "minidaemon.c", line 552: warning: argument mismatch > > > "minidaemon.c", line 560: warning: argument mismatch > > > "minidaemon.c", line 591: warning: pointer to void or function > > used in > > > arithmetic > > > "minidaemon.c", line 679: warning: argument mismatch > > > "minidaemon.c", line 781: warning: enum type mismatch: op "=" > > > "minidaemon.c", line 812: warning: argument mismatch > > > "minidaemon.c", line 842: warning: enum type mismatch: op "=" > > > "minidaemon.c", line 854: warning: enum type mismatch: op "=" > > > "minidaemon.c", line 1072: warning: argument mismatch > > > "minidaemon.c", line 1114: warning: argument mismatch > > > "minidaemon.c", line 1130: warning: argument mismatch > > > "minidaemon.c", line 1213: warning: argument mismatch > > > "minidaemon.c", line 1214: warning: enum type mismatch: op "=" > > > "minidaemon.c", line 1222: warning: argument mismatch > > > "minidaemon.c", line 1244: warning: argument mismatch > > > "minidaemon.c", line 1247: warning: argument mismatch > > > "minidaemon.c", line 1278: warning: argument mismatch > > > "minidaemon.c", line 1281: warning: argument mismatch > > > "minidaemon.c", line 1288: warning: argument mismatch > > > "minidaemon.c", line 1309: warning: argument mismatch > > > "minidaemon.c", line 1369: warning: argument mismatch > > > "minidaemon.c", line 1417: warning: argument mismatch > > > "minidaemon.c", line 1443: warning: argument mismatch > > > "minidaemon.c", line 1492: warning: argument mismatch > > > "minidaemon.c", line 1533: warning: argument mismatch > > > /opt/apps/sun/sunstudio12/bin/cc > > > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02- > > 28/lib/shared\" > > > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > > > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02- > > 28/etc/mvapich.conf\" -DVAPI > > > -D_SMP_ -DUSE_SSH -c -o mpirun_util.o > > mpirun_util.c> "mpirun_util.c", line 25: warning: implicit > > function declaration: vsnprintf > > > "mpirun_util.c", line 33: warning: implicit function > > declaration: perror > > > "mpirun_util.c", line 77: warning: implicit function > > declaration: strlen > > > "mpirun_util.c", line 84: warning: implicit function > > declaration: strcat > > > /opt/apps/sun/sunstudio12/bin/cc > > > -DLD_LIBRARY_PATH_MPI=\"/tmp/mvapich-1.0-2008-02- > > 28/lib/shared\" > > > -DMPI_PREFIX=\"/tmp/mvapich-1.0-2008-02-28/\" > > > -DPARAM_GLOBAL=\"/tmp/mvapich-1.0-2008-02- > > 28/etc/mvapich.conf\" -DVAPI > > > -D_SMP_ -DUSE_SSH mpirun_rsh.c > > pmgr_collective_common.o > > > pmgr_collective_mpirun.o minidaemon.o mpirun_util.o -lm - > > o mpirun_rsh > > > "mpirun_rsh.c", line 579: warning: implicit function > > declaration: > > > pmgr_processops > > > mpirun_rsh.o(.text+0xf1b): In function `main': > > > : undefined reference to `env2int' > > > mpirun_rsh.o(.text+0x70cc): In function `mpispawn_checkin': > > > : undefined reference to `env2int' > > > make[4]: *** [mpirun_rsh] Error 1 > > > make[3]: *** [mpirun_rsh] Error 2 > > > Exit status from make was 2 > > > make[2]: *** [mpilib] Error 1 > > > make[1]: *** [mpi-modules] Error 2 > > > make: *** [mpi] Error 2 > > > Failure in building MVAPICH. > > > Please file an error report to mvapich-discuss@cse.ohio- > > state.edu with > > > all your log files. > > > > > > _______________________________________________ > > > mvapich-discuss mailing list > > > mvapich-discuss@cse.ohio-state.edu > > > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss> > _______________________________________________ > > mvapich-discuss mailing list > > mvapich-discuss@cse.ohio-state.edu > > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From forum.san at gmail.com Fri May 9 03:14:41 2008 From: forum.san at gmail.com (Sangamesh B) Date: Fri May 9 03:14:58 2008 Subject: [mvapich-discuss] caused collective abort of all ranks + signal 9 In-Reply-To: References: Message-ID: Hi, My answers are inline: On Tue, May 6, 2008 at 9:15 PM, Matthew Koop wrote: > Sangamesh, > > Can you run any of the included benchmarks with the OFED package? Try > running the ibv_rc_pingpoing test between nodes in your system to first > make sure the fabric is healthy. > Tested some of OFED pingpong tests on two nodes ibc12 (as server) and ibc11 (as slave). The output is: Test 1: [root@compute-0-12 bin]# ./ibv_rc_pingpong local address: LID 0x0006, QPN 0x0c0405, PSN 0xf32003 remote address: LID 0x0005, QPN 0x3b0405, PSN 0xcde263 [root@compute-0-11 bin]# ./ibv_rc_pingpong ibc12 local address: LID 0x0005, QPN 0x3b0405, PSN 0xcde263 remote address: LID 0x0006, QPN 0x0c0405, PSN 0xf32003 Failed status 9 for wr_id 2 Test 2: [root@compute-0-12 bin]# ibv_ud_pingpong local address: LID 0x0006, QPN 0x0d0405, PSN 0x9affd4 remote address: LID 0x0005, QPN 0x3c0405, PSN 0x6223f3 [root@compute-0-11 bin]# ibv_ud_pingpong ibc12 local address: LID 0x0005, QPN 0x3c0405, PSN 0x6223f3 remote address: LID 0x0006, QPN 0x0d0405, PSN 0x9affd4 Test 3: [root@compute-0-12 bin]# ibv_uc_pingpong local address: LID 0x0006, QPN 0x200405, PSN 0x6f491e remote address: LID 0x0005, QPN 0x3e0405, PSN 0xc7e4ff [root@compute-0-11 bin]# ibv_uc_pingpong ibc12 local address: LID 0x0005, QPN 0x3e0405, PSN 0xc7e4ff remote address: LID 0x0006, QPN 0x200405, PSN 0x6f491e I think Test 2 and Test 3 are successful. In each test, I pressed Ctrl C to comeout of test. Are these successful? > > Also, can you give us some additional information on your setup? What type > of cards are these? I don't have knowledge about cards. This is my first mvapich install. I guess these might be SDR cards. > Also, how did you set the 'ulimit -l unlimited'. I set it on comandline. > We > suggest placing it in /etc/init.d/sshd on all nodes and restarting sshd > (and mpd). This will ensure that the processes started will inherit the > modified ulimit settings. > Did. But again the same error. The program runs, but in the end gives the error: [root@compute-0-12 mvapich2_ofed_intel]# /opt/mvapich2_ofed_intel/bin/mpiexec -np 2 -env MV2_USE_COALESCE 0 -env MV2_VBUF_TOTAL_SIZE 9216 ./samplmvaofedintel Process 0 of 2 executed on compute-0-12.local Process 1 of 2 executed on compute-0-11.local rank 1 in job 6 compute-0-12.local_36014 caused collective abort of all ranks exit status of rank 1: killed by signal 9 [root@compute-0-12 mvapich2_ofed_intel]# Tried to build mvapich2 with different 'make' files. i.e. make.mvapich2.vapi and make.mvapich2.udapl. Both of these gave library missing error: ' 'CC=/opt/intel/cce/10.1.015/bin/icc' 'CFLAGS=-D_X86_64_ -DONE_SIDED -DUSE_INLINE -DRDMA _FAST_PATH -DUSE_HEADER_CACHING -DLAZY_MEM_UNREGISTER -D_SMP_ -D_PCI_EX_ -D_SDR_ -DMPIDI_ CH3_CHANNEL_RNDV -DMPID_USE_SEQUENCE_NUMBERS -D_SMALL_CLUSTER -I/opt/OFED/include -O2 ' 'CXX=/opt/intel/cce/10.1.015/bin/icpc' 'F77=/opt/intel/fce/10.1.015/bin/ifort' 'F90=/opt/ intel/fce/10.1.015/bin/ifort' 'FFLAGS=-L/opt/OFED/lib64' Running on system: Linux compute-0-12.local 2.6.9-34.0.2.ELsmp #1 SMP Fri Jul 7 18:22:55 CDT 2006 x86_64 x86_64 x86_64 GNU/Linux Executing mpich2prereq in /root/mvapich2-1.0.2/src/mpid/osu_ch3 with mrail Executing mpich2prereq in /root/mvapich2-1.0.2/src/mpid/osu_ch3/channels/mrail sourcing /root/mvapich2-1.0.2/src/pm/mpd/mpich2prereq sourcing /root/mvapich2-1.0.2/src/pm/mpd/setup_pm checking for gcc... /opt/intel/cce/10.1.015/bin/icc checking for C compiler default output file name... configure: error: C compiler cannot c reate executables See `config.log' for more details. Failure in configuration. $ ./configure --prefix=/opt/mvapich2_vapi_intel --with-device=osu_ch3:mrail --with-rdma= vapi --with-pm=mpd --disable-romio --without-mpe configure:3202: $? = 0 configure:3225: checking for C compiler default output file name configure:3228: /opt/intel/cce/10.1.015/bin/icc -D_X86_64_ -DONE_SIDED -DUSE_INLINE -DRDM A_FAST_PATH -DUSE_HEADER_CACHING -DLAZY_MEM_UNREGISTER -D_SMP_ -D_PCI_EX_ -D_SDR_ -DMPIDI _CH3_CHANNEL_RNDV -DMPID_USE_SEQUENCE_NUMBERS -D_SMALL_CLUSTER -I/opt/OFED/include -O2 conftest.c -L/opt/OFED/lib64 -lmtl_common -lvapi -lpthread -lmosal -lmpga >&5 ld: cannot find -lmtl_common configure:3231: $? = 1 configure: failed program was: | /* confdefs.h. */ The 'configure' is looking these libraries inside OFED. I located these libraries. They are not available. Is it problem with OFED? (It is installed with buliding rpms.) OR, vapi and udapl version requires other libraries than OFED? And:- [root@compute-0-12 ~]# ofed_info OFED-1.2.5 ofa_kernel-1.2.5: ... And:- [root@compute-0-12 ~]# ibhosts Ca : 0x0002c90200272778 ports 1 "MT25204 InfiniHostLx Mellanox Technologies" Ca : 0x0002c9020027277c ports 1 "MT25204 InfiniHostLx Mellanox Technologies" Ca : 0x0002c902002728a4 ports 1 "MT25204 InfiniHostLx Mellanox Technologies" Ca : 0x0002c9020027276c ports 1 "MT25204 InfiniHostLx Mellanox Technologies" Ca : 0x0002c90200272798 ports 1 "MT25204 InfiniHostLx Mellanox Technologies" [root@compute-0-12 With the link: http://www.opensubscriber.com/message/general@lists.openfabrics.org/8956919.html I came to know its a bug. Shall I reinstall OFED (without using rpms), for building mvapich2 with vapi or udapl What's the solution? Thanks in advance, Sangamesh > Thanks, > > Matt > > On Tue, 6 May 2008, Sangamesh B wrote: > > > Hi all, > > > > > > *I got some problem, can someone help me on this issue.* > > > > *The scenario is : We have a Rocks(4.2) cluster with 12 nodes. We > installed > > Infiniband cards newly in 5 nodes(Masternode doesn't have IB card). > > Installation of OFED is successful and IP got assigned.* > > > > *I installed Mvapich2 in that and created password free environment from > > computenode-0-8 to 12(the nodes which have IB card). So far everything > is > > fine, And the MPD is booting up also. * > > > > *I've compiled a sample MPI program and tried to execute it and I got the > > following kind of results:* > > > > Scenario 1: Using root to execute Hellow.o (compiled with mvapich2-mpicc) > > > > [root@compute-0-8 test]# /opt/mvapich2_ps/bin/mpiexec -np 2 > /test/Hellow.o > > Hello world from process 0 of 2 > > Hello world from process 1 of 2 > > rank 1 in job 8 compute-0-8.local_34399 caused collective abort of all > > ranks > > exit status of rank 1: killed by signal 9 > > rank 0 in job 8 compute-0-8.local_34399 caused collective abort of all > > ranks > > exit status of rank 0: killed by signal 9 > > > > Scenario 2: Using user id (srinu) to execute the same file. > > > > [srinu@compute-0-8 test]$ /opt/mvapich2_ps/bin/mpiexec -np 2 > /test/Hellow.o > > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. > > This will severely limit memory registrations. > > libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes. > > This will severely limit memory registrations. > > Fatal error in MPI_Init: > > Other MPI error, error stack: > > MPIR_Init_thread(259)....: Initialization failed > > MPID_Init(102)...........: channel initialization failed > > MPIDI_CH3_Init(178)......: > > MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type > > rdma_iba_hca_init(645)...: cannot create cq > > Fatal error in MPI_Init: > > Other MPI error, error stack: > > MPIR_Init_thread(259)....: Initialization failed > > MPID_Init(102)...........: channel initialization failed > > MPIDI_CH3_Init(178)......: > > MPIDI_CH3I_RMDA_init(208): Failed to Initialize HCA type > > rdma_iba_hca_init(645)...: cannot create cq > > rank 1 in job 9 compute-0-8.local_34399 caused collective abort of all > > ranks > > exit status of rank 1: return code 1 > > > > For 2nd scenario, I found solution from net such as ulimit ?l unlimited. > > But further, this also produced same error as of 1st scenario. > > Can someone solve this error? > > > > Thanks in advance, > > > > Sangamesh > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080509/f94e7c45/attachment.html From shrini at vt.edu Fri May 9 17:41:31 2008 From: shrini at vt.edu (Sai Shrinivas) Date: Fri May 9 22:59:34 2008 Subject: [mvapich-discuss] (no subject) Message-ID: <200805092141.m49Lfvps024558@authsmtp2.cc.vt.edu> Hi, I am shrini I have the following errors: I do not know where and how to fix this. Thanks bash: line 1: 11016 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=93 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux bash: line 1: 9179 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=144 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux bash: line 1: 11362 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=89 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux bash: line 1: 10753 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=102 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux bash: line 1: 11139 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=87 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux bash: line 1: 10415 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=138 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux bash: line 1: 10779 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=100 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux bash: line 1: 7490 Segmentation fault /usr/bin/env LD_LIBRARY_PATH=/apps/local/packages/mvapich-0.9.9-32/lib/shared::/local/lib :/local/lib64:/apps/lib:/apps/lib64 MPIRUN_MPD=0 MPIRUN_HOST=n889.tcf-int.vt.edu MPIRUN_PORT=34066 MPIRUN_RANK=159 MPIRUN_NPROCS=160 MPIRUN_ID=18498 NOT_USE_TOTALVIEW=1 GFORTRAN_UNBUFFERED_ALL=y ./gen.ex.f90.linux Sai Shrinivas Sreedharan HPCFD Lab, 114-K Randolph Hall Mechanical Engineering Department Virginia Tech Blacksburg, VA 24061 Phone: 540-231-2349 "Understanding the motion of fluids is the basis for a tremendous amount of engineering and technology in contemporary life. Planes fly and ships sail because scientists understand the rules of how fluids like water and air behave under varying conditions." -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080509/385ec574/attachment-0001.html From rcbord at wm.edu Wed May 14 10:32:53 2008 From: rcbord at wm.edu (rcbord@wm.edu) Date: Wed May 14 10:33:07 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: References: Message-ID: Amith, I was able to test mvapich1 but I get the same bad results MPI_Cart_Create is returning two communication values. When I add the environment variable VIADEV_USE_SHMEM_COLL=0 and the MPI_sendrecv function changes the dest and source values that are set by the mpi_cart_shift function. Note these errors only occur with F90. I have C/C++ codes that work correctly. I can re-compile any or all mvapich-0.9.9, mvapich1 and mvapich2 easily enough using the make.mvapich.gen2 scripts. What flags should I be using for the pgi-7.0 compiler? I can update the compiler too if necessary. Chris Bording Application Analyst High Performance Computing Group Information Technology The College of William and Mary (757)-221-3488 rcbord@wm.edu On Wed, 7 May 2008, amith rajith mamidala wrote: > Hi, > > Can you try these two options? > > 1. use MVAPICH-1.0 and see if you are seeing the issue. > > 2.Can you run the code by passing the environment variable: > VIADEV_USE_SHMEM_COLL=0 with mvapich > > > Thanks, > Amith > > > On Wed, 7 May 2008 rcbord@wm.edu wrote: > >> Hi, >> I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with >> ofed 1.2 for infinaband. We have be able to use it for 6 months without >> any problem. I tried to port a users fortran code that runs under >> solaris-sparc without any problems. The code is fairly vanilla fotran-90 >> with mpi I know because I wrote it. When I tried to port it to the Linux >> Cluster it hangs a mpisendrecv function. I included a simple test code >> that shows that the new communicator (comm2d) returned by the >> MPI_Cart_create function has multiple values. Same code run on the >> sun sparc cluster returns a single value. I am guessing it could be how >> I compiled MVAPICH, but have tried a few more flags without any success. >> I tried this with MVAPICH2 also and got the same error, but I compiled it >> in a similar fashion. Could it be a PGI problem? >> >> >> Has anyone else seen this? Thanks for any help! >> >> Program mpicart >> ! With input files >> ! Conversion of efit2d.f90 to 3d JPB 11-2007 >> >> Implicit none >> >> include "mpif.h" >> >> Interface >> Subroutine read_model(Pdim0,Pdim1) >> Integer, Intent(out) ::Pdim0,Pdim1 >> end Subroutine read_model >> end Interface >> >> Double Precision :: t1,t2 >> ! >> ! define MPI variables >> ! >> Integer :: Pid, N_proc, ierr >> Integer :: comm2d = 0 >> Integer :: status >> Integer :: period(2),Pdim_size(2),coords(2) >> Integer :: nbrleft,nbrright,nbrtop,nbrbottom >> Integer :: Pdim0, Pdim1 >> Integer :: i, t, numt >> >> call MPI_INIT(ierr) >> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) >> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) >> >> if (Pid .EQ. 0) then >> call read_model(Pdim0,Pdim1) >> >> end if >> ! >> ! Broad cast input parameters values to all processors >> ! >> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & >> & MPI_COMM_WORLD,ierr) >> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & >> & MPI_COMM_WORLD,ierr) >> >> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >> >> Pdim_size(1) = Pdim0 >> Pdim_size(2) = Pdim1 >> >> period(1) = 0 >> period(2) = 0 >> >> call MPI_Dims_create(N_proc,2,Pdim_size,ierr) >> >> call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) >> do i = 0,N_proc-1 >> if (Pid == i) then >> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d >> end if >> end do >> >> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >> call MPI_Comm_free(comm2d,ierr) >> call MPI_FINALIZE(ierr) >> >> end program mpicart >> >> ###################### OUTPUT ######################### >> >> Model parameters for mpicart test >> >> Processor Topology is ( 3 by 4 ) >> >> pid 0 mpi_comm_2d 140 >> pid 11 mpi_comm_2d 138 >> pid 2 mpi_comm_2d 140 >> pid 4 mpi_comm_2d 140 >> pid 6 mpi_comm_2d 140 >> pid 8 mpi_comm_2d 140 >> pid 3 mpi_comm_2d 138 >> pid 10 mpi_comm_2d 140 >> pid 1 mpi_comm_2d 138 >> pid 5 mpi_comm_2d 138 >> pid 7 mpi_comm_2d 138 >> pid 9 mpi_comm_2d 138 >> >> >> Chris Bording >> Application Analyst >> High Performance Computing Group >> Information Technology >> The College of William and Mary >> (757)-221-3488 >> rcbord@wm.edu >> _______________________________________________ >> mvapich-discuss mailing list >> mvapich-discuss@cse.ohio-state.edu >> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss >> > > > From mamidala at cse.ohio-state.edu Wed May 14 10:40:34 2008 From: mamidala at cse.ohio-state.edu (amith rajith mamidala) Date: Wed May 14 10:40:47 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: Message-ID: Hi Chris, Can you try out mvapich1 with tcp? You can use make.mvapich.tcp to compile. This way we will narrow down the code path causing the problem. Thanks, Amith On Wed, 14 May 2008 rcbord@wm.edu wrote: > Amith, > I was able to test mvapich1 but I get the same bad results > MPI_Cart_Create is returning two communication values. > > When I add the environment variable VIADEV_USE_SHMEM_COLL=0 > and the MPI_sendrecv function changes the dest and source values > that are set by the mpi_cart_shift function. > Note these errors only occur with F90. I have C/C++ codes > that work correctly. > > I can re-compile any or all mvapich-0.9.9, mvapich1 and mvapich2 > easily enough using the make.mvapich.gen2 scripts. What flags > should I be using for the pgi-7.0 compiler? I can update the > compiler too if necessary. > > > Chris Bording > Application Analyst > High Performance Computing Group > Information Technology > The College of William and Mary > (757)-221-3488 > rcbord@wm.edu > > On Wed, 7 May 2008, amith rajith mamidala wrote: > > > Hi, > > > > Can you try these two options? > > > > 1. use MVAPICH-1.0 and see if you are seeing the issue. > > > > 2.Can you run the code by passing the environment variable: > > VIADEV_USE_SHMEM_COLL=0 with mvapich > > > > > > Thanks, > > Amith > > > > > > On Wed, 7 May 2008 rcbord@wm.edu wrote: > > > >> Hi, > >> I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with > >> ofed 1.2 for infinaband. We have be able to use it for 6 months without > >> any problem. I tried to port a users fortran code that runs under > >> solaris-sparc without any problems. The code is fairly vanilla fotran-90 > >> with mpi I know because I wrote it. When I tried to port it to the Linux > >> Cluster it hangs a mpisendrecv function. I included a simple test code > >> that shows that the new communicator (comm2d) returned by the > >> MPI_Cart_create function has multiple values. Same code run on the > >> sun sparc cluster returns a single value. I am guessing it could be how > >> I compiled MVAPICH, but have tried a few more flags without any success. > >> I tried this with MVAPICH2 also and got the same error, but I compiled it > >> in a similar fashion. Could it be a PGI problem? > >> > >> > >> Has anyone else seen this? Thanks for any help! > >> > >> Program mpicart > >> ! With input files > >> ! Conversion of efit2d.f90 to 3d JPB 11-2007 > >> > >> Implicit none > >> > >> include "mpif.h" > >> > >> Interface > >> Subroutine read_model(Pdim0,Pdim1) > >> Integer, Intent(out) ::Pdim0,Pdim1 > >> end Subroutine read_model > >> end Interface > >> > >> Double Precision :: t1,t2 > >> ! > >> ! define MPI variables > >> ! > >> Integer :: Pid, N_proc, ierr > >> Integer :: comm2d = 0 > >> Integer :: status > >> Integer :: period(2),Pdim_size(2),coords(2) > >> Integer :: nbrleft,nbrright,nbrtop,nbrbottom > >> Integer :: Pdim0, Pdim1 > >> Integer :: i, t, numt > >> > >> call MPI_INIT(ierr) > >> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) > >> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) > >> > >> if (Pid .EQ. 0) then > >> call read_model(Pdim0,Pdim1) > >> > >> end if > >> ! > >> ! Broad cast input parameters values to all processors > >> ! > >> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & > >> & MPI_COMM_WORLD,ierr) > >> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & > >> & MPI_COMM_WORLD,ierr) > >> > >> call MPI_BARRIER(MPI_COMM_WORLD,ierr) > >> > >> Pdim_size(1) = Pdim0 > >> Pdim_size(2) = Pdim1 > >> > >> period(1) = 0 > >> period(2) = 0 > >> > >> call MPI_Dims_create(N_proc,2,Pdim_size,ierr) > >> > >> call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) > >> do i = 0,N_proc-1 > >> if (Pid == i) then > >> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d > >> end if > >> end do > >> > >> call MPI_BARRIER(MPI_COMM_WORLD,ierr) > >> call MPI_Comm_free(comm2d,ierr) > >> call MPI_FINALIZE(ierr) > >> > >> end program mpicart > >> > >> ###################### OUTPUT ######################### > >> > >> Model parameters for mpicart test > >> > >> Processor Topology is ( 3 by 4 ) > >> > >> pid 0 mpi_comm_2d 140 > >> pid 11 mpi_comm_2d 138 > >> pid 2 mpi_comm_2d 140 > >> pid 4 mpi_comm_2d 140 > >> pid 6 mpi_comm_2d 140 > >> pid 8 mpi_comm_2d 140 > >> pid 3 mpi_comm_2d 138 > >> pid 10 mpi_comm_2d 140 > >> pid 1 mpi_comm_2d 138 > >> pid 5 mpi_comm_2d 138 > >> pid 7 mpi_comm_2d 138 > >> pid 9 mpi_comm_2d 138 > >> > >> > >> Chris Bording > >> Application Analyst > >> High Performance Computing Group > >> Information Technology > >> The College of William and Mary > >> (757)-221-3488 > >> rcbord@wm.edu > >> _______________________________________________ > >> mvapich-discuss mailing list > >> mvapich-discuss@cse.ohio-state.edu > >> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > >> > > > > > > > From rcbord at wm.edu Wed May 14 10:54:40 2008 From: rcbord at wm.edu (rcbord@wm.edu) Date: Wed May 14 10:54:57 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: References: Message-ID: Amith, Should I use the default gnu compilers or do you want me to build with pgi? Chris Bording Application Analyst High Performance Computing Group Information Technology The College of William and Mary (757)-221-3488 rcbord@wm.edu On Wed, 14 May 2008, amith rajith mamidala wrote: > Hi Chris, > > Can you try out mvapich1 with tcp? You can use make.mvapich.tcp to > compile. This way we will narrow down the code path causing the problem. > > Thanks, > Amith > > On Wed, 14 May 2008 rcbord@wm.edu wrote: > >> Amith, >> I was able to test mvapich1 but I get the same bad results >> MPI_Cart_Create is returning two communication values. >> >> When I add the environment variable VIADEV_USE_SHMEM_COLL=0 >> and the MPI_sendrecv function changes the dest and source values >> that are set by the mpi_cart_shift function. >> Note these errors only occur with F90. I have C/C++ codes >> that work correctly. >> >> I can re-compile any or all mvapich-0.9.9, mvapich1 and mvapich2 >> easily enough using the make.mvapich.gen2 scripts. What flags >> should I be using for the pgi-7.0 compiler? I can update the >> compiler too if necessary. >> >> >> Chris Bording >> Application Analyst >> High Performance Computing Group >> Information Technology >> The College of William and Mary >> (757)-221-3488 >> rcbord@wm.edu >> >> On Wed, 7 May 2008, amith rajith mamidala wrote: >> >>> Hi, >>> >>> Can you try these two options? >>> >>> 1. use MVAPICH-1.0 and see if you are seeing the issue. >>> >>> 2.Can you run the code by passing the environment variable: >>> VIADEV_USE_SHMEM_COLL=0 with mvapich >>> >>> >>> Thanks, >>> Amith >>> >>> >>> On Wed, 7 May 2008 rcbord@wm.edu wrote: >>> >>>> Hi, >>>> I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with >>>> ofed 1.2 for infinaband. We have be able to use it for 6 months without >>>> any problem. I tried to port a users fortran code that runs under >>>> solaris-sparc without any problems. The code is fairly vanilla fotran-90 >>>> with mpi I know because I wrote it. When I tried to port it to the Linux >>>> Cluster it hangs a mpisendrecv function. I included a simple test code >>>> that shows that the new communicator (comm2d) returned by the >>>> MPI_Cart_create function has multiple values. Same code run on the >>>> sun sparc cluster returns a single value. I am guessing it could be how >>>> I compiled MVAPICH, but have tried a few more flags without any success. >>>> I tried this with MVAPICH2 also and got the same error, but I compiled it >>>> in a similar fashion. Could it be a PGI problem? >>>> >>>> >>>> Has anyone else seen this? Thanks for any help! >>>> >>>> Program mpicart >>>> ! With input files >>>> ! Conversion of efit2d.f90 to 3d JPB 11-2007 >>>> >>>> Implicit none >>>> >>>> include "mpif.h" >>>> >>>> Interface >>>> Subroutine read_model(Pdim0,Pdim1) >>>> Integer, Intent(out) ::Pdim0,Pdim1 >>>> end Subroutine read_model >>>> end Interface >>>> >>>> Double Precision :: t1,t2 >>>> ! >>>> ! define MPI variables >>>> ! >>>> Integer :: Pid, N_proc, ierr >>>> Integer :: comm2d = 0 >>>> Integer :: status >>>> Integer :: period(2),Pdim_size(2),coords(2) >>>> Integer :: nbrleft,nbrright,nbrtop,nbrbottom >>>> Integer :: Pdim0, Pdim1 >>>> Integer :: i, t, numt >>>> >>>> call MPI_INIT(ierr) >>>> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) >>>> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) >>>> >>>> if (Pid .EQ. 0) then >>>> call read_model(Pdim0,Pdim1) >>>> >>>> end if >>>> ! >>>> ! Broad cast input parameters values to all processors >>>> ! >>>> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & >>>> & MPI_COMM_WORLD,ierr) >>>> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & >>>> & MPI_COMM_WORLD,ierr) >>>> >>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >>>> >>>> Pdim_size(1) = Pdim0 >>>> Pdim_size(2) = Pdim1 >>>> >>>> period(1) = 0 >>>> period(2) = 0 >>>> >>>> call MPI_Dims_create(N_proc,2,Pdim_size,ierr) >>>> >>>> call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) >>>> do i = 0,N_proc-1 >>>> if (Pid == i) then >>>> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d >>>> end if >>>> end do >>>> >>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >>>> call MPI_Comm_free(comm2d,ierr) >>>> call MPI_FINALIZE(ierr) >>>> >>>> end program mpicart >>>> >>>> ###################### OUTPUT ######################### >>>> >>>> Model parameters for mpicart test >>>> >>>> Processor Topology is ( 3 by 4 ) >>>> >>>> pid 0 mpi_comm_2d 140 >>>> pid 11 mpi_comm_2d 138 >>>> pid 2 mpi_comm_2d 140 >>>> pid 4 mpi_comm_2d 140 >>>> pid 6 mpi_comm_2d 140 >>>> pid 8 mpi_comm_2d 140 >>>> pid 3 mpi_comm_2d 138 >>>> pid 10 mpi_comm_2d 140 >>>> pid 1 mpi_comm_2d 138 >>>> pid 5 mpi_comm_2d 138 >>>> pid 7 mpi_comm_2d 138 >>>> pid 9 mpi_comm_2d 138 >>>> >>>> >>>> Chris Bording >>>> Application Analyst >>>> High Performance Computing Group >>>> Information Technology >>>> The College of William and Mary >>>> (757)-221-3488 >>>> rcbord@wm.edu >>>> _______________________________________________ >>>> mvapich-discuss mailing list >>>> mvapich-discuss@cse.ohio-state.edu >>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss >>>> >>> >>> >>> >> > > From mamidala at cse.ohio-state.edu Wed May 14 11:18:11 2008 From: mamidala at cse.ohio-state.edu (amith rajith mamidala) Date: Wed May 14 11:18:24 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: Message-ID: Hi Chris, For using tcp, you can try it out with gnu first. Thanks, Amith On Wed, 14 May 2008 rcbord@wm.edu wrote: > Amith, > Should I use the default gnu compilers or do you want me > to build with pgi? > > Chris Bording > Application Analyst > High Performance Computing Group > Information Technology > The College of William and Mary > (757)-221-3488 > rcbord@wm.edu > > On Wed, 14 May 2008, amith rajith mamidala wrote: > > > Hi Chris, > > > > Can you try out mvapich1 with tcp? You can use make.mvapich.tcp to > > compile. This way we will narrow down the code path causing the problem. > > > > Thanks, > > Amith > > > > On Wed, 14 May 2008 rcbord@wm.edu wrote: > > > >> Amith, > >> I was able to test mvapich1 but I get the same bad results > >> MPI_Cart_Create is returning two communication values. > >> > >> When I add the environment variable VIADEV_USE_SHMEM_COLL=0 > >> and the MPI_sendrecv function changes the dest and source values > >> that are set by the mpi_cart_shift function. > >> Note these errors only occur with F90. I have C/C++ codes > >> that work correctly. > >> > >> I can re-compile any or all mvapich-0.9.9, mvapich1 and mvapich2 > >> easily enough using the make.mvapich.gen2 scripts. What flags > >> should I be using for the pgi-7.0 compiler? I can update the > >> compiler too if necessary. > >> > >> > >> Chris Bording > >> Application Analyst > >> High Performance Computing Group > >> Information Technology > >> The College of William and Mary > >> (757)-221-3488 > >> rcbord@wm.edu > >> > >> On Wed, 7 May 2008, amith rajith mamidala wrote: > >> > >>> Hi, > >>> > >>> Can you try these two options? > >>> > >>> 1. use MVAPICH-1.0 and see if you are seeing the issue. > >>> > >>> 2.Can you run the code by passing the environment variable: > >>> VIADEV_USE_SHMEM_COLL=0 with mvapich > >>> > >>> > >>> Thanks, > >>> Amith > >>> > >>> > >>> On Wed, 7 May 2008 rcbord@wm.edu wrote: > >>> > >>>> Hi, > >>>> I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with > >>>> ofed 1.2 for infinaband. We have be able to use it for 6 months without > >>>> any problem. I tried to port a users fortran code that runs under > >>>> solaris-sparc without any problems. The code is fairly vanilla fotran-90 > >>>> with mpi I know because I wrote it. When I tried to port it to the Linux > >>>> Cluster it hangs a mpisendrecv function. I included a simple test code > >>>> that shows that the new communicator (comm2d) returned by the > >>>> MPI_Cart_create function has multiple values. Same code run on the > >>>> sun sparc cluster returns a single value. I am guessing it could be how > >>>> I compiled MVAPICH, but have tried a few more flags without any success. > >>>> I tried this with MVAPICH2 also and got the same error, but I compiled it > >>>> in a similar fashion. Could it be a PGI problem? > >>>> > >>>> > >>>> Has anyone else seen this? Thanks for any help! > >>>> > >>>> Program mpicart > >>>> ! With input files > >>>> ! Conversion of efit2d.f90 to 3d JPB 11-2007 > >>>> > >>>> Implicit none > >>>> > >>>> include "mpif.h" > >>>> > >>>> Interface > >>>> Subroutine read_model(Pdim0,Pdim1) > >>>> Integer, Intent(out) ::Pdim0,Pdim1 > >>>> end Subroutine read_model > >>>> end Interface > >>>> > >>>> Double Precision :: t1,t2 > >>>> ! > >>>> ! define MPI variables > >>>> ! > >>>> Integer :: Pid, N_proc, ierr > >>>> Integer :: comm2d = 0 > >>>> Integer :: status > >>>> Integer :: period(2),Pdim_size(2),coords(2) > >>>> Integer :: nbrleft,nbrright,nbrtop,nbrbottom > >>>> Integer :: Pdim0, Pdim1 > >>>> Integer :: i, t, numt > >>>> > >>>> call MPI_INIT(ierr) > >>>> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) > >>>> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) > >>>> > >>>> if (Pid .EQ. 0) then > >>>> call read_model(Pdim0,Pdim1) > >>>> > >>>> end if > >>>> ! > >>>> ! Broad cast input parameters values to all processors > >>>> ! > >>>> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & > >>>> & MPI_COMM_WORLD,ierr) > >>>> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & > >>>> & MPI_COMM_WORLD,ierr) > >>>> > >>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) > >>>> > >>>> Pdim_size(1) = Pdim0 > >>>> Pdim_size(2) = Pdim1 > >>>> > >>>> period(1) = 0 > >>>> period(2) = 0 > >>>> > >>>> call MPI_Dims_create(N_proc,2,Pdim_size,ierr) > >>>> > >>>> call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) > >>>> do i = 0,N_proc-1 > >>>> if (Pid == i) then > >>>> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d > >>>> end if > >>>> end do > >>>> > >>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) > >>>> call MPI_Comm_free(comm2d,ierr) > >>>> call MPI_FINALIZE(ierr) > >>>> > >>>> end program mpicart > >>>> > >>>> ###################### OUTPUT ######################### > >>>> > >>>> Model parameters for mpicart test > >>>> > >>>> Processor Topology is ( 3 by 4 ) > >>>> > >>>> pid 0 mpi_comm_2d 140 > >>>> pid 11 mpi_comm_2d 138 > >>>> pid 2 mpi_comm_2d 140 > >>>> pid 4 mpi_comm_2d 140 > >>>> pid 6 mpi_comm_2d 140 > >>>> pid 8 mpi_comm_2d 140 > >>>> pid 3 mpi_comm_2d 138 > >>>> pid 10 mpi_comm_2d 140 > >>>> pid 1 mpi_comm_2d 138 > >>>> pid 5 mpi_comm_2d 138 > >>>> pid 7 mpi_comm_2d 138 > >>>> pid 9 mpi_comm_2d 138 > >>>> > >>>> > >>>> Chris Bording > >>>> Application Analyst > >>>> High Performance Computing Group > >>>> Information Technology > >>>> The College of William and Mary > >>>> (757)-221-3488 > >>>> rcbord@wm.edu > >>>> _______________________________________________ > >>>> mvapich-discuss mailing list > >>>> mvapich-discuss@cse.ohio-state.edu > >>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > >>>> > >>> > >>> > >>> > >> > > > > > From rcbord at wm.edu Wed May 14 11:46:56 2008 From: rcbord at wm.edu (rcbord@wm.edu) Date: Wed May 14 11:47:13 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: References: Message-ID: Hi Amith, The make.mvapich.tcp script fails Changed the F77 value to gfortran as we don't have g77 on the system. I changed to the configure flags to: ./configure --prefix=$PREFIX --with-device=ch_p4 --with-arch=LINUX --without-mpe Here is the output of make.mvapich.tcp where it fails. Am I using the right device and arch values? make overtake /usr/mpi/mvapich-0.9.9/bin/mpicc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -DHAVE_MPICHCONF_H -DHAVE_STDLIB_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STRING_H=1 -DUSE_STDARG=1 -DHAVE_LONG_DOUBLE=1 -DHAVE_LONG_LONG_INT=1 -DHAVE_PROTOTYPES=1 -DHAVE_SIGNAL_H=1 -DHAVE_SIGACTION=1 -DHAVE_SLEEP=1 -DHAVE_SYSCONF=1 -c overtake.c /usr/mpi/mvapich-0.9.9/bin/mpicc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -DHAVE_MPICHCONF_H -DHAVE_STDLIB_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STRING_H=1 -DUSE_STDARG=1 -DHAVE_LONG_DOUBLE=1 -DHAVE_LONG_LONG_INT=1 -DHAVE_PROTOTYPES=1 -DHAVE_SIGNAL_H=1 -DHAVE_SIGACTION=1 -DHAVE_SLEEP=1 -DHAVE_SYSCONF=1 -c test.c /usr/mpi/mvapich-0.9.9/bin/mpicc -o overtake overtake.o test.o make clean make --no-print-directory mpi-addons make --no-print-directory mpelib make --no-print-directory mpi-utils gcc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -c /usr/mpi/mvapich-0.9.9/mpid/server/serv_p4.c -I./ -o serv_p4.o gcc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -c -I./ server_ssl.c gcc -o serv_p4 serv_p4.o server_ssl.o -lcrypt cp: cannot stat `/usr/mpi/mvapich-0.9.9/bin/tarch': No such file or directory cp: cannot stat `/usr/mpi/mvapich-0.9.9/bin/tdevice': No such file or directory make[1]: *** [mpi-utils] Error 1 make: *** [mpi] Error 2 Failure in Building MVAPICH Please file an error report to mvapich-discuss@cse.ohio-state.edu with all your log files. Chris Bording Application Analyst High Performance Computing Group Information Technology The College of William and Mary (757)-221-3488 rcbord@wm.edu On Wed, 14 May 2008, amith rajith mamidala wrote: > > Hi Chris, > > For using tcp, you can try it out with gnu first. > > Thanks, > Amith > > On Wed, 14 May 2008 rcbord@wm.edu wrote: > >> Amith, >> Should I use the default gnu compilers or do you want me >> to build with pgi? >> >> Chris Bording >> Application Analyst >> High Performance Computing Group >> Information Technology >> The College of William and Mary >> (757)-221-3488 >> rcbord@wm.edu >> >> On Wed, 14 May 2008, amith rajith mamidala wrote: >> >>> Hi Chris, >>> >>> Can you try out mvapich1 with tcp? You can use make.mvapich.tcp to >>> compile. This way we will narrow down the code path causing the problem. >>> >>> Thanks, >>> Amith >>> >>> On Wed, 14 May 2008 rcbord@wm.edu wrote: >>> >>>> Amith, >>>> I was able to test mvapich1 but I get the same bad results >>>> MPI_Cart_Create is returning two communication values. >>>> >>>> When I add the environment variable VIADEV_USE_SHMEM_COLL=0 >>>> and the MPI_sendrecv function changes the dest and source values >>>> that are set by the mpi_cart_shift function. >>>> Note these errors only occur with F90. I have C/C++ codes >>>> that work correctly. >>>> >>>> I can re-compile any or all mvapich-0.9.9, mvapich1 and mvapich2 >>>> easily enough using the make.mvapich.gen2 scripts. What flags >>>> should I be using for the pgi-7.0 compiler? I can update the >>>> compiler too if necessary. >>>> >>>> >>>> Chris Bording >>>> Application Analyst >>>> High Performance Computing Group >>>> Information Technology >>>> The College of William and Mary >>>> (757)-221-3488 >>>> rcbord@wm.edu >>>> >>>> On Wed, 7 May 2008, amith rajith mamidala wrote: >>>> >>>>> Hi, >>>>> >>>>> Can you try these two options? >>>>> >>>>> 1. use MVAPICH-1.0 and see if you are seeing the issue. >>>>> >>>>> 2.Can you run the code by passing the environment variable: >>>>> VIADEV_USE_SHMEM_COLL=0 with mvapich >>>>> >>>>> >>>>> Thanks, >>>>> Amith >>>>> >>>>> >>>>> On Wed, 7 May 2008 rcbord@wm.edu wrote: >>>>> >>>>>> Hi, >>>>>> I am having an issue with the MVAPICH-0.9.9 compiled with PGI-7.0 with >>>>>> ofed 1.2 for infinaband. We have be able to use it for 6 months without >>>>>> any problem. I tried to port a users fortran code that runs under >>>>>> solaris-sparc without any problems. The code is fairly vanilla fotran-90 >>>>>> with mpi I know because I wrote it. When I tried to port it to the Linux >>>>>> Cluster it hangs a mpisendrecv function. I included a simple test code >>>>>> that shows that the new communicator (comm2d) returned by the >>>>>> MPI_Cart_create function has multiple values. Same code run on the >>>>>> sun sparc cluster returns a single value. I am guessing it could be how >>>>>> I compiled MVAPICH, but have tried a few more flags without any success. >>>>>> I tried this with MVAPICH2 also and got the same error, but I compiled it >>>>>> in a similar fashion. Could it be a PGI problem? >>>>>> >>>>>> >>>>>> Has anyone else seen this? Thanks for any help! >>>>>> >>>>>> Program mpicart >>>>>> ! With input files >>>>>> ! Conversion of efit2d.f90 to 3d JPB 11-2007 >>>>>> >>>>>> Implicit none >>>>>> >>>>>> include "mpif.h" >>>>>> >>>>>> Interface >>>>>> Subroutine read_model(Pdim0,Pdim1) >>>>>> Integer, Intent(out) ::Pdim0,Pdim1 >>>>>> end Subroutine read_model >>>>>> end Interface >>>>>> >>>>>> Double Precision :: t1,t2 >>>>>> ! >>>>>> ! define MPI variables >>>>>> ! >>>>>> Integer :: Pid, N_proc, ierr >>>>>> Integer :: comm2d = 0 >>>>>> Integer :: status >>>>>> Integer :: period(2),Pdim_size(2),coords(2) >>>>>> Integer :: nbrleft,nbrright,nbrtop,nbrbottom >>>>>> Integer :: Pdim0, Pdim1 >>>>>> Integer :: i, t, numt >>>>>> >>>>>> call MPI_INIT(ierr) >>>>>> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) >>>>>> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) >>>>>> >>>>>> if (Pid .EQ. 0) then >>>>>> call read_model(Pdim0,Pdim1) >>>>>> >>>>>> end if >>>>>> ! >>>>>> ! Broad cast input parameters values to all processors >>>>>> ! >>>>>> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & >>>>>> & MPI_COMM_WORLD,ierr) >>>>>> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & >>>>>> & MPI_COMM_WORLD,ierr) >>>>>> >>>>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >>>>>> >>>>>> Pdim_size(1) = Pdim0 >>>>>> Pdim_size(2) = Pdim1 >>>>>> >>>>>> period(1) = 0 >>>>>> period(2) = 0 >>>>>> >>>>>> call MPI_Dims_create(N_proc,2,Pdim_size,ierr) >>>>>> >>>>>> call MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) >>>>>> do i = 0,N_proc-1 >>>>>> if (Pid == i) then >>>>>> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d >>>>>> end if >>>>>> end do >>>>>> >>>>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >>>>>> call MPI_Comm_free(comm2d,ierr) >>>>>> call MPI_FINALIZE(ierr) >>>>>> >>>>>> end program mpicart >>>>>> >>>>>> ###################### OUTPUT ######################### >>>>>> >>>>>> Model parameters for mpicart test >>>>>> >>>>>> Processor Topology is ( 3 by 4 ) >>>>>> >>>>>> pid 0 mpi_comm_2d 140 >>>>>> pid 11 mpi_comm_2d 138 >>>>>> pid 2 mpi_comm_2d 140 >>>>>> pid 4 mpi_comm_2d 140 >>>>>> pid 6 mpi_comm_2d 140 >>>>>> pid 8 mpi_comm_2d 140 >>>>>> pid 3 mpi_comm_2d 138 >>>>>> pid 10 mpi_comm_2d 140 >>>>>> pid 1 mpi_comm_2d 138 >>>>>> pid 5 mpi_comm_2d 138 >>>>>> pid 7 mpi_comm_2d 138 >>>>>> pid 9 mpi_comm_2d 138 >>>>>> >>>>>> >>>>>> Chris Bording >>>>>> Application Analyst >>>>>> High Performance Computing Group >>>>>> Information Technology >>>>>> The College of William and Mary >>>>>> (757)-221-3488 >>>>>> rcbord@wm.edu >>>>>> _______________________________________________ >>>>>> mvapich-discuss mailing list >>>>>> mvapich-discuss@cse.ohio-state.edu >>>>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss >>>>>> >>>>> >>>>> >>>>> >>>> >>> >>> >> > > From rcbord at wm.edu Wed May 14 12:20:10 2008 From: rcbord at wm.edu (rcbord@wm.edu) Date: Wed May 14 12:20:27 2008 Subject: [mvapich-discuss] MPI_Cart_Create In-Reply-To: References: Message-ID: Hi Amith, I could not make the tcp version of mvapich-0.9.9 using gnu or pgi. I was able to compile the tcp version of mvapich1 with gnu but it did not build the mpif77 or mpif90. So I will skip the mvapich-0.9.9 testing for now. Chris Bording Application Analyst High Performance Computing Group Information Technology The College of William and Mary (757)-221-3488 rcbord@wm.edu On Wed, 14 May 2008 rcbord@wm.edu wrote: > Hi Amith, > The make.mvapich.tcp script fails > Changed the F77 value to gfortran as we don't have g77 on the system. > I changed to the configure flags to: > ./configure --prefix=$PREFIX --with-device=ch_p4 --with-arch=LINUX > --without-mpe > > Here is the output of make.mvapich.tcp where it fails. Am I using the > right device and arch values? > > > > make overtake > /usr/mpi/mvapich-0.9.9/bin/mpicc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR > -DHAVE_MPICHCONF_H -DHAVE_STDLIB_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STRING_H=1 > -DUSE_STDARG=1 -DHAVE_LONG_DOUBLE=1 -DHAVE_LONG_LONG_INT=1 > -DHAVE_PROTOTYPES=1 -DHAVE_SIGNAL_H=1 -DHAVE_SIGACTION=1 -DHAVE_SLEEP=1 > -DHAVE_SYSCONF=1 -c overtake.c > /usr/mpi/mvapich-0.9.9/bin/mpicc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR > -DHAVE_MPICHCONF_H -DHAVE_STDLIB_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STRING_H=1 > -DUSE_STDARG=1 -DHAVE_LONG_DOUBLE=1 -DHAVE_LONG_LONG_INT=1 > -DHAVE_PROTOTYPES=1 -DHAVE_SIGNAL_H=1 -DHAVE_SIGACTION=1 -DHAVE_SLEEP=1 > -DHAVE_SYSCONF=1 -c test.c > /usr/mpi/mvapich-0.9.9/bin/mpicc -o overtake overtake.o test.o > make clean > make --no-print-directory mpi-addons > make --no-print-directory mpelib > make --no-print-directory mpi-utils > gcc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -c > /usr/mpi/mvapich-0.9.9/mpid/server/serv_p4.c -I./ -o serv_p4.o > gcc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -c -I./ server_ssl.c > gcc -o serv_p4 serv_p4.o server_ssl.o -lcrypt > cp: cannot stat `/usr/mpi/mvapich-0.9.9/bin/tarch': No such file or directory > cp: cannot stat `/usr/mpi/mvapich-0.9.9/bin/tdevice': No such file or > directory > make[1]: *** [mpi-utils] Error 1 > make: *** [mpi] Error 2 > Failure in Building MVAPICH > Please file an error report to mvapich-discuss@cse.ohio-state.edu with all > your log files. > > > > > Chris Bording > Application Analyst > High Performance Computing Group Information Technology > The College of William and Mary > (757)-221-3488 > rcbord@wm.edu > > On Wed, 14 May 2008, amith rajith mamidala wrote: > >> >> Hi Chris, >> >> For using tcp, you can try it out with gnu first. >> >> Thanks, >> Amith >> >> On Wed, 14 May 2008 rcbord@wm.edu wrote: >> >>> Amith, >>> Should I use the default gnu compilers or do you want me >>> to build with pgi? >>> >>> Chris Bording >>> Application Analyst >>> High Performance Computing Group >>> Information Technology >>> The College of William and Mary >>> (757)-221-3488 >>> rcbord@wm.edu >>> >>> On Wed, 14 May 2008, amith rajith mamidala wrote: >>> >>>> Hi Chris, >>>> >>>> Can you try out mvapich1 with tcp? You can use make.mvapich.tcp to >>>> compile. This way we will narrow down the code path causing the >>>> problem. >>>> >>>> Thanks, >>>> Amith >>>> >>>> On Wed, 14 May 2008 rcbord@wm.edu wrote: >>>> >>>>> Amith, >>>>> I was able to test mvapich1 but I get the same bad results >>>>> MPI_Cart_Create is returning two communication values. >>>>> >>>>> When I add the environment variable VIADEV_USE_SHMEM_COLL=0 >>>>> and the MPI_sendrecv function changes the dest and source values >>>>> that are set by the mpi_cart_shift function. >>>>> Note these errors only occur with F90. I have C/C++ codes >>>>> that work correctly. >>>>> >>>>> I can re-compile any or all mvapich-0.9.9, mvapich1 and mvapich2 >>>>> easily enough using the make.mvapich.gen2 scripts. What flags >>>>> should I be using for the pgi-7.0 compiler? I can update the >>>>> compiler too if necessary. >>>>> >>>>> >>>>> Chris Bording >>>>> Application Analyst >>>>> High Performance Computing Group >>>>> Information Technology >>>>> The College of William and Mary >>>>> (757)-221-3488 >>>>> rcbord@wm.edu >>>>> >>>>> On Wed, 7 May 2008, amith rajith mamidala wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> Can you try these two options? >>>>>> >>>>>> 1. use MVAPICH-1.0 and see if you are seeing the issue. >>>>>> >>>>>> 2.Can you run the code by passing the environment variable: >>>>>> VIADEV_USE_SHMEM_COLL=0 with mvapich >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Amith >>>>>> >>>>>> >>>>>> On Wed, 7 May 2008 rcbord@wm.edu wrote: >>>>>> >>>>>>> Hi, >>>>>>> I am having an issue with the MVAPICH-0.9.9 compiled with >>>>>>> PGI-7.0 with >>>>>>> ofed 1.2 for infinaband. We have be able to use it for 6 months >>>>>>> without >>>>>>> any problem. I tried to port a users fortran code that runs >>>>>>> under >>>>>>> solaris-sparc without any problems. The code is fairly vanilla >>>>>>> fotran-90 >>>>>>> with mpi I know because I wrote it. When I tried to port it to >>>>>>> the Linux >>>>>>> Cluster it hangs a mpisendrecv function. I included a simple >>>>>>> test code >>>>>>> that shows that the new communicator (comm2d) returned by the >>>>>>> MPI_Cart_create function has multiple values. Same code run on >>>>>>> the >>>>>>> sun sparc cluster returns a single value. I am guessing it >>>>>>> could be how >>>>>>> I compiled MVAPICH, but have tried a few more flags without any >>>>>>> success. >>>>>>> I tried this with MVAPICH2 also and got the same error, but I >>>>>>> compiled it >>>>>>> in a similar fashion. Could it be a PGI problem? >>>>>>> >>>>>>> >>>>>>> Has anyone else seen this? Thanks for any help! >>>>>>> >>>>>>> Program mpicart >>>>>>> ! With input files >>>>>>> ! Conversion of efit2d.f90 to 3d JPB 11-2007 >>>>>>> >>>>>>> Implicit none >>>>>>> >>>>>>> include "mpif.h" >>>>>>> >>>>>>> Interface >>>>>>> Subroutine read_model(Pdim0,Pdim1) >>>>>>> Integer, Intent(out) ::Pdim0,Pdim1 >>>>>>> end Subroutine read_model >>>>>>> end Interface >>>>>>> >>>>>>> Double Precision :: t1,t2 >>>>>>> ! >>>>>>> ! define MPI variables >>>>>>> ! >>>>>>> Integer :: Pid, N_proc, ierr >>>>>>> Integer :: comm2d = 0 >>>>>>> Integer :: status >>>>>>> Integer :: period(2),Pdim_size(2),coords(2) >>>>>>> Integer :: nbrleft,nbrright,nbrtop,nbrbottom >>>>>>> Integer :: Pdim0, Pdim1 >>>>>>> Integer :: i, t, numt >>>>>>> >>>>>>> call MPI_INIT(ierr) >>>>>>> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr) >>>>>>> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr) >>>>>>> >>>>>>> if (Pid .EQ. 0) then >>>>>>> call read_model(Pdim0,Pdim1) >>>>>>> >>>>>>> end if >>>>>>> ! >>>>>>> ! Broad cast input parameters values to all processors >>>>>>> ! >>>>>>> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, & >>>>>>> & MPI_COMM_WORLD,ierr) >>>>>>> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, & >>>>>>> & MPI_COMM_WORLD,ierr) >>>>>>> >>>>>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >>>>>>> >>>>>>> Pdim_size(1) = Pdim0 >>>>>>> Pdim_size(2) = Pdim1 >>>>>>> >>>>>>> period(1) = 0 >>>>>>> period(2) = 0 >>>>>>> >>>>>>> call MPI_Dims_create(N_proc,2,Pdim_size,ierr) >>>>>>> >>>>>>> call >>>>>>> MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr) >>>>>>> do i = 0,N_proc-1 >>>>>>> if (Pid == i) then >>>>>>> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d >>>>>>> end if >>>>>>> end do >>>>>>> >>>>>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr) >>>>>>> call MPI_Comm_free(comm2d,ierr) >>>>>>> call MPI_FINALIZE(ierr) >>>>>>> >>>>>>> end program mpicart >>>>>>> >>>>>>> ###################### OUTPUT ######################### >>>>>>> >>>>>>> Model parameters for mpicart test >>>>>>> >>>>>>> Processor Topology is ( 3 by 4 ) >>>>>>> >>>>>>> pid 0 mpi_comm_2d 140 >>>>>>> pid 11 mpi_comm_2d 138 >>>>>>> pid 2 mpi_comm_2d 140 >>>>>>> pid 4 mpi_comm_2d 140 >>>>>>> pid 6 mpi_comm_2d 140 >>>>>>> pid 8 mpi_comm_2d 140 >>>>>>> pid 3 mpi_comm_2d 138 >>>>>>> pid 10 mpi_comm_2d 140 >>>>>>> pid 1 mpi_comm_2d 138 >>>>>>> pid 5 mpi_comm_2d 138 >>>>>>> pid 7 mpi_comm_2d 138 >>>>>>> pid 9 mpi_comm_2d 138 >>>>>>> >>>>>>> >>>>>>> Chris Bording >>>>>>> Application Analyst >>>>>>> High Performance Computing Group >>>>>>> Information Technology >>>>>>> The College of William and Mary >>>>>>> (757)-221-3488 >>>>>>> rcbord@wm.edu >>>>>>> _______________________________________________ >>>>>>> mvapich-discuss mailing list >>>>>>> mvapich-discuss@cse.ohio-state.edu >>>>>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >> >> > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From yogyas at gmail.com Thu May 15 06:28:50 2008 From: yogyas at gmail.com (yogeshwar sonawane) Date: Thu May 15 06:29:04 2008 Subject: [mvapich-discuss] small, medium, large cluster distinction ? Message-ID: Hi all, While compiling MVAPICH2 with udapl library, there are three options. One option is to choose from small, medium, large cluster. What should be the criteria to decide the cluster size i.e. small, medium or large ? OR On what basis, one should choose the cluster size ? What is the optimal range for the criteria ? Thanks, Yogeshwar From chai.15 at osu.edu Thu May 15 10:48:45 2008 From: chai.15 at osu.edu (LEI CHAI) Date: Thu May 15 10:49:56 2008 Subject: [mvapich-discuss] small, medium, large cluster distinction ? In-Reply-To: References: Message-ID: An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080515/50bb3e03/attachment-0001.html From forum.san at gmail.com Thu May 15 12:21:44 2008 From: forum.san at gmail.com (Sangamesh B) Date: Thu May 15 12:22:00 2008 Subject: [mvapich-discuss] mvapich2 makefile + required libraries Message-ID: Hi all, In mvapich2 package, different makefiles are given: make.mvapich2.ofa make.mvapich2.vapi make.mvapich2.udapl I'm using Mellanox hardware and OFED-1.2.5.5 from http://www.openfabrics.org/ I found many libraries in http://www.openfabrics.org/OldDownloads.htm But not getting what additional libraries to use apart from OFED. Can some one give a listing of required libraries for the above listed different makefiles. Thanks, Sangamesh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080515/3d430d0a/attachment.html From koop at cse.ohio-state.edu Thu May 15 13:29:56 2008 From: koop at cse.ohio-state.edu (Matthew Koop) Date: Thu May 15 13:30:09 2008 Subject: [mvapich-discuss] mvapich2 makefile + required libraries In-Reply-To: Message-ID: Sangamesh, Please refer to the MVAPICH2 user guide for information on all of the makefiles available. As mentioned in our offline discussion, you must use the 'make.mvapich.ofa' script with OFED. There is no need to download any other packages/libraries. https://mvapich.cse.ohio-state.edu/support/mvapich2_user_guide.pdf Thanks, Matt On Thu, 15 May 2008, Sangamesh B wrote: > Hi all, > > In mvapich2 package, different makefiles are given: > > make.mvapich2.ofa > make.mvapich2.vapi > make.mvapich2.udapl > > I'm using Mellanox hardware and OFED-1.2.5.5 from > http://www.openfabrics.org/ > > I found many libraries in http://www.openfabrics.org/OldDownloads.htm > > But not getting what additional libraries to use apart from OFED. > > Can some one give a listing of required libraries for the above listed > different makefiles. > > Thanks, > Sangamesh > From biswajit at crlindia.com Fri May 16 02:24:23 2008 From: biswajit at crlindia.com (biswajit@crlindia.com) Date: Fri May 16 02:24:27 2008 Subject: [mvapich-discuss] source code documents Message-ID: I have following queries:- 1. Do you have any documentation of source code for mvapich2-1.0.2 ?? 2. Where is the defination of the functions for Channel ( i.e MPIDI_CH3_iStartMsg, MPIDI_CH3_iStartRndvMsg ) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080516/b4ede276/attachment.html From bino at nacad.ufrj.br Fri May 16 15:16:21 2008 From: bino at nacad.ufrj.br (Albino dos Anjos Aveleda) Date: Fri May 16 15:45:08 2008 Subject: [mvapich-discuss] Compile error Message-ID: <20080516180939.M51255@nacad.ufrj.br> Hi, I am trying to build the mvapich2-1.0.2 using Intel compiler. My machine is SGI Altix ICE and my configure options are bellow: ./configure '--prefix=/opt/mvapich2-1.0.2-1/intel' \ '--with-device=osu_ch3:mrail' '--with-rdma=gen2' '--with-pm=mpd' \ '--enable-romio' '--enable-sharedlibs=gcc' '--without-mpe' \ 'CC=icc' 'CFLAGS=-D__INTEL_COMPILER -D_EM64T_ -D_SMP_ \ -DUSE_HEADER_CACHING -DONE_SIDED -DMPIDI_CH3_CHANNEL_RNDV \ -DMPID_USE_SEQUENCE_NUMBERS -DRDMA_CM -I/usr/include -O2' \ 'CXX=icpc' 'F77=ifort' 'F90=ifort' 'FFLAGS=-L/usr/lib64' In attachment is the make.log file. How do I fix this? Best regards, Albino -- Albino A. Aveleda bino@nacad.ufrj.br Network Manager +55 21 2562-8080 NACAD-COPPE/UFRJ http://www.nacad.ufrj.br Federal University of Rio de Janeiro (UFRJ) -------------- next part -------------- A non-text attachment was scrubbed... Name: make.log.gz Type: application/x-gzip-compressed Size: 28005 bytes Desc: not available Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080516/8d85d67e/make.log-0001.bin From peter.cebull at inl.gov Fri May 16 17:35:10 2008 From: peter.cebull at inl.gov (Peter Cebull) Date: Fri May 16 17:35:57 2008 Subject: [mvapich-discuss] Compile error In-Reply-To: <20080516180939.M51255@nacad.ufrj.br> References: <20080516180939.M51255@nacad.ufrj.br> Message-ID: <482DFE0E.5030609@inl.gov> An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080516/dec11dd5/attachment.html -------------- next part -------------- #!/bin/bash # Most variables here can be overridden by exporting them in the environment # before running this script. Default values have been provided if the # environment variable is not already set. source ./make.mvapich2.def # The target architecture. If not exported outside of this script, # it will be found automatically or prompted for if necessary. # Supported: "_IA32_", "_IA64_", "_EM64T_", "_X86_64_" if [ -z "$ARCH" ]; then arch fi if [ $ARCH = "SOLARIS" ] || [ $ARCH = "MAC_OSX" ]; then die_setup "MVAPICH2 OFA is not supported on this architecture: $ARCH." fi # Mandatory variables. All are checked except CXX and F90. OPEN_IB_HOME=${OPEN_IB_HOME:-/usr} #PREFIX=${PREFIX:-/usr/local/mvapich2/mvapich2-1.0.2/intel-dbg} #export CC=${CC:-icc} #export CXX=${CXX:-icpc} #export F77=${F77:-ifort} #export F90=${F90:-ifort} #PREFIX=${PREFIX:-/usr/local/mvapich2/mvapich2-1.0.2/gcc-dbg} #export CC=${CC:-gcc} #export CXX=${CXX:-g++} #export F77=${F77:-gfortran} #export F90=${F90:-gfortran} #PREFIX=${PREFIX:-/usr/local/mvapich2/mvapich2-1.0.2/gcc-opt} #export CC=${CC:-gcc} #export CXX=${CXX:-g++} #export F77=${F77:-gfortran} #export F90=${F90:-gfortran} PREFIX=${PREFIX:-/usr/local/mvapich2/mvapich2-1.0.2/intel-opt} export CC=${CC:-icc} export CXX=${CXX:-icpc} export F77=${F77:-ifort} export F90=${F90:-ifort} # Check mandatory variable settings. if [ -z "$OPEN_IB_HOME" ] || [ -z "$PREFIX" ] || [ -z "$CC" ] || [ -z "$F77" ]; then die_setup "Please set mandatory variables in this script or the environment." elif [ ! -d $OPEN_IB_HOME ]; then die_setup "OPEN_IB_HOME directory $OPEN_IB_HOME does not exist." elif [ -d $OPEN_IB_HOME/lib64 ]; then OPEN_IB_LIB=$OPEN_IB_HOME/lib64 elif [ -d $OPEN_IB_HOME/lib ]; then OPEN_IB_LIB=$OPEN_IB_HOME/lib else die_setup "Could not find the OPEN_IB_HOME/lib64 or OPEN_IB_HOME/lib directory." fi # Set this to override automatic optimization setting (-O2). OPT_FLAG=${OPT_FLAG:--fPIC -O2} export MPI_F90FLAGS=$OPT_FLAG # Checkpoint-Restart Support. This is disabled by default. Enable with # "yes". If this is enabled, the following rules apply: # # - PTMALLOC must be enabled # - multithreading cannot be enabled # - rdma-cm support cannot be enabled ENABLE_CKPT=${ENABLE_CKPT:-} if [ "$ENABLE_CKPT" = "yes" ]; then if [ ! -z "$BLCR_HOME" ]; then BLCR_LIB="-L$BLCR_HOME/lib -lcr" BLCR_INC="-I$BLCR_HOME/include" CR_FLAG="-DCKPT -DSYNC_CKPT" SMP_FLAG="" export MPIEXEC_CR=mpiexec_cr # to compile mpiexec_cr LD_LIBRARY_PATH=$BLCR_HOME/lib:$LD_LIBRARY_PATH else die_setup "Please install BLCR and have set BLCR_HOME pointing to the directory." fi else SMP_FLAG="-D_SMP_" fi # PTMALLOC support for MVAPICH2 memory hooks. Enabling this will allow # MVAPICH2 to release memory to the Operating System (when registration # cache is enabled). Enabled by default. Disable with "no". PTMALLOC=${PTMALLOC:-} if [ "$PTMALLOC" = "no" ]; then if [ "$ENABLE_CKPT" = "yes" ]; then die_setup "Checkpoint-Restart requires PTMALLOC support." fi PTMALLOC="-DDISABLE_PTMALLOC" else PTMALLOC="" fi # Whether or not to build rdma-cm support. Disabled by default. # Enable with "yes". RDMA_CM_SUPPORT=${RDMA_CM_SUPPORT:-"yes"} if [ "$RDMA_CM_SUPPORT" = "yes" ]; then if [ "$ENABLE_CKPT" = "yes" ]; then die_setup "Checkpoint-Restart does not support rdma-cm." fi RDMA_CM_LIBS="-lrdmacm" RDMA_CM_FLAG="-DRDMA_CM" # RDMA_CM_FLAG="-DRDMA_CM -DOFED_VERSION_1_1" fi # Whether or not to build with ROMIO MPI I/O support. Disabled by default. # Enable with "yes". ROMIO=${ROMIO:-} if [ "$ROMIO" = "yes" ]; then ROMIO="--enable-romio" else ROMIO="--disable-romio" fi # Whether or not to build with shared library support. Disabled by default. # Enabled with "yes". SHARED_LIBS=${SHARED_LIBS:-} if [ "$SHARED_LIBS" = "yes" ]; then SHARED_LIBS="--enable-sharedlibs=gcc" else SHARED_LIBS="" fi export LD_LIBRARY_PATH=$OPEN_IB_LIB:$LD_LIBRARY_PATH export LIBS=${LIBS:--L${OPEN_IB_LIB} ${BLCR_LIB} ${RDMA_CM_LIBS} -libverbs -libumad -lpthread} export FFLAGS=${FFLAGS:--L${OPEN_IB_LIB} -fPIC} export CFLAGS=${CFLAGS:--fPIC -D${ARCH} ${SMP_FLAG} -DUSE_HEADER_CACHING ${PTMALLOC} -DONE_SIDED -DMPIDI_CH3_CHANNEL_RNDV -DMPID_USE_SEQUENCE_NUMBERS ${RDMA_CM_FLAG} ${CR_FLAG} ${BLCR_INC} -I${OPEN_IB_HOME}/include $OPT_FLAG} # Prelogue make distclean &>/dev/null rm -rf *.cache *.log *.status lib bin set -o pipefail echo "Configuring MVAPICH2..." #./configure --prefix=${PREFIX} \ # --with-device=osu_ch3:mrail --with-rdma=gen2 --with-pm=mpd \ # --enable-g=dbg --enable-shared=gcc --enable-sharedlibs=gcc --enable-debuginfo --enable-totalview \ # ${ROMIO} ${SHARED_LIBS} --without-mpe 2>&1 |tee config-mine.log ./configure --prefix=${PREFIX} \ --with-device=osu_ch3:mrail --with-rdma=gen2 --with-pm=mpd \ --enable-shared=gcc --enable-sharedlibs=gcc \ ${ROMIO} ${SHARED_LIBS} --without-mpe 2>&1 |tee config-mine.log ret=$? tail config-mine.log test $ret = 0 || die "configuration." # Build MVAPICH2 echo "Building MVAPICH2..." make 2>&1 |tee make-mine.log ret=$? tail make-mine.log test $ret = 0 || die "building MVAPICH2." # Install MVAPICH2 echo "MVAPICH2 installation..." rm -f install-mine.log make install 2>&1 |tee install-mine.log ret=$? tail install-mine.log test $ret = 0 || die "installing MVAPICH2." # Epilogue echo "Congratulations on successfully building MVAPICH2. Please send your feedback to mvapich-discuss@cse.ohio-state.edu." From bino at nacad.ufrj.br Mon May 19 09:49:05 2008 From: bino at nacad.ufrj.br (Albino dos Anjos Aveleda) Date: Mon May 19 10:12:31 2008 Subject: [mvapich-discuss] Compile error In-Reply-To: <482DFE0E.5030609@inl.gov> References: <20080516180939.M51255@nacad.ufrj.br> <482DFE0E.5030609@inl.gov> Message-ID: <20080519124751.M11610@nacad.ufrj.br> Hi Peter, Thank you for your make script. It is working now. Best regards, Albino On Fri, 16 May 2008 15:35:10 -0600, Peter Cebull wrote > Albino, > > Attached is the make script I used to build MVAPICH2-1.0.2 on our > AltixICE (SGI ProPack 5 Service Pack 3). I think your build is > gettingconfused by /usr/include/mpi.h, which is part of SGI's Message > PassingToolkit. I had a similar problem on our system. Take a look at > theattached file, I'm thinking that you don't want "-I/usr/include" > onyour configure line. > > Good Luck, > Peter > > Albino dos Anjos Aveleda wrote:Hi, > > I am trying to build the mvapich2-1.0.2 using Intel compiler. > My machine is SGI Altix ICE and my configure options are bellow: > > ./configure '--prefix=/opt/mvapich2-1.0.2-1/intel' \ > '--with-device=osu_ch3:mrail' '--with-rdma=gen2' '--with-pm=mpd' \ > '--enable-romio' '--enable-sharedlibs=gcc' '--without-mpe' \ > 'CC=icc' 'CFLAGS=-D__INTEL_COMPILER -D_EM64T_ -D_SMP_ \ > -DUSE_HEADER_CACHING -DONE_SIDED -DMPIDI_CH3_CHANNEL_RNDV \ > -DMPID_USE_SEQUENCE_NUMBERS -DRDMA_CM -I/usr/include -O2' \ > 'CXX=icpc' 'F77=ifort' 'F90=ifort' 'FFLAGS=-L/usr/lib64' > > In attachment is the make.log file. > How do I fix this? > > Best regards, > Albino > > -- > Peter Cebull > HPC User Consultant > Idaho National Laboratory > P.O. Box 1625, MS3605 > Idaho Falls, ID 83415 > Phone: 208-526-1909 > Email: Peter.Cebull@inl.gov From Terrence.LIAO at total.com Wed May 21 16:34:09 2008 From: Terrence.LIAO at total.com (Terrence.LIAO@total.com) Date: Wed May 21 16:34:30 2008 Subject: [mvapich-discuss] problem run mvapich1.0 on AMD quad core cluster In-Reply-To: <200805161957.m4GJurNK023735@cse.ohio-state.edu> Message-ID: Hi, mvapich-discuss, We have a AMD quad core cluster with RHEL 4.5, OFED 1.2.5 with PGI compiler. I build mvapich1.0 as usually, but mpi hello world hung at MPI_INIT(). I can run with OpenMPI build. Also, the ib_write_bw and ibv_rc_pingpong ran fine. Any clue where I did wrong? Thank you very much. -- Terrence -------------------------------------------------------- Terrence Liao, Ph.D. Research Computer Scientist TOTAL E&P RESEARCH & TECHNOLOGY USA, LLC 1201 Louisiana, Suite 1800, Houston, TX 77002 Tel: 713.647.3498 Fax: 713.647.3638 Email: terrence.liao@total.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080521/064067cc/attachment.html From Terrence.LIAO at total.com Fri May 23 08:21:39 2008 From: Terrence.LIAO at total.com (Terrence.LIAO@total.com) Date: Fri May 23 08:22:10 2008 Subject: [mvapich-discuss] Re: problem run mvapich1.0 on AMD quad core cluster In-Reply-To: Message-ID: Hi, mvapich-discuss I find a work around on my mvapich problem. The build with -D_SMP_ and -D_SMP_RNDV flags removed from make.mvapich.gen2 works on my cluster. Thank you very much. -- Terrence -------------------------------------------------------- Terrence Liao, Ph.D. Research Computer Scientist TOTAL E&P RESEARCH & TECHNOLOGY USA, LLC 1201 Louisiana, Suite 1800, Houston, TX 77002 Tel: 713.647.3498 Fax: 713.647.3638 Email: terrence.liao@total.com Terrence LIAO/HOU/US/EP/Corp 05/21/2008 03:34 PM To mvapich-discuss@cse.ohio-state.edu cc Terrence LIAO/HOU/US/EP/Corp@E&P Subject problem run mvapich1.0 on AMD quad core cluster Hi, mvapich-discuss, We have a AMD quad core cluster with RHEL 4.5, OFED 1.2.5 with PGI compiler. I build mvapich1.0 as usually, but mpi hello world hung at MPI_INIT(). I can run with OpenMPI build. Also, the ib_write_bw and ibv_rc_pingpong ran fine. Any clue where I did wrong? Thank you very much. -- Terrence -------------------------------------------------------- Terrence Liao, Ph.D. Research Computer Scientist TOTAL E&P RESEARCH & TECHNOLOGY USA, LLC 1201 Louisiana, Suite 1800, Houston, TX 77002 Tel: 713.647.3498 Fax: 713.647.3638 Email: terrence.liao@total.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080523/22d393c7/attachment.html From ihara at sun.com Fri May 23 09:08:13 2008 From: ihara at sun.com (Shuichi Ihara) Date: Fri May 23 09:08:36 2008 Subject: [mvapich-discuss] mpif77 error Message-ID: <4836C1BD.4060700@sun.com> Hi I'm attempting to build mvapich-0.9.9(from OFED-1.2.5.5) with PGI-7.1. As first, the package was built, but we couldn't see mpif77 and mpif90 in this package. Here is configure's log. It seems "-B" option is not recognized to pgf77 and it can't be passed, then pgif77 and pgf90 were skipped. -snip checking whether the compiler pgcc runs... yes checking whether the compiler pgf77 runs... no Fortran compiler returned non-zero return code Output from test was pgf77-Error-Unknown switch: -B checking for correct handling of conditionals...... yes -snip That's ok. I removed "-B" option from CFLAGS in spec file of mvapich. After that, I could build mvapich which includes mpif77 and mpif90. However, when I run mpif77 and mpif90 to compile mpi application, I can see below errors -bash-3.1$ /usr/mpi/pgi/mvapich-0.9.9/bin/mpif77 hello_f77.f hello_f77.o: In function `MAIN_': hello_f77.f:(.text+0x1f): undefined reference to `mpi_comm_rank_' hello_f77.f:(.text+0x31): undefined reference to `mpi_comm_size_' hello_f77.f:(.text+0xc1): undefined reference to `mpi_finalize_' mpicc and mpiCC are no problem. Any ideas? Thanks, -Ihara From David_Kewley at Dell.com Fri May 23 22:23:37 2008 From: David_Kewley at Dell.com (David_Kewley@Dell.com) Date: Fri May 23 22:23:57 2008 Subject: [mvapich-discuss] Bug: deadlock between ibv_destroy_srq and async_thread Message-ID: I have a user running a 192-way job using MVAPICH2 1.0.1 and OFED 1.2.5.5, where MPI_Finalize() does not return. In the two example jobs I've examined, 189 processes exited, but the other three hung. The ranks that hung were different in the two examples, so I don't think the "3" is significant. All processes I've looked at appear to be stuck in the same way. In normal running, each process has four threads. When the process gets stuck, only the original thread remains. Here is a gdb backtrace from one: #0 0x00000036b2608b3a in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/tls/libpthread.so.0 #1 0x0000002a9595405b in ibv_cmd_destroy_srq (srq=0x82b370) at src/cmd.c:582 #2 0x0000002a962b5419 in mthca_destroy_srq (srq=0x82b3bc) at src/verbs.c:475 #3 0x0000002a9564878e in MPIDI_CH3I_CM_Finalize () from /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so #4 0x0000002a955c053b in MPIDI_CH3_Finalize () from /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so #5 0x0000002a95626202 in MPID_Finalize () from /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so #6 0x0000002a955f7fee in PMPI_Finalize () from /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so #7 0x0000002a955f7eae in pmpi_finalize_ () from /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so #8 0x0000000000459ff8 in stoprog_ () #9 0x000000000047afa6 in MAIN__ () #10 0x0000000000405d62 in main () After hours of opportunity to study the MVAPICH2 code :), I think I tracked it down to lines 1302-1306 in rdma_iba_init.c: if (MPIDI_CH3I_RDMA_Process.has_srq) { pthread_cancel(MPIDI_CH3I_RDMA_Process.async_thread[i]); pthread_join(MPIDI_CH3I_RDMA_Process.async_thread[i], NULL); ibv_destroy_srq(MPIDI_CH3I_RDMA_Process.srq_hndl[i]); } Consider what would happen if async_thread() were processing a IBV_EVENT_SRQ_LIMIT_REACHED event when pthread_cancel() was called on async_thread(). async_thread() has already called ibv_get_async_event() for this event, but it has not yet called ibv_ack_async_event(). The result would be the observed deadlock in this part of ibv_cmd_destroy_srq(): pthread_mutex_lock(&srq->mutex); while (srq->events_completed != resp.events_reported) pthread_cond_wait(&srq->cond, &srq->mutex); pthread_mutex_unlock(&srq->mutex); That is, events_completed == events_reported-1 at this point. The pthread_cond_signal() would be called, and events_completed could be made equal to events_reported, only by by calling ibv_ack_async_event() on this event. But that will never happen because async_thread() is the only code that would have done that, and it's already been pthread_cancel()'d and pthread_join()'d before ibv_destroy_srq() is called. I think the fix is to add some sort of synchronization between async_thread() and the code that calls the pthread_cancel() on it. To the MVAPICH developers: Do you think you can work up a fix soon, and forward the patch for testing? Thanks, David David Kewley Dell Infrastructure Consulting Services Onsite Engineer at the Maui HPC Center Cell: 602-460-7617 David_Kewley@Dell.com Dell Services: http://www.dell.com/services/ How am I doing? Email my manager Russell_Kelly@Dell.com with any feedback. From christian.guggenberger at rzg.mpg.de Sat May 24 11:10:58 2008 From: christian.guggenberger at rzg.mpg.de (Christian Guggenberger) Date: Sat May 24 11:11:14 2008 Subject: [mvapich-discuss] Bug: deadlock between ibv_destroy_srq and async_thread In-Reply-To: References: Message-ID: <20080524151058.GA5849@daltons.rzg.mpg.de> On Fri, May 23, 2008 at 09:23:37PM -0500, David_Kewley@dell.com wrote: > > #0 0x00000036b2608b3a in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/tls/libpthread.so.0 > #1 0x0000002a9595405b in ibv_cmd_destroy_srq (srq=0x82b370) at > src/cmd.c:582 > #2 0x0000002a962b5419 in mthca_destroy_srq (srq=0x82b3bc) at > src/verbs.c:475 > #3 0x0000002a9564878e in MPIDI_CH3I_CM_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #4 0x0000002a955c053b in MPIDI_CH3_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #5 0x0000002a95626202 in MPID_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #6 0x0000002a955f7fee in PMPI_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #7 0x0000002a955f7eae in pmpi_finalize_ () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #8 0x0000000000459ff8 in stoprog_ () > #9 0x000000000047afa6 in MAIN__ () > #10 0x0000000000405d62 in main () > > I think the fix is to add some sort of synchronization between > async_thread() and the code that calls the pthread_cancel() on it. To > the > MVAPICH developers: Do you think you can work up a fix soon, and forward > the patch for testing? a short-term workaround would be to disable SRQ at runtime with the appropriate environment settings. We had seen exactly the same backtrace, but so far devel has not found the real cause/fix for it. Just curious - what distro/arch are you using ? cheers. - Christian From panda at cse.ohio-state.edu Sat May 24 13:57:38 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Sat May 24 13:57:53 2008 Subject: [mvapich-discuss] Bug: deadlock between ibv_destroy_srq and async_thread In-Reply-To: <20080524151058.GA5849@daltons.rzg.mpg.de> Message-ID: David - Thanks for reporting this problem. Christian - Thanks for the temporary workaround solution. We are taking a look at it and will keep you updated updated on the status/fix. Thanks, DK On Sat, 24 May 2008, Christian Guggenberger wrote: > On Fri, May 23, 2008 at 09:23:37PM -0500, David_Kewley@dell.com wrote: > > > > #0 0x00000036b2608b3a in pthread_cond_wait@@GLIBC_2.3.2 () from > > /lib64/tls/libpthread.so.0 > > #1 0x0000002a9595405b in ibv_cmd_destroy_srq (srq=0x82b370) at > > src/cmd.c:582 > > #2 0x0000002a962b5419 in mthca_destroy_srq (srq=0x82b3bc) at > > src/verbs.c:475 > > #3 0x0000002a9564878e in MPIDI_CH3I_CM_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #4 0x0000002a955c053b in MPIDI_CH3_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #5 0x0000002a95626202 in MPID_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #6 0x0000002a955f7fee in PMPI_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #7 0x0000002a955f7eae in pmpi_finalize_ () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #8 0x0000000000459ff8 in stoprog_ () > > #9 0x000000000047afa6 in MAIN__ () > > #10 0x0000000000405d62 in main () > > > > I think the fix is to add some sort of synchronization between > > async_thread() and the code that calls the pthread_cancel() on it. To > > the > > MVAPICH developers: Do you think you can work up a fix soon, and forward > > the patch for testing? > > a short-term workaround would be to disable SRQ at runtime with the > appropriate environment settings. We had seen exactly the same backtrace, > but so far devel has not found the real cause/fix for it. Just curious - > what distro/arch are you using ? > > cheers. > - Christian > > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From David_Kewley at Dell.com Sat May 24 14:38:25 2008 From: David_Kewley at Dell.com (David_Kewley@Dell.com) Date: Sat May 24 14:38:42 2008 Subject: [mvapich-discuss] Bug: deadlock between ibv_destroy_srq andasync_thread In-Reply-To: <20080524151058.GA5849@daltons.rzg.mpg.de> References: <20080524151058.GA5849@daltons.rzg.mpg.de> Message-ID: > -----Original Message----- > From: Christian Guggenberger [mailto:christian.guggenberger@rzg.mpg.de] > Sent: Saturday, May 24, 2008 5:11 AM > > On Fri, May 23, 2008 at 09:23:37PM -0500, David_Kewley@dell.com wrote: > > > > #0 0x00000036b2608b3a in pthread_cond_wait@@GLIBC_2.3.2 () from > > /lib64/tls/libpthread.so.0 > > #1 0x0000002a9595405b in ibv_cmd_destroy_srq (srq=0x82b370) at > > src/cmd.c:582 > > #2 0x0000002a962b5419 in mthca_destroy_srq (srq=0x82b3bc) at > > src/verbs.c:475 > > #3 0x0000002a9564878e in MPIDI_CH3I_CM_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #4 0x0000002a955c053b in MPIDI_CH3_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #5 0x0000002a95626202 in MPID_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #6 0x0000002a955f7fee in PMPI_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #7 0x0000002a955f7eae in pmpi_finalize_ () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #8 0x0000000000459ff8 in stoprog_ () > > #9 0x000000000047afa6 in MAIN__ () > > #10 0x0000000000405d62 in main () > > > > I think the fix is to add some sort of synchronization between > > async_thread() and the code that calls the pthread_cancel() on it. To > > the > > MVAPICH developers: Do you think you can work up a fix soon, and forward > > the patch for testing? > > a short-term workaround would be to disable SRQ at runtime with the > appropriate environment settings. We had seen exactly the same backtrace, > but so far devel has not found the real cause/fix for it. Just curious - > what distro/arch are you using ? Thanks for the suggestion! Looking at the manual, I see that's done with mpiexec ... -env MV2_USE_SRQ 0 We're using RHEL4 x86_64 with a recent but locally modified RHEL4 kernel. David From ihara at sun.com Sun May 25 21:46:16 2008 From: ihara at sun.com (Shuichi Ihara) Date: Sun May 25 21:46:36 2008 Subject: [mvapich-discuss] mpif77 error In-Reply-To: <4836C1BD.4060700@sun.com> References: <4836C1BD.4060700@sun.com> Message-ID: <483A1668.7010308@sun.com> Hi, I just tried with "-fPIC" to FFLAGS and CXXFLAGS on 0.9.9 and 1.0, then it worked well. Thanks, -Ihara Shuichi Ihara wrote: > Hi > > I'm attempting to build mvapich-0.9.9(from OFED-1.2.5.5) with PGI-7.1. > As first, the package was built, but we couldn't see mpif77 and mpif90 in this package. > Here is configure's log. It seems "-B" option is not recognized to pgf77 and it can't > be passed, then pgif77 and pgf90 were skipped. > > -snip > checking whether the compiler pgcc runs... yes > checking whether the compiler pgf77 runs... no > Fortran compiler returned non-zero return code > Output from test was > pgf77-Error-Unknown switch: -B > checking for correct handling of conditionals...... yes > -snip > > That's ok. I removed "-B" option from CFLAGS in spec file of mvapich. > After that, I could build mvapich which includes mpif77 and mpif90. > However, when I run mpif77 and mpif90 to compile mpi application, I can see > below errors > > -bash-3.1$ /usr/mpi/pgi/mvapich-0.9.9/bin/mpif77 hello_f77.f > hello_f77.o: In function `MAIN_': > hello_f77.f:(.text+0x1f): undefined reference to `mpi_comm_rank_' > hello_f77.f:(.text+0x31): undefined reference to `mpi_comm_size_' > hello_f77.f:(.text+0xc1): undefined reference to `mpi_finalize_' > > mpicc and mpiCC are no problem. Any ideas? > > Thanks, > > -Ihara > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss From panda at cse.ohio-state.edu Sun May 25 23:06:12 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Sun May 25 23:06:27 2008 Subject: [mvapich-discuss] mpif77 error In-Reply-To: <483A1668.7010308@sun.com> Message-ID: > Hi, > > I just tried with "-fPIC" to FFLAGS and CXXFLAGS on 0.9.9 and 1.0, > then it worked well. Thanks for letting us know that this worked and the problem is resolved. DK > Thanks, > > -Ihara > > Shuichi Ihara wrote: > > Hi > > > > I'm attempting to build mvapich-0.9.9(from OFED-1.2.5.5) with PGI-7.1. > > As first, the package was built, but we couldn't see mpif77 and mpif90 in this package. > > Here is configure's log. It seems "-B" option is not recognized to pgf77 and it can't > > be passed, then pgif77 and pgf90 were skipped. > > > > -snip > > checking whether the compiler pgcc runs... yes > > checking whether the compiler pgf77 runs... no > > Fortran compiler returned non-zero return code > > Output from test was > > pgf77-Error-Unknown switch: -B > > checking for correct handling of conditionals...... yes > > -snip > > > > That's ok. I removed "-B" option from CFLAGS in spec file of mvapich. > > After that, I could build mvapich which includes mpif77 and mpif90. > > However, when I run mpif77 and mpif90 to compile mpi application, I can see > > below errors > > > > -bash-3.1$ /usr/mpi/pgi/mvapich-0.9.9/bin/mpif77 hello_f77.f > > hello_f77.o: In function `MAIN_': > > hello_f77.f:(.text+0x1f): undefined reference to `mpi_comm_rank_' > > hello_f77.f:(.text+0x31): undefined reference to `mpi_comm_size_' > > hello_f77.f:(.text+0xc1): undefined reference to `mpi_finalize_' > > > > mpicc and mpiCC are no problem. Any ideas? > > > > Thanks, > > > > -Ihara > > _______________________________________________ > > mvapich-discuss mailing list > > mvapich-discuss@cse.ohio-state.edu > > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > From forum.san at gmail.com Tue May 27 07:00:03 2008 From: forum.san at gmail.com (Sangamesh B) Date: Tue May 27 07:00:21 2008 Subject: [mvapich-discuss] VAPI_PROTOCOL_R3' failed Message-ID: Hi All, A DL-POLY application job on a 5-node Infiniband setup cluster gave following error: infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion `rndv->protocol == VAPI_PROTOCOL_R3' failed. infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion `rndv->protocol == VAPI_PROTOCOL_R3' failed. infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion `rndv->protocol == VAPI_PROTOCOL_R3' failed. rank 1 in job 20 compute-0-12.local_32785 caused collective abort of all ranks exit status of rank 1: killed by signal 9 The job runs for 20-30 minutes and gives above error. This is with mvapich2 + ofed-1.2.5.5 + Mellanox HCA's. Any idea what might be the wrong? Thanks, Sangamesh -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080527/9d68ec7e/attachment-0001.html From forum.san at gmail.com Wed May 28 03:00:39 2008 From: forum.san at gmail.com (Sangamesh B) Date: Wed May 28 03:00:55 2008 Subject: [mvapich-discuss] Re: VAPI_PROTOCOL_R3' failed In-Reply-To: References: Message-ID: There was no reply from the list. Following is some more info about the DLPOLY + mvapich2 + ofed-1.2.5.5 + Mellanox HCA job, which gets into hang after a certain number of iterations. The same job with mpich2 + ethernet runs fine without any problems. And produces the final result also. With mvapich2, the job runs upto some iterations and stops calculation. It doesn't give any error at this point. But the output file which gets updated at each iteration will not show progress. One more point is, I repeatedly submitted the same mvapich2 job. In each case it stops at same iteration. Any mvapich2 variables have to be set? Thanks, Sangamesh On Tue, May 27, 2008 at 4:30 PM, Sangamesh B wrote: > Hi All, > > A DL-POLY application job on a 5-node Infiniband setup cluster gave > following error: > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > rank 1 in job 20 compute-0-12.local_32785 caused collective abort of all > ranks > exit status of rank 1: killed by signal 9 > > The job runs for 20-30 minutes and gives above error. > > This is with mvapich2 + ofed-1.2.5.5 + Mellanox HCA's. > > Any idea what might be the wrong? > > Thanks, > Sangamesh > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080528/c0010b1c/attachment.html From panda at cse.ohio-state.edu Wed May 28 07:13:23 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Wed May 28 07:13:38 2008 Subject: [mvapich-discuss] Re: VAPI_PROTOCOL_R3' failed In-Reply-To: Message-ID: We are taking a look at this error and will get back to you soon. DK On Wed, 28 May 2008, Sangamesh B wrote: > There was no reply from the list. > > Following is some more info about the DLPOLY + mvapich2 + ofed-1.2.5.5 + > Mellanox HCA job, which gets into hang after a certain number of iterations. > > The same job with mpich2 + ethernet runs fine without any problems. And > produces the final result also. > > With mvapich2, the job runs upto some iterations and stops calculation. It > doesn't give any error at this point. But the output file which gets updated > at each iteration will not show progress. > > One more point is, I repeatedly submitted the same mvapich2 job. In each > case it stops at same iteration. > > Any mvapich2 variables have to be set? > > Thanks, > Sangamesh > > On Tue, May 27, 2008 at 4:30 PM, Sangamesh B wrote: > > > Hi All, > > > > A DL-POLY application job on a 5-node Infiniband setup cluster gave > > following error: > > > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > rank 1 in job 20 compute-0-12.local_32785 caused collective abort of all > > ranks > > exit status of rank 1: killed by signal 9 > > > > The job runs for 20-30 minutes and gives above error. > > > > This is with mvapich2 + ofed-1.2.5.5 + Mellanox HCA's. > > > > Any idea what might be the wrong? > > > > Thanks, > > Sangamesh > > > From huanwei at cse.ohio-state.edu Wed May 28 15:50:36 2008 From: huanwei at cse.ohio-state.edu (wei huang) Date: Wed May 28 15:50:50 2008 Subject: [mvapich-discuss] Re: VAPI_PROTOCOL_R3' failed In-Reply-To: Message-ID: Hi Sangamesh, Would you please let us know more information so that we can look further into this issue? *) Is mvapich2-1.0.2 being used here? *) Are you using the default compiling scripts and default environment variables; *) Is your application using thread at some stage or not? *) Would you please apply the attached patch which prints out some information wrt the assertion failure? *) Would you please try to set the following environment variable (separately) during your run to see if any one of them helps? -env MV2_USE_RDMA_FAST_PATH 0 -env MV2_USE_SRQ 0 -env MV2_USE_COALESCE 0 -env MV2_USE_SHM_COLL 0 Thanks Regards, Wei Huang 774 Dreese Lab, 2015 Neil Ave, Dept. of Computer Science and Engineering Ohio State University OH 43210 Tel: (614)292-8501 On Wed, 28 May 2008, Dhabaleswar Panda wrote: > We are taking a look at this error and will get back to you soon. > > DK > > On Wed, 28 May 2008, Sangamesh B wrote: > > > There was no reply from the list. > > > > Following is some more info about the DLPOLY + mvapich2 + ofed-1.2.5.5 + > > Mellanox HCA job, which gets into hang after a certain number of iterations. > > > > The same job with mpich2 + ethernet runs fine without any problems. And > > produces the final result also. > > > > With mvapich2, the job runs upto some iterations and stops calculation. It > > doesn't give any error at this point. But the output file which gets updated > > at each iteration will not show progress. > > > > One more point is, I repeatedly submitted the same mvapich2 job. In each > > case it stops at same iteration. > > > > Any mvapich2 variables have to be set? > > > > Thanks, > > Sangamesh > > > > On Tue, May 27, 2008 at 4:30 PM, Sangamesh B wrote: > > > > > Hi All, > > > > > > A DL-POLY application job on a 5-node Infiniband setup cluster gave > > > following error: > > > > > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: Assertion > > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > > rank 1 in job 20 compute-0-12.local_32785 caused collective abort of all > > > ranks > > > exit status of rank 1: killed by signal 9 > > > > > > The job runs for 20-30 minutes and gives above error. > > > > > > This is with mvapich2 + ofed-1.2.5.5 + Mellanox HCA's. > > > > > > Any idea what might be the wrong? > > > > > > Thanks, > > > Sangamesh > > > > > > > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > -------------- next part -------------- Index: src/mpid/osu_ch3/channels/mrail/src/rdma/ch3_rndvtransfer.c =================================================================== --- src/mpid/osu_ch3/channels/mrail/src/rdma/ch3_rndvtransfer.c (revision 2532) +++ src/mpid/osu_ch3/channels/mrail/src/rdma/ch3_rndvtransfer.c (working copy) @@ -223,6 +223,15 @@ case VAPI_PROTOCOL_R3: rndv = (cts_pkt == NULL) ? NULL : &cts_pkt->rndv; sreq->mrail.partner_id = cts_pkt->receiver_req_id; + if (rndv->protocol != VAPI_PROTOCOL_R3) { + fprintf(stderr, "Unexpected protocol type %d\n", rndv->protocol); + fprintf(stderr, "Rndv buf %p (alloc %d), size %d, offset %d, " + "r_addr %p, d_entry %p\n", + req->mrail.rndv_buf, req->mrail.rndv_buf_alloc, + req->mrail.rndv_buf_sz, req->mrail.rndv_buf_off, + req->mrail.remote_addr, req->mrail.d_entry); + + } assert(rndv->protocol == VAPI_PROTOCOL_R3); break; case VAPI_PROTOCOL_RGET: From koop at cse.ohio-state.edu Wed May 28 17:35:13 2008 From: koop at cse.ohio-state.edu (Matthew Koop) Date: Wed May 28 17:35:27 2008 Subject: [mvapich-discuss] Bug: deadlock between ibv_destroy_srq and async_thread In-Reply-To: Message-ID: David, Can you try the attached patch and let us know if it solves your problem? This adds some synchronization between the threads. Thanks, Matt On Fri, 23 May 2008 David_Kewley@Dell.com wrote: > I have a user running a 192-way job using MVAPICH2 1.0.1 and OFED > 1.2.5.5, > where MPI_Finalize() does not return. In the two example jobs I've > examined, > 189 processes exited, but the other three hung. The ranks that hung > were > different in the two examples, so I don't think the "3" is significant. > > All processes I've looked at appear to be stuck in the same way. In > normal > running, each process has four threads. When the process gets stuck, > only the > original thread remains. Here is a gdb backtrace from one: > > #0 0x00000036b2608b3a in pthread_cond_wait@@GLIBC_2.3.2 () from > /lib64/tls/libpthread.so.0 > #1 0x0000002a9595405b in ibv_cmd_destroy_srq (srq=0x82b370) at > src/cmd.c:582 > #2 0x0000002a962b5419 in mthca_destroy_srq (srq=0x82b3bc) at > src/verbs.c:475 > #3 0x0000002a9564878e in MPIDI_CH3I_CM_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #4 0x0000002a955c053b in MPIDI_CH3_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #5 0x0000002a95626202 in MPID_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #6 0x0000002a955f7fee in PMPI_Finalize () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #7 0x0000002a955f7eae in pmpi_finalize_ () from > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > #8 0x0000000000459ff8 in stoprog_ () > #9 0x000000000047afa6 in MAIN__ () > #10 0x0000000000405d62 in main () > > After hours of opportunity to study the MVAPICH2 code :), I think I > tracked it > down to lines 1302-1306 in rdma_iba_init.c: > > if (MPIDI_CH3I_RDMA_Process.has_srq) { > pthread_cancel(MPIDI_CH3I_RDMA_Process.async_thread[i]); > pthread_join(MPIDI_CH3I_RDMA_Process.async_thread[i], > NULL); > ibv_destroy_srq(MPIDI_CH3I_RDMA_Process.srq_hndl[i]); > } > > Consider what would happen if async_thread() were processing a > IBV_EVENT_SRQ_LIMIT_REACHED event when pthread_cancel() was called on > async_thread(). async_thread() has already called ibv_get_async_event() > for this event, but it has not yet called ibv_ack_async_event(). The > result would be the observed deadlock in this part of > ibv_cmd_destroy_srq(): > > pthread_mutex_lock(&srq->mutex); > while (srq->events_completed != resp.events_reported) > pthread_cond_wait(&srq->cond, &srq->mutex); > pthread_mutex_unlock(&srq->mutex); > > That is, events_completed == events_reported-1 at this point. The > pthread_cond_signal() would be called, and events_completed could be > made > equal to events_reported, only by by calling ibv_ack_async_event() on > this > event. But that will never happen because async_thread() is the only > code > that would have done that, and it's already been pthread_cancel()'d and > pthread_join()'d before ibv_destroy_srq() is called. > > I think the fix is to add some sort of synchronization between > async_thread() and the code that calls the pthread_cancel() on it. To > the > MVAPICH developers: Do you think you can work up a fix soon, and forward > the patch for testing? > > Thanks, > David > > > David Kewley > Dell Infrastructure Consulting Services > Onsite Engineer at the Maui HPC Center > Cell: 602-460-7617 > David_Kewley@Dell.com > > Dell Services: http://www.dell.com/services/ > How am I doing? Email my manager Russell_Kelly@Dell.com with any > feedback. > > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > -------------- next part -------------- Index: src/mpid/osu_ch3/channels/mrail/src/gen2/ibv_channel_manager.c =================================================================== --- src/mpid/osu_ch3/channels/mrail/src/gen2/ibv_channel_manager.c (revision 2532) +++ src/mpid/osu_ch3/channels/mrail/src/gen2/ibv_channel_manager.c (working copy) @@ -786,6 +786,14 @@ fprintf(stderr, "Error getting event!\n"); } + for(i = 0; i < rdma_num_hcas; i++) { + if(MPIDI_CH3I_RDMA_Process.nic_context[i] == context) { + hca_num = i; + } + } + + pthread_mutex_lock(&MPIDI_CH3I_RDMA_Process.async_mutex_lock[hca_num]); + switch (event.event_type) { /* Fatal */ case IBV_EVENT_CQ_ERR: @@ -837,11 +845,6 @@ pthread_spin_lock(&MPIDI_CH3I_RDMA_Process.srq_post_spin_lock); - for(i = 0; i < rdma_num_hcas; i++) { - if(MPIDI_CH3I_RDMA_Process.nic_context[i] == context) { - hca_num = i; - } - } if(-1 == hca_num) { /* Was not able to find the context, @@ -914,6 +917,7 @@ } ibv_ack_async_event(&event); + pthread_mutex_unlock(&MPIDI_CH3I_RDMA_Process.async_mutex_lock[hca_num]); } } Index: src/mpid/osu_ch3/channels/mrail/src/gen2/rdma_impl.h =================================================================== --- src/mpid/osu_ch3/channels/mrail/src/gen2/rdma_impl.h (revision 2532) +++ src/mpid/osu_ch3/channels/mrail/src/gen2/rdma_impl.h (working copy) @@ -111,6 +111,7 @@ struct ibv_srq *srq_hndl[MAX_NUM_HCAS]; pthread_spinlock_t srq_post_spin_lock; pthread_mutex_t srq_post_mutex_lock[MAX_NUM_HCAS]; + pthread_mutex_t async_mutex_lock[MAX_NUM_HCAS]; pthread_cond_t srq_post_cond[MAX_NUM_HCAS]; uint32_t srq_zero_post_counter[MAX_NUM_HCAS]; pthread_t async_thread[MAX_NUM_HCAS]; Index: src/mpid/osu_ch3/channels/mrail/src/gen2/rdma_iba_init.c =================================================================== --- src/mpid/osu_ch3/channels/mrail/src/gen2/rdma_iba_init.c (revision 2532) +++ src/mpid/osu_ch3/channels/mrail/src/gen2/rdma_iba_init.c (working copy) @@ -126,6 +126,10 @@ rdma_num_rails = rdma_num_hcas * rdma_num_ports * rdma_num_qp_per_port; + for(i = 0; i < rdma_num_hcas; i++) { + pthread_mutex_init(&MPIDI_CH3I_RDMA_Process.async_mutex_lock[i], 0); + } + DEBUG_PRINT("num_qp_per_port %d, num_rails = %d\n", rdma_num_qp_per_port, rdma_num_rails); @@ -776,9 +780,13 @@ if (MPIDI_CH3I_RDMA_Process.has_srq) { pthread_cond_destroy(&MPIDI_CH3I_RDMA_Process.srq_post_cond[i]); pthread_mutex_destroy(&MPIDI_CH3I_RDMA_Process.srq_post_mutex_lock[i]); + + pthread_mutex_lock(&MPIDI_CH3I_RDMA_Process.async_mutex_lock[i]); pthread_cancel(MPIDI_CH3I_RDMA_Process.async_thread[i]); pthread_join(MPIDI_CH3I_RDMA_Process.async_thread[i], NULL); err = ibv_destroy_srq(MPIDI_CH3I_RDMA_Process.srq_hndl[i]); + pthread_mutex_unlock(&MPIDI_CH3I_RDMA_Process.async_mutex_lock[i]); + pthread_mutex_destroy(&MPIDI_CH3I_RDMA_Process.async_mutex_lock[i]); if (err) MPIU_Error_printf("Failed to destroy SRQ (%d)\n", err); } From David_Kewley at Dell.com Wed May 28 17:41:08 2008 From: David_Kewley at Dell.com (David_Kewley@Dell.com) Date: Wed May 28 17:42:02 2008 Subject: [mvapich-discuss] Bug: deadlock between ibv_destroy_srq and async_thread In-Reply-To: References: Message-ID: Matt, Thank you very much. Glancing at the patch, looks good to me. I'll apply and test it ASAP, should be able to report the results tomorrow or Friday. David > -----Original Message----- > From: Matthew Koop [mailto:koop@cse.ohio-state.edu] > Sent: Wednesday, May 28, 2008 11:35 AM > To: Kewley, David > Cc: mvapich-discuss@cse.ohio-state.edu > Subject: Re: [mvapich-discuss] Bug: deadlock between ibv_destroy_srq and > async_thread > > David, > > Can you try the attached patch and let us know if it solves your problem? > This adds some synchronization between the threads. > > Thanks, > > Matt > > On Fri, 23 May 2008 David_Kewley@Dell.com wrote: > > > I have a user running a 192-way job using MVAPICH2 1.0.1 and OFED > > 1.2.5.5, > > where MPI_Finalize() does not return. In the two example jobs I've > > examined, > > 189 processes exited, but the other three hung. The ranks that hung > > were > > different in the two examples, so I don't think the "3" is significant. > > > > All processes I've looked at appear to be stuck in the same way. In > > normal > > running, each process has four threads. When the process gets stuck, > > only the > > original thread remains. Here is a gdb backtrace from one: > > > > #0 0x00000036b2608b3a in pthread_cond_wait@@GLIBC_2.3.2 () from > > /lib64/tls/libpthread.so.0 > > #1 0x0000002a9595405b in ibv_cmd_destroy_srq (srq=0x82b370) at > > src/cmd.c:582 > > #2 0x0000002a962b5419 in mthca_destroy_srq (srq=0x82b3bc) at > > src/verbs.c:475 > > #3 0x0000002a9564878e in MPIDI_CH3I_CM_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #4 0x0000002a955c053b in MPIDI_CH3_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #5 0x0000002a95626202 in MPID_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #6 0x0000002a955f7fee in PMPI_Finalize () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #7 0x0000002a955f7eae in pmpi_finalize_ () from > > /opt/mvapich2/1.0.1/intel/10.1.015/lib/libmpich.so > > #8 0x0000000000459ff8 in stoprog_ () > > #9 0x000000000047afa6 in MAIN__ () > > #10 0x0000000000405d62 in main () > > > > After hours of opportunity to study the MVAPICH2 code :), I think I > > tracked it > > down to lines 1302-1306 in rdma_iba_init.c: > > > > if (MPIDI_CH3I_RDMA_Process.has_srq) { > > pthread_cancel(MPIDI_CH3I_RDMA_Process.async_thread[i]); > > pthread_join(MPIDI_CH3I_RDMA_Process.async_thread[i], > > NULL); > > ibv_destroy_srq(MPIDI_CH3I_RDMA_Process.srq_hndl[i]); > > } > > > > Consider what would happen if async_thread() were processing a > > IBV_EVENT_SRQ_LIMIT_REACHED event when pthread_cancel() was called on > > async_thread(). async_thread() has already called ibv_get_async_event() > > for this event, but it has not yet called ibv_ack_async_event(). The > > result would be the observed deadlock in this part of > > ibv_cmd_destroy_srq(): > > > > pthread_mutex_lock(&srq->mutex); > > while (srq->events_completed != resp.events_reported) > > pthread_cond_wait(&srq->cond, &srq->mutex); > > pthread_mutex_unlock(&srq->mutex); > > > > That is, events_completed == events_reported-1 at this point. The > > pthread_cond_signal() would be called, and events_completed could be > > made > > equal to events_reported, only by by calling ibv_ack_async_event() on > > this > > event. But that will never happen because async_thread() is the only > > code > > that would have done that, and it's already been pthread_cancel()'d and > > pthread_join()'d before ibv_destroy_srq() is called. > > > > I think the fix is to add some sort of synchronization between > > async_thread() and the code that calls the pthread_cancel() on it. To > > the > > MVAPICH developers: Do you think you can work up a fix soon, and forward > > the patch for testing? > > > > Thanks, > > David > > > > > > David Kewley > > Dell Infrastructure Consulting Services > > Onsite Engineer at the Maui HPC Center > > Cell: 602-460-7617 > > David_Kewley@Dell.com > > > > Dell Services: http://www.dell.com/services/ > > How am I doing? Email my manager Russell_Kelly@Dell.com with any > > feedback. > > > > _______________________________________________ > > mvapich-discuss mailing list > > mvapich-discuss@cse.ohio-state.edu > > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > > From chai.15 at osu.edu Thu May 29 13:10:50 2008 From: chai.15 at osu.edu (LEI CHAI) Date: Thu May 29 13:11:07 2008 Subject: [mvapich-discuss] Re: problem run mvapich1.0 on AMD quad core cluster In-Reply-To: References: Message-ID: For everybody's information, the problem was solved off-line. It was due to IP setup issues on the machines. I'm attaching the latest correspondance below: On Thu, 29 May 2008, Lei Chai wrote: > Glad to know mvapich works for you with this workaround. I cannot think of > any places that this change will break. I think it should work fine. > > Lei > > > On Thu, 29 May 2008 Terrence.LIAO@total.com wrote: > > > Hi, Lei, > > > > I have one final question on get_host_id().??? Problem we have come from > > hostname "nod200" for example on our compute nodes has local IP according > > to /etc/hosts file.? However, the IB interface has name "nib200", since > > "nib200" is not listed on /etc/hosts, it get the IP from NIS.? I have > > modified the get_host_id() by doing? myhostname[1]='i'; myhostname[2]='b', > > to convert myhostname from nodXXX to nibXXX,? since gethostname() return > > nodXXX.? With this small change, the mpi-code can run.? My question is > > this modification break mvapich in some other location? > > > > Thank you very much. > > > > > > P.S. We keep away from /etc/hosts in order to avoid manually editing it > > which might introduce potential human error into the /etc/hosts. > > > > -- Terrence > > -------------------------------------------------------- > > Terrence Liao, Ph.D. > > Research Computer Scientist > > TOTAL E&P RESEARCH & TECHNOLOGY USA, LLC > > 1201 Louisiana, Suite 1800, Houston, TX 77002 > > Tel: 713.647.3498? Fax: 713.647.3638 > > Email: terrence.liao@total.com ----- Original Message ----- From: Terrence.LIAO@total.com Date: Friday, May 23, 2008 8:22 am Subject: [mvapich-discuss] Re: problem run mvapich1.0 on AMD quad core cluster To: Terrence.LIAO@total.com Cc: mvapich-discuss@cse.ohio-state.edu > Hi, mvapich-discuss > I find a work around on my mvapich problem. ?The build with ?-D_SMP_ and -D_SMP_RNDV flags removed from make.mvapich.gen2 works on my cluster. ? > Thank you very much. > -- Terrence > -------------------------------------------------------- > Terrence Liao, Ph.D. > Research Computer Scientist > TOTAL E&P RESEARCH & TECHNOLOGY USA, LLC > 1201 Louisiana, Suite 1800, Houston, TX 77002 > Tel: 713.647.3498 ?Fax: 713.647.3638 > Email: terrence.liao@total.com > Terrence LIAO/HOU/US/EP/Corp 05/21/2008 03:34 PM > To > mvapich-discuss@cse.ohio-state.edu > cc > Terrence LIAO/HOU/US/EP/Corp@E&P > Subject > problem run mvapich1.0 ?on AMD quad core clusterLink > Hi, mvapich-discuss, > We have a AMD quad core cluster with RHEL 4.5, OFED 1.2.5 with PGI compiler. ?I build mvapich1.0 as usually, but mpi hello world hung at MPI_INIT(). ?I can run with OpenMPI build. ?Also, the ib_write_bw and ibv_rc_pingpong ran fine. ?Any clue where I did wrong? > Thank you very much. > -- Terrence > -------------------------------------------------------- > Terrence Liao, Ph.D. > Research Computer Scientist > TOTAL E&P RESEARCH & TECHNOLOGY USA, LLC > 1201 Louisiana, Suite 1800, Houston, TX 77002 > Tel: 713.647.3498 ?Fax: 713.647.3638 > Email: terrence.liao@total.com > _______________________________________________ > mvapich-discuss mailing list > mvapich-discuss@cse.ohio-state.edu > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080529/93df6376/attachment.html From David_Kewley at Dell.com Thu May 29 23:06:43 2008 From: David_Kewley at Dell.com (David_Kewley@Dell.com) Date: Thu May 29 23:07:37 2008 Subject: [mvapich-discuss] Followup: mvapich2 issue regarding mpd timeout in mpiexec Message-ID: This is a followup to this thread: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2007-May/000834 .html between Greg Bauer and Qi Gao. We had the same problem that Greg saw -- failure of mpiexec, with the characteristic error message "no msg recvd from mpd when expecting ack of request". It was resolved for us by setting recvTimeout in mpiexec.py to a higher value, just as Greg suggested and Qi concurred. The default value is 20; we chose 200 (we did not experiment with values between these two, so lower may work in many cases). I think this change should be made permanent in MVAPICH2. I do not think it will negatively impact anyone, because in the four cases where this timeout is used, if the timeout expires mpiexec immediately makes an error exit anyway. So the worst consequence is that mpiexec would take longer to fail (3 minutes longer if 200 is used instead of 20). The user who encounters this timeout has to fix the root cause of the timeout in order to get any work done, so they are not likely to encounter it repeatedly and thereby lose lots of runtime simply because the timeout is large. Is this analysis correct? Meanwhile, this change would clearly help at least some people with large clusters. We see failure with the default recvTimeout between 900 and 1000 processes; larger recvTimeout allows us to scale to 3000 processes and beyond. The default setting does not cause failure if I make a simple, direct call to mpiexec. I only see it when I use mpirun.lsf to launch a large job. I think the failure in the LSF case is due to the longer time it presumably takes to launch LSF's TaskStarter for every process, etc. The time required seems to be O(#processes) in the LSF case. (We have LSF 6.2, with a local custom wrapper script for TaskStarter). If you agree that this change to the value of recvTimeout is OK, please implement this one-line change in MVAPICH2, and consider contributing it upstream to MPICH2 as well. If you decline to make this change, at least it's now on the web that this change does fix the problem. :) Thanks, David David Kewley Dell Infrastructure Consulting Services Onsite Engineer at the Maui HPC Center Cell: 602-460-7617 David_Kewley@Dell.com Dell Services: http://www.dell.com/services/ How am I doing? Email my manager Russell_Kelly@Dell.com with any feedback. From forum.san at gmail.com Fri May 30 08:41:00 2008 From: forum.san at gmail.com (Sangamesh B) Date: Fri May 30 08:41:20 2008 Subject: [mvapich-discuss] Re: VAPI_PROTOCOL_R3' failed In-Reply-To: References: Message-ID: Hi, My answers are inline: On Thu, May 29, 2008 at 1:20 AM, wei huang wrote: > Hi Sangamesh, > > Would you please let us know more information so that we can look further > into this issue? > > *) Is mvapich2-1.0.2 being used here? > [root@compute-0-8 mvapich2-1.0.2]# /opt/mvapich2/bin/mpich2version Version: mvapich2-1.0 Device: osu_ch3:mrail Configure Options: '--prefix=/opt/mvapich2' '--with-device=osu_ch3:mrail' '--with-rdma=gen2' '--with-pm=mpd' '--disable-romio' '--without-mpe' 'CC=gcc' 'CFLAGS=-D_X86_64_ -D_SMP_ -DUSE_HEADER_CACHING -DONE_SIDED -DMPIDI_CH3_CHANNEL_RNDV -DMPID_USE_SEQUENCE_NUMBERS -DRDMA_CM -I/usr//include -O2' 'CXX=g++' 'F77=gfortran' 'F90=gfortran' 'FFLAGS=-L/usr//lib64' CC: gcc -D_X86_64_ -D_SMP_ -DUSE_HEADER_CACHING -DONE_SIDED -DMPIDI_CH3_CHANNEL_RNDV -DMPID_USE_SEQUENCE_NUMBERS -DRDMA_CM -I/usr//include -O2 CXX: g++ F77: gfortran -L/usr//lib64 F90: gfortran > *) Are you using the default compiling scripts and default environment > variables; > Yes. We built mvapich2 with make.mvapich2.ofa script + GNU (gcc, gfortran) compilers + ofed-1.2.5.5. > > *) Is your application using thread at some stage or not? > *) Would you please apply the attached patch which prints out some > information wrt the assertion failure? > After patch: [root@compute-0-8 mvapich2-1.0.2]# patch -p0 protocol == VAPI_PROTOCOL_R3' failed. Rndv buf (nil) (alloc 0), size 13248, offset 0, r_addr (nil), d_entry (nil) DLPOLY_NEW: ch3_rndvtransfer.c:235: MPIDI_CH3_Rndv_transfer: Assertion `rndv->protocol == VAPI_PROTOCOL_R3' failed. rank 1 in job 1 compute-0-8.local_36493 caused collective abort of all ranks exit status of rank 1: killed by signal 9 *) Would you please try to set the following environment variable > (separately) during your run to see if any one of them helps? > > -env MV2_USE_RDMA_FAST_PATH 0 > -env MV2_USE_SRQ 0 > -env MV2_USE_COALESCE 0 > -env MV2_USE_SHM_COLL 0 > Tried these, before applying patch: No success When the job run on one node, it progressed upto 65000 and stopped. At this stage, I used gdb to debug. The gdb output may give some clue: [bhaskar@compute-0-12 test_NEW]$ gdb infin_check 25063 GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-unknown-linux-gnu"... infin_check: No such file or directory. Attaching to process 25063 Reading symbols from /home/bhaskar/dl_poly_2.18/execute/infin_check...(no debugging symbols found)...done. Reading symbols from /lib64/tls/libpthread.so.0...(no debugging symbols found)...done. [Thread debugging using libthread_db enabled] [New Thread 0x2a95ae74e0 (LWP 25063)] [New Thread 0x40200960 (LWP 25067)] Loaded symbols for /lib64/tls/libpthread.so.0 Reading symbols from /usr/lib64/librdmacm.so.1...done. Loaded symbols for /usr/lib64/librdmacm.so.1 Reading symbols from /usr/lib64/libibverbs.so.1...done. Loaded symbols for /usr/lib64/libibverbs.so.1 Reading symbols from /usr/lib64/libibumad.so.1...done. Loaded symbols for /usr/lib64/libibumad.so.1 Reading symbols from /usr/lib64/libgfortran.so.0...done. Loaded symbols for /usr/lib64/libgfortran.so.0 Reading symbols from /lib64/tls/libm.so.6...done. Loaded symbols for /lib64/tls/libm.so.6 Reading symbols from /lib64/libgcc_s.so.1...done. Loaded symbols for /lib64/libgcc_s.so.1 Reading symbols from /lib64/tls/libc.so.6...done. Loaded symbols for /lib64/tls/libc.so.6 Reading symbols from /lib64/libdl.so.2...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/ld-linux-x86-64.so.2...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib64/libibcommon.so.1...done. Loaded symbols for /usr/lib64/libibcommon.so.1 Reading symbols from /usr/lib64/libcxgb3-rdmav2.so...done. Loaded symbols for /usr/lib64/libcxgb3-rdmav2.so Reading symbols from /usr/lib64/libmthca-rdmav2.so...done. Loaded symbols for /usr/lib64/libmthca-rdmav2.so Reading symbols from /usr/lib64/libipathverbs-rdmav2.so...done. Loaded symbols for /usr/lib64/libipathverbs-rdmav2.so Reading symbols from /usr/lib64/libmlx4-rdmav2.so...done. Loaded symbols for /usr/lib64/libmlx4-rdmav2.so Reading symbols from /lib64/libnss_files.so.2...done. Loaded symbols for /lib64/libnss_files.so.2 0x00000000005aafd8 in MPIDI_CH3I_SMP_pull_header () (gdb) bt #0 0x00000000005aafd8 in MPIDI_CH3I_SMP_pull_header () #1 0x00000000005abe6e in MPIDI_CH3I_SMP_read_progress () #2 0x00000000005a8370 in MPIDI_CH3I_Progress () #3 0x0000000000597bd1 in MPIC_Wait () #4 0x0000000000598245 in MPIC_Recv () #5 0x0000000000593c55 in MPIR_Bcast () #6 0x000000000059432e in PMPI_Bcast () #7 0x0000000000592797 in PMPI_Allreduce () #8 0x000000000059073a in pmpi_allreduce_ () #9 0x000000000058ac96 in gdsum_ () #10 0x0000000000418b98 in __utility_module__global_sum_forces () #11 0x000000000047f1d6 in __forces_module__force_manager () #12 0x000000000058a7fd in __driver_module__molecular_dynamics () #13 0x000000000058dfb1 in MAIN__ () #14 0x00000000005ecd9e in main () (gdb) (gdb) frame #0 0x00000000005aafd8 in MPIDI_CH3I_SMP_pull_header () (gdb) frame 1 #1 0x00000000005abe6e in MPIDI_CH3I_SMP_read_progress () (gdb) frame 2 #2 0x00000000005a8370 in MPIDI_CH3I_Progress () (gdb) frame 3 #3 0x0000000000597bd1 in MPIC_Wait () (gdb) frame 4 #4 0x0000000000598245 in MPIC_Recv () (gdb) frame 5 #5 0x0000000000593c55 in MPIR_Bcast () (gdb) frame 6 #6 0x000000000059432e in PMPI_Bcast () (gdb) frame 7 #7 0x0000000000592797 in PMPI_Allreduce () (gdb) frame 8 #8 0x000000000059073a in pmpi_allreduce_ () (gdb) frame 9 #9 0x000000000058ac96 in gdsum_ () (gdb) frame 10 #10 0x0000000000418b98 in __utility_module__global_sum_forces () (gdb) frame 11 #11 0x000000000047f1d6 in __forces_module__force_manager () (gdb) frame 12 #12 0x000000000058a7fd in __driver_module__molecular_dynamics () (gdb) frame 13 #13 0x000000000058dfb1 in MAIN__ () (gdb) frame 14 #14 0x00000000005ecd9e in main () I hope this info helps you. Please let me know if any gdb tests are to be done. > Thanks > > Regards, > Wei Huang > > 774 Dreese Lab, 2015 Neil Ave, > Dept. of Computer Science and Engineering > Ohio State University > OH 43210 > Tel: (614)292-8501 > > > On Wed, 28 May 2008, Dhabaleswar Panda wrote: > > > We are taking a look at this error and will get back to you soon. > > > > DK > > > > On Wed, 28 May 2008, Sangamesh B wrote: > > > > > There was no reply from the list. > > > > > > Following is some more info about the DLPOLY + mvapich2 + ofed-1.2.5.5 > + > > > Mellanox HCA job, which gets into hang after a certain number of > iterations. > > > > > > The same job with mpich2 + ethernet runs fine without any problems. And > > > produces the final result also. > > > > > > With mvapich2, the job runs upto some iterations and stops calculation. > It > > > doesn't give any error at this point. But the output file which gets > updated > > > at each iteration will not show progress. > > > > > > One more point is, I repeatedly submitted the same mvapich2 job. In > each > > > case it stops at same iteration. > > > > > > Any mvapich2 variables have to be set? > > > > > > Thanks, > > > Sangamesh > > > > > > On Tue, May 27, 2008 at 4:30 PM, Sangamesh B > wrote: > > > > > > > Hi All, > > > > > > > > A DL-POLY application job on a 5-node Infiniband setup cluster > gave > > > > following error: > > > > > > > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: > Assertion > > > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: > Assertion > > > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > > > infin_check: ch3_rndvtransfer.c:226: MPIDI_CH3_Rndv_transfer: > Assertion > > > > `rndv->protocol == VAPI_PROTOCOL_R3' failed. > > > > rank 1 in job 20 compute-0-12.local_32785 caused collective abort > of all > > > > ranks > > > > exit status of rank 1: killed by signal 9 > > > > > > > > The job runs for 20-30 minutes and gives above error. > > > > > > > > This is with mvapich2 + ofed-1.2.5.5 + Mellanox HCA's. > > > > > > > > Any idea what might be the wrong? > > > > > > > > Thanks, > > > > Sangamesh > > > > > > > > > > > _______________________________________________ > > mvapich-discuss mailing list > > mvapich-discuss@cse.ohio-state.edu > > http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20080530/0a147d57/attachment-0001.html From panda at cse.ohio-state.edu Fri May 30 16:36:10 2008 From: panda at cse.ohio-state.edu (Dhabaleswar Panda) Date: Fri May 30 16:36:28 2008 Subject: [mvapich-discuss] Announcing the release of MVAPICH 1.0.1 Message-ID: The MVAPICH team is pleased to announce the release of MVAPICH 1.0.1 version. This is a bug-fix release. Specific changes (compared to MVAPICH 1.0.0) are as follows: - Changed "inline" functions to "static inline" to enable Sun Studio 12 compiler. - Update configure to avoid silently falling back to gcc if chosen compiler doesn't function properly. Contributed by Pavel Shamis from Mellanox. - Fix race condition in on-demand startup where viadev.connections could be used before being allocated. Thanks to John Hawkes from Penguin Computing for pointing this out. - Additional cleanup of processes on MPI_Abort - Check for mpispawn in the same directory as mpirun_rsh - Fix for the BLACS segmentation fault problem. Problem occurs when application holds onto MPI communicators not freeing immediately. Thanks to Byoung-Do Kim from TACC for pointing this out. - Don't abort on IBV_EVENT_QP_LAST_WQE_REACHED, instead wait for ibv_poll_cq to show the real error. Thanks to Pavel Shamis from Mellanox for this patch. Detailed CHANGELOG for MVAPICH versions can be obtained by visiting the following URL: http://mvapich.cse.ohio-state.edu/download/mvapich/changes.shtml We strongly encourage MVAPICH users to update their installations to this latest version. For downloading MVAPICH 1.0.1, associated user guide and accessing the SVN, please visit the following URL: http://mvapich.cse.ohio-state.edu This version is also being made available through OFED 1.3.1. All feedbacks, including bug reports and hints for performance tuning, patches and enhancements are welcome. Please post it to the mvapich-discuss mailing list. Thanks, The MVAPICH Team