P2P non-blocking

1.非ブロック通信
(1)非ブロックsender
(a)ほとんどすぐに戻って他のことをすることができます.アプリケーションbufferからsystem bufferにデータがあるかどうかにかかわらず、receiver側のアプリケーションbufferまたはsystem bufferにデータがあるかどうかにかかわらず.
(b)送信動作は、MPI libが適切な時間を選択して行う.
(c)senderは、送信したばかりのメモリユニット(send buffer)をすぐに変更しないほうがいいです(安全ではありません!)、ただし、他のメモリユニットは操作できます.
(d)データを送信したばかりのメモリユニット(send buffer)を操作するには、*wait*関数で送信完了を確認する必要があります.
(e)非ブロック送信は、通信とのオーバーラップ(overlap computation with communication and exploit possible performance gains)の計算を実現することができる.
(2)非ブロックreceiver(原理と非ブロックsender)
2.関数
MPI_Isend (&buf,count,datatype,dest,tag,comm,&request)
Identifies an area in memory to serve as a send buffer. Processing continues immediately without waiting for the message to be copied out from the application buffer. A communication request handle is returned for handling the pending message status. The program should not modify the application buffer until subsequent calls to MPI_Wait or MPI_Test indicate that the non-blocking send has completed. MPI_Irecv (&buf,count,datatype,source,tag,comm,&request)
Identifies an area in memory to serve as a receive buffer. Processing continues immediately without actually waiting for the message to be received and copied into the the application buffer. A communication request handle is returned for handling the pending message status. The program must use calls to MPI_Wait or MPI_Test to determine when the non-blocking receive operation completes and the requested message is available in the application buffer.
3.例

#include"mpi.h"
#include<stdio.h>

int main(int argc, char *argv[]){
        int totalNumTasks, rankID;

        MPI_Init(&argc, &argv);
        MPI_Comm_size(MPI_COMM_WORLD, &totalNumTasks);
        MPI_Comm_rank(MPI_COMM_WORLD, &rankID);
        //get the host where this process is running
        int  nameLength;
        char processor_name[MPI_MAX_PROCESSOR_NAME];
        MPI_Get_processor_name(processor_name,&nameLength);

        int prevRankID = rankID - 1;
        int nextRankID = rankID + 1;
        if(rankID == 0)  prevRankID = totalNumTasks - 1;
        if(rankID == (totalNumTasks - 1)) nextRankID = 0;

        int count = 1;
        MPI_Request request[4];

        char recvBuf1;
        char sendBuf1 = 'R';
        int tag1 = 1;
        MPI_Irecv(&recvBuf1, count, MPI_CHAR, prevRankID, tag1, MPI_COMM_WORLD, &request[0]);
        MPI_Isend(&sendBuf1, count, MPI_CHAR, nextRankID, tag1, MPI_COMM_WORLD, &request[1]);

        char recvBuf2;
        char sendBuf2 = 'L';
        int tag2 = 2;
        MPI_Irecv(&recvBuf2, count, MPI_CHAR, nextRankID, tag2, MPI_COMM_WORLD, &request[2]);
        MPI_Isend(&sendBuf2, count, MPI_CHAR, prevRankID, tag2, MPI_COMM_WORLD, &request[3]);
        //after, non-blocking send and receive, process can do something except modifying the application buffer
        //Here, application buffer is recvBuf1, sendBuf1, recvBuf2, sendBuf2
        //which can overlap the communication and computing 
        //Indeed, you can use other memory areas
        printf("My rankID = %d on Processor = %s, I can do something here.....
", rankID, processor_name);
        //Now to check after MPI_Waitall, after it, the application buffer is safe to reuse 
        MPI_Status status[4];
        MPI_Waitall(4, request, status);
        printf("My rankID = %d, recvBuf1 = %c && source = %d && tag = %d
", 
                rankID, recvBuf1, status[0].MPI_SOURCE, status[0].MPI_TAG);

        printf("My rankID = %d, recvBuf2 = %c && source = %d && tag = %d
", 
                rankID, recvBuf2, status[2].MPI_SOURCE, status[2].MPI_TAG);
        printf("My rankID = %d, Now, my application buffer is safe to reuse.
", rankID);
        MPI_Finalize();
        return 0;
}

4.コンパイル実行

[amao@amao991 mpi-study]$ mpicc p2pNonBlockingOnWhichProcessor.c 
[amao@amao991 mpi-study]$ mpiexec -n 3 -f machinefile ./a.out 
My rankID = 0 on Processor = amao991, I can do something here.....
My rankID = 2 on Processor = amao992, I can do something here.....
My rankID = 1 on Processor = amao991, I can do something here.....
My rankID = 0, recvBuf1 = R && source = 2 && tag = 1
My rankID = 0, recvBuf2 = L && source = 1 && tag = 2
My rankID = 0, Now, my application buffer is safe to reuse.
My rankID = 1, recvBuf1 = R && source = 0 && tag = 1
My rankID = 1, recvBuf2 = L && source = 2 && tag = 2
My rankID = 1, Now, my application buffer is safe to reuse.
My rankID = 2, recvBuf1 = R && source = 1 && tag = 1
My rankID = 2, recvBuf2 = L && source = 0 && tag = 2
My rankID = 2, Now, my application buffer is safe to reuse.

5.まとめ
(1)本例では3つのプロセスが双方向ループを構成し,各プロセスがメッセージを受信すると,手を回して送信する.
(2)non-blocking送信/受信を採用しているため、関数呼び出しが完了した後、プロセスはbufferの受信/送信を操作しない限り、他の文を実行し続けることができる(ここではprintf文を例に挙げる).
(3)最後まで受信bufferを表示するにはMPI_を呼び出すWaitallは承認が完了したことを確認します.