PVM3.4.4 + IPv6 : Full grid connectivity ======================================== * Rafael Martinez Torres ( rafael dot martinez at novagnet dot com ) - nGn as a partner of Euro6IX ( http://www.euro6ix.org) - http://www.ngn.euro6ix.org/IPv6 , http://www.novagnet.com ** Acknowledges to MIT , for releasing its source code. All copyrights belongs to their respective owners. This readme file describes the IPv6 (ANY platform) port of PVM. 0. - PURPOSE: 1. - CHANGELOG: 2. - INSTALLATION NOTES: 3. - RUNNING YOUR PVM-GRID APPLICATIONS ON IPv6: 4. - KNOWN BUGS. 5. - TODO 6. - OPEN QUESTIONS: 7. - SUPPORT: 8. - DISCLAIMER: 0. - PURPOSE: ------------- Some Beowulf clusters are designed so that not all nodes are visible to the external world. This is commonly known as the NAT problem (Network Address Translation) and it is inherently bound to the IPv4 protocol address space of 32 bits. In these cases, there is often a "front end" node which connects to the world, and the rest of the nodes are on an internal network. This has the disadvantage for PVM users running on an outside computer who want to add the Beowulf computer to their configuration: the PVM code on their computer will be unable to communicate with the Beowulf nodes on the internal network. Some time ago, the BEOLIN port of PVM was introduced in order to solve this problem by making the parallelism of the Beowulf computer transparent to the user. This is done in a way similar to the MPP ports of PVM for parallel computers such as the IBM SP2, Cray T3D and others. Today we present a new way to solve this problem based on the new networking protocol: IPv6 (see RFC-2460 for more detailed info). Among new features, address space is virtually increased into "infinite" (128 bits vs. 32 bits). This will eventually enable not only computers, but even cellulars and any sort of electronic devices to connect the Internet. The NAT patch will become use-less, and GRID/P2P(peer-to-peer) applications are thought to be among the main beneficiaries of this new technology. Summing up: IPv6 provides the GRID with "mobile computing", as a complement to "mobile computation" supplid by PVM. Former refers to the fact of nodes of a network mobing about, while latter higlights the notion that running programs need no to be forever tied to a single network node. 1. - CHANGELOG --------------- 23042005: - Highly steady version. Fixing some previous bugs on hex-decimal conversion of ports. 21042005: - A new system of code porting based on scripts is used, making the patch more robust. First: - Fixed the iflist() command: Now you can use the pvmhostfile as usual. - The pvmd-localtask daemon: Initially this daemon was run on a UNIX socket. Then, due to portability problems acrross platforms, it was implemented with a TCP/IP socket on loopbak interface (127.0.0.1). It has been fixed to run on IPv6 loopbak interface (::1) 2.- INSTALLATION NOTES: ------------------- Ordinary PVM deployment skill required. Please, refer to corresponding info (Readme, under pvm3 dir) 0) Prerequisites: - A UNIX O.S implementing IPv6 stack. Usually LINUX kernels >= 2.4 do implement it. Also Freebsd >= 5.1, Solaris >= 8... - Obviously, IPv6 connectivity at your beowulf. If you are in trouble respect to this, try to setup a tunnel "6to4" by reading some correponding bibliography, out of the scope of this document. 1) Download http://www.ngn.euro6ix.org/IPv6/pvm/pvm-3.4.4-v6-23042005.diff http://www.netlib.org/pvm3/pvm3.4.4.tar.gz 2) Unroll the tar.gz and apply the patch. $bash-2.04$ tar xzvf pvm3.tar.gz $bash-2.04$ patch -p0 < pvm-3.4.4-v6-21032005.diff 3) Compile the bundle the next way: Be carefull of using SSH protocol $bash-2.04$ export PVM_RSH=`which ssh` $bash-2.04$ cd pvm3 $bash-2.04$ export PVM_ROOT=`pwd` $bash-2.04$ make CFLOPTS="-DINET6" WARNING: - If you don't use GNU make you may be on trouble when aplying the flag CFLOPTS=-DINET6 . If so, try to adjust manually the file pvm3/conf/.def - As far as I know, (2005-04-31) no official UNIX distro include IPv6-support for NFS and NIS. This means you cannot deploy the Beowulf Cluster on a distributed file system on IPv6 (at least for NFS, maybe others...). The tests have been run by using the ordinary IPv4 NFS ones, I mean, our benchmark intranet could run both IPv4 and IPv6. Obviously, PVM UDP daemons were running IPv6. As a matter of abstraction layer provided by the file system, no problems are expected when these protocols NFS/NIS become IPv6 available... - If you cannot export the NFS outside your intranetwork, you can always adjust values on pvmhostfile in order to indicate default search paths... 3.- RUNNING YOUR PVM-GRID APPLICATIONS ON IPv6: ----------------------------------------------------- Until now, it has been tested only on a i386-[linux-freebsd] Beowulf. More precisely, SuSE 9.2 and FreeBsd 5.1 , but it still not very matured DNS was introduced to make easier the "decimal-dot notation" 154.346.34.145 . IPv6 will make it even harder, with addresses like 2005:35a4:12ff:6730:a423::12. You are encouraged to use always DNS names. If registering DNS entries is not always possible on a benchmark beowulf, just try to write them on the local hosts file (/etc/hosts on linux), as was the use prior to DNS system. For a simple test: 1) bash-2.04$ cd pvm3 bash-2.04$ make e This will compile the examples provided with the tar.gz bundle . 2) bash-2.04$ export PVM_ROOT="/path/to/ipv6/pvm/patch" bash-2.04$ alias pvm='$PVM_ROOT/lib/$PVM_ARCH/pvm' bash-2.04$ export PVM_RSH=`which ssh` # Very important !! bash-2.04$ pvm Take a look into /tmp/pvml.: If you see a line with some dotted address like: LINUX [2004:2a04:800:ee10:3434::2] this means you are running PVM on IPv6 !!! Also type bash-2.04$netstat -uan to check your pvmd, pvmd's daemons running... 3) pvm> add pc2.ipv6lab.inv user@pc2.ipv6lab.inv's password: (type whatever corresponds. To do this automatically, you can enable an ssh-agent. Read original README to know how to) pvm> conf pvm> spawn -5 -> hello You should watch alternate "hello worlds" from the virtual nodes... Take a look into /tmp/pvml.uid... Porting your personal applications. ==================================== As old PVM used to, the new PVM frees you from dealing with low-level aspects like networking connectivity and transport. So,YOU DO NOT HAVE TO MODIFY your original application source code, BUT YOU HAVE TO RE-COMPILE AND LINK IT against the new library you get in the previous phase, namely libpvm.a . Be carefull of exporting the rigth environment PATH: bash-2.04$ export PVM_ROOT="/path/to/ipv6/pvm/patch" 4.- KNOWN BUGS: (Important !!!) ----------------------------- 5.- ALERTS: ------------ - Be sure your firewalls allow free access to ports used by PVM daemons. - Please, use SSH protocol to make the master setup the slaves pvmd daemon. You can do this by overwriting the environment variable PVM_RSH. bash-2.04$ export PVM_RSH=`which ssh` 6.- TODO: ----------- - Benchmarks... Exhaustive benchmarks are needed. 7.- OPEN QUESTIONS: --------------- Those interested on this section are kindly required to read the more detailed report: - http://www.gmonline.demon.co.uk/cscene/CS6/CS6-03.htm and contributing with any sugestion by e-mail to (rafael dot martinez at novagnet dot com) * should PVM support dual-stack protocol (IPv6-IPv4)? In my opinion, no. Tratiditional client/server models based on asymetric processing fit entirely this concept, as explained on the pointed URL, but PVM configuration scheme (globally kept in host table) may be hard to maintain by considering both types of connectivity. Consider the following scenario: at least, the MASTER node is provided dual-stack node. If a IPv6 node is added to the host-table, then IPv4-only nodes will not reach it, becoming likely to be dropped. If a IPv4-only node is entered, then IPv4-NAT we want to avoid will be present again. Thus, would derive on a desychronized host-table among the whole Bewoulf. If you find problems with IP(v6-v4) connectivity, it would be cleaner trying to setup tunnel "6to4" instead of NAT patch. Search for bibliography on how to deploy it. 8.- SUPPORT: --------- Suggestions, contributions are welcome at (rafael dot martinez at novagnet dot com) 9.- DISCLAIMER: ----------- I don't know why the following has to be in all caps, but you know lawyers. THIS IS A DEMONSTRATION RELEASE. THE AUTHOR MAKES NO REPRESENTATION ABOUT THE SUITABILITY OF THIS SOFTWARE FOR ANY OTHER PURPOSE. IT IS PROVIDED "AS IS" WITHOUT EXPRESSED OR IMPLIED WARRANTY. THIS NOTICE MUST BE MAINTAINED IN ALL DISTRIBUTED VERSIONS.