Intel Xeon Phi

Preliminary work has begun on a version of the SIMION binaries compiled for the Intel® Xeon Phi™ coprocessor. This is available experimentally with SIMION 8.2 but it not part of 8.2.

WARNNIG: The following notes are preliminary (April 2014). The Phi binaries are largely untested and it not confirmed yet what if any performance improvement is attained.

The Phi is coprocessor card with around 60 cores and some gigabytes of RAM. It runs a small Linux OS. Jobs can be submitted to it from your main OS (Windows or Linux). The Phi is in some ways an alternative to nVidia CUDA. For background on the Phi, see Wikipedia:Xeon_Phi.

Usage Modes

There are a couple ways software may be run on the Phi:

  1. Use a “native” Phi binary that runs entirely within the Phi.
  2. use a binary that runs on the main OS (Windows or Linux) but “offloads” works to the Phi card.

Presently, SIMION (and SL Tools) non-GUI binaries have been compiled in the manner of #1. GUI binaries for Windows in the manner #2 could be added in the near future without much effort to provide a more seamless Windows GUI experience.

Native Binary Usage

TODO: This is not yet tested.

The usage model would be as follows. This procedure assumes you have some experience with SSH and Linux.

  • Download the SIMION Phi binaries, which are presently in the latest “simion-xeonphi-*.zip” file available through “Check for Updates” on the SIMION main screen. These will be formally incorporated into SIMION 8.2 but until then are provided to 8.1 users for beta testing on a time limited basis.

  • Obtain an SSH shell (included with Linux or using Putty or Cygwin if on Windows).

  • With SSH, create a directory for SIMION files on the Phi. Copy the SIMION Phi binaries (simion and sltools), your simion.key file, and any Intel library dependencies (TODO: libiomp5.so?) to that folder:

    $ ssh mic0 'mkdir /tmp/simion'
$ scp simion sltools simion.key libiomp5.so mic0:/tmp/simion

  • The Phi’s file system is a RAM disk, so any any files written to that disk will eat away from the RAM available to the Phi. Therefore you may want to also mount a folder from the host system in the Phi OS to read and write to files instead in the host system. To do this, see Intel’s documentation on “exportfs”. Or just use your RAM disk for now and copy your simulation files to Phi as well.

  • Run the SIMION binary on the Phi from ssh:

    $ ssh mic0
$ cd /tmp/simion
$ chmod +x simion sltools
$ export LD_LIBRARY_PATH=/tmp/simion:$LD_LIBRARY_PATH
$ ./simion --nogui fly einzel.iob

There may be other ways to bootstrap this (e.g. micnativeloadex).

The SIMION interface for controlling threads is similar in the Phi to that used in traditional Xeon cores (Multicore CPU support). For example, the SIMION --num-threads command line option can be used to control the number of threads used. You may want to experiment to determine the number of threads that gives the optimal performance (e.g. 1, 2, 4, 8, …, 60, 120, 240). Check the number of cores and threads on your Phi card. More threads won’t necessarily give more performance. Some environment variables like KMP_AFFINITY can also affect the assignment of threads to cores: Intel: OpenMP Thread Affinity Control.

Dependencies

TODO: You may need to copy some Intel libraries like “libiomp5.so” to the Phi. See Intel: building-a-native-application-for-intel-xeon-phi-coprocessors and Intel: Manycore Platform Software Stack (MPSS). On Windows, these files may have been installed to C:\Program Files (x86)\Intel\Composer XE 2013 SP1.XXX\compiler\lib\mic.

Hardware Acquisition

Phi cards have somewhat strict system requirements. It is quite likely you will need to buy a workstation or server that has been certified compatible with the Xeon Phi. For details, see

There are, for example, Peak workstations or servers supporting Phi cards sold by Pudget Systems. See also Intel’s Where to Buy list.

As of this writing (April 2014), cloud services with the Phi are harder to come by, and SIMION on Amazon EC2 does not offer Phi instances. If you find any suitable ones, let us know.

Questions

Phi support is very new and not well tested. Please send us your questions and reports of experience with it.