Content Comparison

Overview

...

The term high performance computing (HPC) refers to any computational activity requiring more than a single computer to execute a task. Super computers and computer clusters are used to solve advanced computation problems.

How many cores does the Super Computer have? (Storage, RAM...)

48 compute nodes at 12 cores each = 576 cores

48 compute nodes at 64GB of RAM each = 768GB or roughly 3/4 of TB

48 compute nodes at 500GB local storage each = 23 TB

108 TB of GPFS storage. General Parallel File System, now called Spectrum Scale, is a high-performance clustered file system developed by IBM.

How fast is it?

• 10 TFlops as reported by LINPACK. LINPACK is the benchmarking software used to rate and rank super computers.

• The network for intranode communication is based on Infiniband and is rated for 56Gbits/second.

• Comparison: A CPU currently found in some gaming PCs is the Intel I7 3930K which normally runs in the 3GHz range overclocked to 5GHz will net about 100 GFlops.

• Florida Poly's Supercomputer is getting 10TFlops without overclocking and with plenty of room to grow in capacity and speed.

Who made it?

• This HPC platform was made by IBM. The same people who made Watson the Jeopardy playing Super Computer!

What is it for? - or - What can it be used for?

• The platform is very flexible and can be used for a number of things modeling and simulation mathematics and chemistry to assisting in video rendering or "big-data".

What Operating System (OS) does it use?

• At this time, Florida Poly is using Red Hat Enterprise Linux.

What else is in the racks?

...

Requesting Access How do I package?

• Professors

  • Please contact helpdesk to request access to the supercomputer or to request additional applications.

• Students

  • A professor must request access to the HPC and new application needed for the student. (see above)

Useful Information and commands.

• Most users will use Putty to ssh into the HPC using their Poly email address and password. Usually going to Login.hpc.lab

• To upload files to the HPC please use Filezilla or a similar ftp client. The host is login.hpc.lab, username/password is your FLPoly username/password and port is 22

• Submitting jobs to the LSF (Use bsub)

  1. Compile the hello_world example code provided by default

     1. /opt/ibm/platform_mpi/bin/mpicc -o hello_world.exe /opt/ibm/platform_mpi/help/hello_world.c

  2. Submit a job through LSF to test the message passing

     1. bsub -n 10 -R "span[ptile=1]" -o %J.out "/opt/ibm/platform_mpi/bin/mpirun -lsf -vapi /home/(Username)/hello_world.exe; wait"

  3. Check the output of the %J.out file to verify results

• If you request a Virtual Machine, you will be given the address in which you can connect either by ssh or remote desktop depending on the setup.

• To upload files please use Filezilla or a similar ftp client. The host is the name of the Virtual Machine you were given, username/password is your FLPoly username/password and port is 22

• For the EmberDB cluster please ssh into ember.hpc.lab using your FLPoly username and password.

  • Once connected use

    • Mysql –u (your username) -p –h ember-db

      • It will then ask for your FLPoly password.

Applications

...

Our HPC cluster benchmarks at 10 terraflops--around 100 times the performance of a high-end workstation. It’s been designed for loads that require parallel processing of distributed data sets.

Our HPC Cluster

Requesting Access

Faculty:

Please contact the Helpdesk to request access to our HPC cluster: helpdesk@floridapoly.edu or 863.874.8888.

Students:

Please work with a faculty member to sponsor your work.

Shell and Data Access

Shell Access

• Use SSH to access the command shell:

  • Host: login.hpc.lab

  • Port: 22

  • Credentials : your Florida Poly username and password

File Upload

• Use SFTP or SCP to upload files with the access parameters shown above.

Applications and Packages

Libraries

MPI

The Message Passing Interface (MPI) is a library specification that allows the HPC to pass information between its various nodes and clusters.

Compiling Source

The following compilers are installed:

• IBM PE Runtime (mpicc, et. al) v09.01.02.00u including C, C++, and Fortran.

• GNU Compiler Collection v4.4.7 including C, C++, Objective-C, Fortran, Ada, Go, and D.

Packaged Binaries

You can install applications on our HPC using Spack: a Linux package manager that makes installing scientific software easy. With Spack, you can build a package with multiple versions, configurations, platforms, and compilers, and all of these builds can coexist on the same machine.

• spack find

  This lists

  To list all available packages

  .

  :

  • spack load

    This loads the

    find

• To load a package into your environment

  . For example

  :

  • spack load

  • You can specify a software version as part of the load:

  • spack load python@3.7.3

    this

    loads Python 3.7.3

    into your environment.
  You can then

Python and PIP

If you install Python using Spack you can use PIP to install

...

other modules:

• spack load python

• python3 -mpip install matplotlib

  for example

Apache Hadoop 2.6.0

Apache Hadoop is a collection of open-source software utilities that facilitate using a network of many computers to solve problems involving massive amounts of data and computation. It provides a software framework for distributed storage and processing of big data using the MapReduce programming model.

Apache Spark 1.3.1

HPC Components

...

IBM nx360 M4

...

Model:

...

Dual Intel Xeon Processor E5-2620 v2 6C

...

Total cores per node:

...

12 core with 64GB RAM

...

Hardware threads per core:

...

12

...

Hardware threads per node

...

12 x 12 = 144

...

Clock rate:

...

2.1GHz

...

RAM:

...

8 8GB (1x8GB, 2Rx8, 1.35V) PC3L-12800 CL11 ECC DDR3 1600MHz LP RDIMM

...

Cache:

...

15MB Cache 1600MHz 80W

...

Local storage:

...

108 TB of GPFS storage

Management & Login Nodes

IBM System x3550 M4

The x3550 M4 is a cost- and density-balanced 1U, 2-socket server.

Intel Xeon processor E5-2600 v2 product family.

Supports up to 1866 MHz memory speeds.

Supports up to 768 GB memory with 32 GB LRDIMMs.

Network

The RackSwitch G8124E has support for 1G or 10G, this switch is designed for leveraging 10G Ethernet or have plans to in the futureApache Spark is an open-source distributed general-purpose cluster-computing framework. Spark provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.

Other Applications

If you need an application that’s not available through Spack please contact the Helpdesk: helpdesk@floridapoly.edu or 863.874.8888.

Submitting a Job

Linux binaries

You'll submit jobs through bsub, part of IBM’s LSF workload management platform. IBM provides documentation for bsub.

You may wish to use IBM’s mpirun script to abstract the job from underlying hardware. IBM provides documentation for mpirun.

Some examples:
bsub -n 10 -R "span[ptile=1]"-o ~/hello_world.out "mpirun -lsf -vapi ~/hello_world”

bsub -n 20 -R "span[ptile=6]"-o ~/hello_world.out "python3 ~/hello_world.py”

Python

• Install Python
  spack load python

• Install Python bsub
  python3 -mpip install bsub

• Run your code with bsub
  bsub -n 10 -o my_job.out "python3 my_job.py"

Other

For help with other jobs please contact the Helpdesk: helpdesk@floridapoly.edu or 863.874.8888.