What is HPC ? 

HPC

"High-Performance Computing," or HPC, is the application of "supercomputers" to computational problems that are either too large for standard computers or would take too long. A desktop computer generally has had a single processing chip, commonly called a CPU. An HPC system, on the other hand, is essentially a network of nodes, each of which contains one or more processing chips, as well as its own memory.

Parallel Computing

Programs for HPC systems must be split up into many smaller "programs" called threads, corresponding to each core. To piece the larger program together, the cores must be able communicate with each other efficiently, and the system as a whole must be organized well.

Applications of HPC

A high-performance computing center is likely to deal with a large range of problems typified by the following:

• Chemistry
  • Protein-folding attempts to determine the overall shape of a protein based on the sequence of amino acids; the effort may lead to better understanding of biological processes, and new medicines.
  • Protein-ligand interactions play a huge role in biological processes and are common targets for medicine. Refining computational models allows new medicines to be developed more efficiently.
  • Materials research helps to identify new materials, such as high-temperature superconductors, affordable electrodes for fuel cells, efficient catalysts, new methods of energy storage, etc.
• Physics
  • Simulations of stars help astrophysicists interpret observations.
• Environment Modeling
  • The simulation of earthquakes can be startlingly complex. These simulations can be used to predict which areas are likely to experience a large earthquake, and what the conditions will be. This work helps inform building codes, development planning, and emergency action plans.
  • Weather patterns are notoriously difficult to predict. Supercomputers model weather around the country, and weather models are particularly important during hurricane season. Additionally, new computational models may be used with old data to gauge the models' usefulness in predicting weather patterns.
  • Global climate change is a contentious topic these days, in large part due to the extreme complexity of the problem. As computational models improve, we may be better informed in our policy decisions, and able to act in a more unified manner.
• Industry
  • Products: Prototyping is often an expensive and time-consuming process. Computer modeling can reduce the time and cost of physical prototyping and make optimization much easier. This process is essential not only for obvious industries such as the automotive, aerospace, and pharmaceutical industries, but a wide spectrum of consumer products, from dishwashers to potato chips.
  • Services: Many corporations handle large amounts of data, and deal with complex routings or decisions. Insurers, for instance, use models to calculate risk, while the oil/gas industry uses seismographic data to determine where oil and gas fields are located underground. Wal-Mart optimizes supply chains, and Fed-Ex monitors and routes packages. The financial sector uses models to spot trends and calculate risks.

HPC Glossary

This is a list of basic terms that might be used in HPC.

Autoparallelisation A technique used by a compiler to take a sequential code and distribute the work across multiple processors

Autovectorisation A technique used by a compiler to take a sequential code and replace sequences of operations by vector instructions that can be executed on a vector processor.

Batch queue A subsystem managing an HPC's workload. Users submit batch jobs to the queue to be executed when resources are available. See also Scheduler and Resource Manager

Beowulf A cluster of Linux based PC's, using commodity hardware and open source software

BLAS Basic Linear Algebra Subroutines -- a kernel of linear algebra routines usually heavily optimised by a CPU's vendor to make use performance features of that CPU. Used extensively by Lapack Bus Connection between CPU and memory and/or other peripherals.

C,C++ High level programming languages widely available on HPC computers, standardised by the ANSI committee. C is almost a subset of C++. C++ offers features such as object oriented programming, generic programming and operator overloading.

Cache Small piece of fast, expensive memory between the CPU and main memory

Cache Coherency Hardware support on SMP systems to ensure that each processor's cache has a consistent view of the data in main memory

Checkpoint The state of a running program written to a file to allow the program to restart a calculation from the point saved in the checkpoint file.

Classdesc A technology for passing objects between processes, or saving objects to files in C++. See http://parallel.hpc.unsw.edu.au/classdesc for more details.

Cluster Multiple complete computers connected via a network, and operated as a single system.

Compiler A program that translates high level source code into object code

CPU Central Processing Unit

Cygwin A suite of open source unix-like development tools for Windows based computers

Debugger A tool to interactively stop a program, step the program line by line and examine contents of the program's data

Distributed Memory A parallel computer whose processors' main memory is physically separated from each others -- contrast with shared memory

Ensemble Calculation Multiple copies of a simulation run with a different sequence of random numbers, and the results averaged to produce a more accurate statistical measure.

Flop Floating point operations per second. Addition, Subtraction and Multiplication are considered atomic floating point operations, but Divisions usually performed by combining multiplications and additions as in the long division algorithm

Fortran A very popular high level language used with HPCs. Standardised by the ANSI committee, the standards are known by the year in which the standard was released: Fortran77, Fortran90, Fortran95.

Generic Programming A technique for writing code supporting multiple types. Also known as a template. Supported in C++, and coming soon to Java.

High Performance Computer A computer whose performance exceeds that of standard workstations by at least an order of magnitude.

High Performance Fortran A distributed memory programming language based on Fortran95, abbreviated with HPF

Hyperthreading A technique of running multiple threads on a CPU so that the CPU doesn't stall when waiting for data from memory.

Interpreter A program that executes high level source code without translating it into object code first.

Java An interpreted object oriented high level language widely deployed on the Internet, but not too popular for high performance computing due to performance reasons.

Java Grande is an attempt to promote Java for High Performance Computing.

Lapack A highly efficient, open source linear algebra library widely available on high performance computers. Functions include solutions to linear equations and eigenvectors. Depends heavily on BLAS.

Latency The amount of time it takes to complete an operation - eg the time for a data packet to travel between two computers. cf bandwidth

Library A collection of precompiled (as in object code) routines that can be linked to a program.

Linker A program that links together independently compiled pieces of object code (including those residing in libraries into a single executable file).

Maui An open source scheduler

Multiple Data -- a parallel computer with each processor working independently on its own data -- cf SIMD, SPMD

MPI Message Passing Interface -- an industry standard interface for programming distributed memory computers.

Object Code Compiled machine code, that may contain references to code and data in other object files.

Object Oriented Programming A technique for attaching code to pieces of data (the combination called an object). Whilst this can be done with any language, it is easiest in languages that explicitly support OOP.

OpenMP A standard interface for distributing work across threads of a shared memory computer.

Open Source A program whose source code is available to the general public.

Operator Overloading A technique for redefining the operator symbols (+,-,*,/, etc) to do whatever the programmer wants on arbitrary typed data.

Optimiser Part of a compiler that performs a suite of basic code optimisations automatically.

Paging Transferring data from main memory to disk as part of the virtual memory subsystem.

Parallel Computer A computer having multiple CPUs

Parametric Calculation Multiple copies of a program run, each with a different set of input parameters

PBS An open source batch queuing system

Posix Threads A standard interface for controlling threads on a shared memory computer

Process A distinct task having its own state and address space. cf thread

Profiler A tool used to analyse which parts of a program are most computationally significant.

PVM Parallel Virtual Machine -- an interface for programming distributed memory computers.

RAM Random Access Memory -- the main memory is made out of RAM, so RAM sometimes stands for the main memory

Scheduler A program that controls which batch job runs next, when a resources are available.

Shell An interpreter used for executing commands typed at a keyboard, or scripts.

SIMD Single Instruction, Multiple Data -- A parallel computer performing the same operation, on different items of data at the same time. cf Vector, MIMD, SPMD

SMP A shared memory parallel computer (sometimes symmetric multiprocessor)

Software Pipelining A technique employed by a compiler for improving the performance of a CPU. Given operations that are decomposed into a sequence of steps that use distinct pieces of hardware, software pipelining arranges for the next operation to be started before the first finishes, increasing the amount of hardware used at any one time, and improving the CPUs throughput.

SPMD Single Program, Multiple Data The same program is executed on different items of data, but the operations need not be performed in lockstep (i.e. at each step (global clock tick) all processors execute the same instruction, each on a different datum) as in SIMD. MPI is an SPMD programming environment, whereas PVM is full MIMD.

SDRAM : A faster version of DRAM, mostly employed for main memory SRAM

Static RAM : A fast, expensive RAM often used for vector computers or caches

ssh Secure Shell An encryption protocol used for computer communications

Standard Template Library A standard library of containers (aggregates of objects of arbitrary type, such as vectors, lists, sets etc) for C++. Make extensive use of generic programming.

Stack An area of memory used to store arguments and local variables of subroutines and functions. As a subroutine is called, a stack frame is pushed onto the stack, allocating memory for these variables. As the subroutine exists, the stack frame is popped off, destroying the contents of these variables.

Superlinear A program that actually performs better than $n$ times faster than sequential, when run on $n$ processors. Usually due to cache effects (total cache also increases with processor count).

Swapping see Paging

Threads Independent tasks executing the same code in the same address space. Each thread has its own stack, so local variables are thread private. Global variables are shared.

Vector Computer A computer implementing instructions that work on a vector of data at once. This may be implemented as a pipeline (each operation is further decomposed into steps that each run on their own hardware), or as a SIMD processor. Vector supercomputers do not have a cache, but have very fast main memory instead, consequently outperforming commodity processor.

Virtual Memory Address space is divided up into chunks called pages. A page needs to be in physical memory when accessed by the CPU, but may be stored on disk if not needed for a while, thus freeing physical memory for other processes that need it.

Workstation A networked desktop computer running a modern multitasking operating system such as Unix or MS Windows.

X windows A graphical windows interface where the program performing the graphics (the X-windows client) may be running on a different computer to where the user is seated (called the X-server). The client and server communicate using the X libraries protocol (Xlib).