Home
Client-Server Setup
Simcenter STAR-CCM+ uses a client-server architecture in which the simulation objects are created on the server and the client provides access to these objects for setup and display.
Working With Parallel Servers
A parallel computation is one where the work is computed by more than one process.
Supported MPI Implementations
Regardless of whether you use a local multi-core workstation, remote machines, or a remote cluster to run your parallel server, all approaches require an implementation of the Message Passing Interface (MPI).
Selecting the Fabric
The fabric describes the interface between the MPI layer and the network hardware. For instance, TCP is the fabric interface to Ethernet. Infiniband hardware can be interfaced using different software layers, each described using a different fabric selection.

Share

Link: copied

Breadcrumb: copied

Client-Server Setup
Simcenter STAR-CCM+ uses a client-server architecture in which the simulation objects are created on the server and the client provides access to these objects for setup and display.
- Client-Server Connections
  Simcenter STAR-CCM+ allows you to connect a client to any running server for which you have access, and also to disconnect clients when you choose. The client version must match the server version for a successful connection. Secure connections require additional inputs.
- Mixed or Double Precision
  Simcenter STAR-CCM+ comes in two versions: a mixed precision version and a double precision version. Both versions are provided on the download area of the Support Center portal.
- Running Simcenter STAR-CCM+ on Linux ARM64
  Simcenter STAR-CCM+ is supported on Linux ARM64 for running simulations in batch mode. The graphical user interface (GUI) is not officially supported in this release. Some other restrictions are in place for this platform, including unsupported modules.
- Launching Simcenter STAR-CCM+ and Working With Simulations
  A Simcenter STAR-CCM+ simulation exists on a server, and is accessed by connecting a client to the server on which the simulation exists.
- Working With Parallel Servers
  A parallel computation is one where the work is computed by more than one process.
  - Parallel Servers and Requirements
    A parallel server is a Simcenter STAR-CCM+ server that is set to run on multiple cores or a cluster of machines when it is first launched.
  - Supported MPI Implementations
    Regardless of whether you use a local multi-core workstation, remote machines, or a remote cluster to run your parallel server, all approaches require an implementation of the Message Passing Interface (MPI).
    - Using Open MPI
      Open MPI is an actively developed and widely used open source MPI distribution. Simcenter STAR-CCM+ includes two distributions of different versions within its installer, and supports a locally installed Version 4.
    - Using Cray MPI
      Simcenter STAR-CCM+ supports Cray’s optimized MPI libraries on Cray EX and Cray XT/XE/XC clusters.
    - Selecting the Fabric
      The fabric describes the interface between the MPI layer and the network hardware. For instance, TCP is the fabric interface to Ethernet. Infiniband hardware can be interfaced using different software layers, each described using a different fabric selection.
    - Registering Your Credentials for Distributed Simulations on Windows
    - Using Microsoft MPI
      Microsoft MPI (MS-MPI) is only available under Windows.
    - Using Intel MPI
      The Intel MPI library is a multifabric message-passing library that implements the open-source MPICH specification. The library is design for maximum performance on clusters that use Intel processors.
    - Using UCX
      UCX is a communication library which is primarily used and recommended on Mellanox InfiniBand systems. The library can be used with both the Intel MPI and Open MPI distributions of Simcenter STAR-CCM+ on Linux systems.
    - Using HPE MPI
      HPE MPT is the default MPI implementation on HPE cluster systems. It can be selected via the Simcenter STAR-CCM+ command line option -mpi hpe. Simcenter STAR-CCM+ currently only supports HPE MPT, not HPE HMPT.
    - Using Fujitsu Open MPI
      Simcenter STAR-CCM+ supports the Fujitsu Open MPI variant for A64FX clusters with Tofu interconnects.
    - Using Custom MPI Implementations
      If you want to use an MPI library which is not ABI-compatible, but whose mpiexec takes the same command line arguments as one of the supported MPI implementations, you can leverage the MPItrampoline / MPIwrapper tandem which is used by Simcenter STAR-CCM+.
  - Using a Local Multi-Core Workstation
  - Using a Remote Multi-Core Workstation
  - Using a Group of Machines
  - Troubleshooting Parallel Servers and MPI
    This section contains some troubleshooting tips when working with parallel servers and MPI distributions.
  - Balancing
  - Getting Parallel Help through the Command Line
    You can get information on the parallel options through the command line.
  - Viewing the Partitioning
  - Using Batch Management Systems
    Batch systems manage the allocation of computational jobs to a single large computer or a cluster of smaller computers.
  - Vectorization
    Simcenter STAR-CCM+ uses AVX-2 instructions on all Xeon processors that support the AVX-2 instruction set—currently, the Haswell, Broadwell, and Skylake processors.
  - Running Distributed Parallel Simulations on Multiple Windows Machines
- GPGPU Computation
  The idea behind general-purpose computing on graphics processing units (GPGPU computation) is that you assign computations that are traditionally solved on a CPU (central processing unit) to a graphics processing unit (GPU). The nature of operations on the GPU means that a single unit can deliver computational performance comparable to using many CPUs in parallel. GPGPU computation is only supported for Simcenter STAR-CCM+ servers running under Linux.
- Teamcenter Share
  Teamcenter Share is a collaboration platform that you can access directly within the Simcenter STAR-CCM+ workspace as well as in a web browser. Teamcenter Share allows you to upload and share simulation files and related data, such as Viewer (.sce) files. Through the integration within Simcenter STAR-CCM+, you can download and open files from this platform.
- Command-Line Reference

Selecting the Fabric

The fabric describes the interface between the MPI layer and the network hardware. For instance, TCP is the fabric interface to Ethernet. Infiniband hardware can be interfaced using different software layers, each described using a different fabric selection.

Each MPI implementation defaults to the most suitable fabric. In most cases, you do not need to specify the fabric. You need to specify the fabric only if the MPI distribution fails to automatically pick the correct fabric or if you wish to use a non-default fabric. The fabrics available for selection depend upon their availability on the system, and the MPI implementation being used.

On Cray systems, Cray MPI must be used and you have no control over fabric selection. On Fujitsu ARM64 systems with Tofu network interconnects, Fujitsu Open MPI must be used and there is no control over fabric selection. On these systems, the locally installed MPI distribution is tuned with the hardware to provide optimal performance.

The fabric can be specified using the command line option:

-fabric <fabric>

If you enter the fabric option in uppercase (e.g. -fabric TCP), this specifies no fall-back. If you specify the fabric option in lowercase (e.g. -fabric tcp), Simcenter STAR-CCM+ falls back to an alternative fabric if the specified fabric is not available, and if an alternative fabric is available and supported by the MPI implementation.


Fabric Option	Fabric	Notes
-fabric IBV	Infiniband OpenFabrics Alliance Verbs API (IBV)	Generally not recommended for use with Open MPI (use, for example, UCX instead)
-fabric OFI	Open Fabric Interfaces (OFI) / Libfabric	Supports several interconnects through providers
-fabric OPA	Intel Omni-Path	Recommended on Intel Omni-Path interconnects
-fabric PSM	Intel Performance Scaled Messaging (PSM)	Recommended on Intel True Scale architectures
-fabric PSM2	Intel Performance Scaled Messaging 2	Recommended on Intel Omni-Path interconnects
-fabric TCP	Ethernet	Use only if no high-performance interconnect is available
-fabric UCX	Unified Communication X (UCX)	Recommended on Mellanox InfiniBand and Rockport architectures

To show the fabric selected by MPI, where applicable, use:

-fabricverbose