FE Sparse Direct Solver Reference

In finite element simulations, Simcenter STAR-CCM+ solves the system of linear equations using direct solvers.

The direct solver takes advantage of the modern SIMD architectures. When you use the Solid Stress and Finite Element Solid Energy solvers alongside the direct solver, the models benefit from using hardware with, for example, AVX-512 capabilities.

Sparse Direct Solver Properties

Verbosity
Specifies how much information is printed to the Output window. The available options are None, Low, and High. To visualize an estimate of your case memory requirements in the Output window, choose Low or High. The memory requirements depend on the model degrees of freedom.
Mode
Specifies whether the solver uses the physical memory (In-Core) or disk memory (Out-Of-Core) to store the matrix. The available options are:
  • Auto (default)—Simcenter STAR-CCM+ decides whether to run the solver in IC (In-Core) or OOC (Out-Of-Core) mode, based on the memory available to the system.
  • IC—In-Core (IC) mode. The solver only uses the RAM memory. In terms of CPU time, this mode is faster than the OOC mode.
  • OOC—Out-of-Core (OOC) mode. The solver also uses the disk memory. This mode requires that you specify a temporary write directory for data storage. This mode is slower than IC but more robust, and is intended for simulations with high memory requirements.
Temp Write Directory
Allows you to specify a directory on your system disk to use for temporary storage of data when running in OOC mode (either directly or through the Auto mode). The available options are:
  • System Tmp (default)—Simcenter STAR-CCM+ uses the system temporary directory for storage. In general, the default system tmp directory is limited in size.
  • Current—Simcenter STAR-CCM+ uses the current working directory, that is, the directory from which you launch Simcenter STAR-CCM+.

In both cases, the path and name of the specified directory must not contain non-ASCII characters.

For large simulations, make sure that you choose a directory that has sufficient space for storage. You can either:
  • Launch Simcenter STAR-CCM+ from a directory with large empty space and set Temp Write Directory to Current.
  • Before launching Simcenter STAR-CCM+, specify a different system temporary directory by setting the environment variable TMP on Windows, or TMPDIR on Linux. Then, set Temp Write Directory to System Tmp.

When running the simulation, Simcenter STAR-CCM+ creates subdirectories in the specified Temp Write Directory, and then removes them automatically when you close the simulation. If the simulation ends unexpectedly, delete the subdirectories manually.

Stop if Not Enough Mem
If activated, it prevents you from running the simulation when the memory requirements for the simulation exceed the memory available to your system. If deactivated, the simulation is run regardless of the memory requirements and availability.

In solid stress simulations, you can use the Estimate Memory option for the Solid Stress Solver to visualize an estimate of the memory requirements in the Output window. See Solid Stress Solver Reference.

Sparse Direct Solver Controls

Direct Solver Option
This option allows you to specify which direct solver Simcenter STAR-CCM+ uses for solving the sparse system of linear equations (Eqn. (4839)). The available solvers are:
  • MUMPS—available for serial and parallel execution. MUMPS is a multi-frontal solver applicable to both shared-memory and distributed-memory systems [937].
  • HYBRID MUMPS—additional solver available for parallel execution. This solver uses a hybrid MPI and thread-parallel execution model, which, compared to MUMPS, reduces the memory usage. This method is default only for new finite element solid stress simulations.
All solvers make extensive use of graph partitioning available from the Metis, BLAS, and MKL libraries for speed and accuracy. The solvers can solve "out of core" when the available physical memory is not sufficient to store the matrix.
When changing the solver option during a simulation run, the change takes effect at the next factorization step (see Sparse Direct Solver). However, if you change the Verbosity during a simulation run the change takes effect immediately.
Hybrid MUMPS Shared Memory Scope
This option allows you to specify the memory scope for the HYBRID MUMPS direct solver. The available scopes are:
  • Host (default)—a single Simcenter STAR-CCM+ process runs on the entire host and threading is used within this process. This option generally results in the lowest memory usage.
  • Socket—several CPU sockets can be installed on a single host. This option enables one process per CPU socket and threading within each process. For a single CPU socket per host, this option is equivalent to Host.
  • NUMA—in some hardware environments, a single CPU socket can consist of several NUMA (non-uniform memory access) nodes. This option enables one process per NUMA node and threading within each process. For a single NUMA node per CPU socket, this option is equivalent to Socket.
In general, smaller scopes like NUMA can yield to an increase in performance. However, the smaller the scope, the higher the memory usage. To find the optimal option for your case, you can start by running the solver using a large scope (Host) and then incrementally decrease the scope, evaluating gains in performance.
For simulation files created in versions of Simcenter STAR-CCM+ from v13.06 onwards, the default option is Host. For simulation files created in previous versions of Simcenter STAR-CCM+, the default option is NUMA.