Allocating Cores to Abaqus in Batch Queuing Systems

When running both Simcenter STAR-CCM+ and Abaqus as parallel jobs on batch queuing systems, Simcenter STAR-CCM+ allows you to set the number of cores to allocate to Abaqus in the Simcenter STAR-CCM+ launch command. It does this through a second parameter on the -np option, that is: 

-np [starccm+_cores],[abaqus_cores]

To run an Abaqus Co-Simulation on a batch queuing system:

  1. Edit the launch command in the run script and allocate the desired number of cores for each process. In the following example, Simcenter STAR-CCM+ is allocated 24 cores, and Abaqus is allocated 8 cores: 
    [INSTALL_DIR]/star/bin/starccm+ -np 24,8 -batchsystem <system> -batch <macro_file>.java

    The Abaqus value (8) that is defined in the above argument automatically overwrites the value of Number of CPUs in the Execution Properties. See Specifying Co-Simulation Settings in Abaqus.

    NoteThe Simcenter STAR-CCM+ CPU Binding feature for parallel jobs can prevent a co-simulation from running in parallel as expected. You are advised to deactivate this option when you launch more than one process per core. To deactivate CPU Binding, launch Simcenter STAR-CCM+ with the -cpubind off argument. In general, deactivating CPU Binding decreases performance. See CPU Binding.
  2. Submit the run script to the batch management system.

Once the run script starts, Simcenter STAR-CCM+ generates a machine file from the hosts that the queuing system allocates. Simcenter STAR-CCM+ uses the number of cores defined by the first value in the -np argument. When Simcenter STAR-CCM+ subsequently launches Abaqus, it allocates to Abaqus the number of cores defined by the second value in the -np argument using hosts not already allocated to run Simcenter STAR-CCM+ .

The total number of cores requested from the queuing system determines whether Simcenter STAR-CCM+ and Abaqus are run on separate cores or if they run on the same cores.

  • To run each code on separate cores, request the sum of the cores designated on the -np argument from the queuing system.
  • You can run each code on the same cores for cases where the codes are not running concurrently, (that is, the Time-Marching Sequence Option is set to STAR-CCM+ Leads or Abaqus Leads). The memory footprint of each code must fit within the local memory of each host. If these conditions apply, then ask the queuing system for the largest of the two numbers appended to the -np argument. The two codes then use the same cores but run in sequence (so that Simcenter STAR-CCM+ sleeps while Abaqus runs, and vice versa).