Rocks: RLIMIT MEMLOCK is 32768 OFED Mpirun

From Define Wiki
Jump to navigation Jump to search
Problem
  • Error when running openmpi: RLIMIT_MEMLOCK is 32768

With OpenFabrics (and therefore the openib BTL component), you need to set the available locked memory to a large number (or better yet, unlimited) -- the defaults with most Linux installations are usually too low for most HPC applications that utilize OpenFabrics. Failure to do so will result in a error message similar to one of the following (the messages have changed throughout the release versions of Open MPI):

Resolution

Soft: The "soft" value is how much memory is allowed to be locked by user processes by default. Set it by adding a line in /etc/security/limits.conf: 

     * soft memlock <number>

Hard: The "hard" value is the maximum amount of memory that a user process can lock. Set it by adding a line in /etc/security/limits.conf: 

     * hard memlock <number>

where "<number>" is the maximum number of bytes that you want user processes to be allowed to lock (presumably rounded down to an integral number of pages). "<number>" can also be "unlimited".

Example Configuration File
  • Correct /etc/security/limits.conf
* soft memlock unlimited
* hard memlock unlimited
  • NOTE
    • Ensure this file is sync'd correctly across the cluster (411 or CFM).
    • You may need to restart schedule process on the compute node
  service sgeexecd restart (SGE)
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=Rocks:_RLIMIT_MEMLOCK_is_32768_OFED_Mpirun&oldid=2495"