View source for Rocks: RLIMIT MEMLOCK is 32768 OFED Mpirun

===== Problem =====

* Error when running openmpi: RLIMIT_MEMLOCK is 32768

With OpenFabrics (and therefore the openib BTL component), you need to set the available locked memory to a large number (or better yet, unlimited) -- the defaults with most Linux installations are usually too low for most HPC applications that utilize OpenFabrics. Failure to do so will result in a error message similar to one of the following (the messages have changed throughout the release versions of Open MPI):

===== Resolution =====

<b>Soft</b>: The "soft" value is how much memory is allowed to be locked by user processes by default. Set it by adding a line in /etc/security/limits.conf:

      * soft memlock <number>

<b>Hard</b>: The "hard" value is the maximum amount of memory that a user process can lock. Set it by adding a line in /etc/security/limits.conf:

      * hard memlock <number>

where "<number>" is the maximum number of bytes that you want user processes to be allowed to lock (presumably rounded down to an integral number of pages). "<number>" can also be "unlimited".

===== Example Configuration File =====
* Correct /etc/security/limits.conf
<syntaxhighlight>
* soft memlock unlimited
* hard memlock unlimited
</syntaxhighlight>

* <b>NOTE</b>
** Ensure this file is sync'd correctly across the cluster (411 or CFM).
** You may need to restart schedule process on the compute node
<pre>
  service sgeexecd restart (SGE)
</pre>