Lustre intel: Setting up a Windows client

From Define Wiki
Jump to navigation Jump to search

The following instructions are for one (or multiple) linux system(s), which will serve as the middle-layer cluster, which "translates" Lustre to Windows, using Samba.

There are two components that need to be installed:

  1. CTDB
  2. Samba

Pre-Requisites

  • Install Lustre Client packages
  • Lustre Client File System mounted with full cluster-coherent locking support on all CTDB nodes:
mkdir -m 0755 -p /mnt/lfs
mount -t lustre -oflock 172.28.50.2@tcp0:172.28.50.3@tcp0:/lfs1 /mnt/lfs/
  • Networking
    Each CTDB node will require three network interfaces on three separate networks inorder to operate correctly:
    1. a high performance data network interface, used to mount the Lustre client file system. This is typically an InfiniBand or 10Gb Ethernet fabric
    2. a private network interface over which all CTDB nodes communicate, sending heartbeats, exchanging data and cluster commands
    3. a public network interface over which clients will connect to the services running on the CTDB cluster, such as Samba
    In our case, networks 1 and 2 are the same (pxe). Here is an example of a CTDB Framework on Lustre:
Error creating thumbnail: File missing

Install CTDB

Install CTDB as follows: 

sudo yum -y install ctdb

Configure CTDB

On RHEL 6, the global configuration file is /etc/sysconfig/ctdb. At a minimum, CTDB needs the following information:

  • The list of IP addresses of every member of the CTDB cluster on the private network. This is stored in a plain text file referred to as the node list, the location of which is governed by the variable CTDB_NODES in the global configuration.
    In our one node example:
[root@ctdb-node ~] cat /etc/ctdb/nodes 
172.28.0.131
  • The list of public IP addresses, along with the subnet mask and the network interface, that will be used for hosting public services. This information is kept in a plain text file referenced by the global configuration variable CTDB_PUBLIC_ADDRESSES. The format of the IPs should be <IP Address>/<NetMask> <Network Device>.
    In our one node example:
[root@ctdb-node ~] cat /etc/ctdb/public_addresses 
10.0.0.129/8 eth0
  • The network interface for the public network should not be configured by the operating system with any of the CTDB public IP addresses. It is possible for the interface to be configured and active with an address that lies outside the range used by CTDB, but it is far simpler to prevent the operating system from attempting to manage the network interface device in the first place. In the configuration file /etc/sysconfig/network-scripts/ifcfg-<device>, set the following variables as shown:
ONBOOT=no
NM_CONTROLLED=no
BOOTPROTO=none
  • The location of the recovery lock file on the shared file system (in this case, the recovery lock is kept on Lustre). The recovery lock location is kept in the global configuration variable CTDB_RECOVERY_LOCK.
    For example:
# Lustre is mounted at /lustre/scratch.
# Create a directory to contain the CTDB recovery lock:
[root@ctdb-node ~] mkdir -p -m 0700 /mnt/lfs/.ctdb
[root@ctdb-node ~] ls -ld /mnt/lfs/.ctdb
drwx------ 2 root root 4096 Oct 10 17:56 /mnt/lfs/.ctdb

CTDB Configuration example

CTDB_RECOVERY_LOCK="/mnt/lfs/.ctdb/recovery_lock"
CTDB_PUBLIC_ADDRESSES=/etc/ctdb/public_addresses
CTDB_NODES=/etc/ctdb/nodes
CTDB_LOGFILE=/var/log/log.ctdb
CTDB_DEBUGLEVEL=ERR
CTDB_SYSLOG=no

Starting the CTDB Cluster

[root@ctdb-node ~] sudo service ctdb start
[root@ctdb-node ~] sudo ctdb status
Number of nodes:1
pnn:0 172.28.0.131     OK (THIS NODE)
Generation:81835478
Size:1
hash:0 lmaster:0
Recovery mode:NORMAL (0)
Recovery master:0

Install Samba

sudo yum -y install samba samba-client samba-common samba-doc samba-winbind samba-winbind-clients

Configure and Start Samba

[root@ctdb-node ~] mv /etc/samba/smb.conf /etc/samba/smb.conf-backup
[root@ctdb-node ~] vim /etc/samba/smb.conf #create a new config file with the following info
[root@ctdb-node ~] cat /etc/samba/smb.conf
[global]
netbios name = usedForWindows
workgroup = bostonLustre
server string = [%h] Samba %v Server (Lustre)
security = user
encrypt passwords = yes
passdb backend = tdbsam
clustering = yes

[root@ctdb-node ~] testparm -s #test samba
Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
Loaded services file OK.
Server role: ROLE_STANDALONE
[global]
	workgroup = BOSTONLUSTRE
	netbios name = USEDFORWINDOWS
	server string = [%h] Samba %v Server (Lustre)
	clustering = Yes
	idmap config * : backend = tdb

[root@ctdb-node ~] id ajax 
uid=500(ajax) gid=500(ajax) groups=500(ajax)
[root@ctdb-node ~] sudo smbpasswd -a ajax #add an EXISTING user in samba database
New SMB password:
Retype new SMB password:
Added user ajax

[root@ctdb-node ~] sudo service nmb start
[root@ctdb-node ~] sudo service smb start

[root@ctdb-node ~] service iptables stop #stop iptables

[root@ctdb-node ~] smbclient -L //USEDFORWINDOWS -N
Anonymous login successful
Domain=[BOSTONLUSTRE] OS=[Unix] Server=[Samba 3.6.23-14.el6_6]

	Sharename       Type      Comment
	---------       ----      -------
	lfs             Disk      
	IPC$            IPC       IPC Service ([ctdb-node] Samba 3.6.23-14.el6_6 Server (Lustre))
Anonymous login successful
Domain=[BOSTONLUSTRE] OS=[Unix] Server=[Samba 3.6.23-14.el6_6]

	Server               Comment
	---------            -------
	USEDFORWINDOWS       [ctdb-node] Samba 3.6.23-14.el6_6 Server (Lustre

	Workgroup            Master
	---------            -------
	BOSTONLUSTRE         USEDFORWINDOWS

[root@ctdb-node ~] smbclient -L //USEDFORWINDOWS -U ajax #test the config for the user
Enter ajaxs password:
Domain=[BOSTONLUSTRE] OS=[Unix] Server=[Samba 3.6.23-14.el6_6]

	Sharename       Type      Comment
	---------       ----      -------
	lfs             Disk      
	IPC$            IPC       IPC Service ([ctdb-node] Samba 3.6.23-14.el6_6 Server (Lustre))
Domain=[BOSTONLUSTRE] OS=[Unix] Server=[Samba 3.6.23-14.el6_6]

	Server               Comment
	---------            -------
	USEDFORWINDOWS       [ctdb-node] Samba 3.6.23-14.el6_6 Server (Lustre

	Workgroup            Master
	---------            -------
	BOSTONLUSTRE         USEDFORWINDOWS

Add Samba to CTDB

Preparation

sudo service ctdb stop
sudo service nmb stop
sudo service smb stop

Update the Samba Configuration

To turn on clustering support, edit the Samba configuration file (/etc/samba/smb.conf) created in the previous section and add the following to the [global] section: 

clustering = yes

Update the CTDB Configuration

Edit the CTDB configuration file (default: /etc/sysconfig/ctdb) and add the following entries: 

CTDB_MANAGES_SAMBA=yes
CTDB_SERVICE_NMB=nmb
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=Lustre_intel:_Setting_up_a_Windows_client&oldid=6264"