Difference between revisions of "FhGFS Installation and Configuration"
Jump to navigation
Jump to search
| Line 5: | Line 5: | ||
http://www.fhgfs.com/wiki/images/sysarch2.png | http://www.fhgfs.com/wiki/images/sysarch2.png | ||
| − | == | + | == Prepare Hosts == |
* Add the FhGFS repo to your environment | * Add the FhGFS repo to your environment | ||
<syntaxhighlight> | <syntaxhighlight> | ||
| Line 16: | Line 16: | ||
rpm --import http://www.fhgfs.com/release/latest-stable/gpg/RPM-GPG-KEY-fhgfs | rpm --import http://www.fhgfs.com/release/latest-stable/gpg/RPM-GPG-KEY-fhgfs | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| + | |||
| + | == Setup the Hosts == | ||
| + | * In this configuration we will be running an 8 node setup | ||
| + | ** Node1, Management | ||
| + | ** Node2, Metadata | ||
| + | ** Node3, Storage Server | ||
| + | ** Node4, Storage Server | ||
| + | ** Nodes5-8, Clients | ||
| + | |||
| + | == Management Server == | ||
| + | * Install the management server | ||
| + | <syntaxhighlight> | ||
| + | yum install fhgfs-mgmtd | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | == MetaData Server == | ||
| + | * Install the metadata server package | ||
| + | <syntaxhighlight> | ||
| + | yum install fhgfs-meta | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | == Storage Server(s) == | ||
| + | * On each storage server | ||
| + | <syntaxhighlight> | ||
| + | yum install fhgfs-storage | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | == Clients == | ||
| + | * On each client | ||
| + | <syntaxhighlight> | ||
| + | yum install fhgfs-client | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | * For each client, there will need to be a kernel built. | ||
| + | * This happens automatically when the service starts | ||
| + | * Ensure 'Development tools' and 'kernel-devel' are installed | ||
| + | * Verify the kernel in <tt>/usr/src/kernels</tt> matches the output of <tt>uname -r</tt> | ||
| + | <syntaxhighlight> | ||
| + | $ pwd | ||
| + | /usr/src/kernels | ||
| + | $ uname -r | ||
| + | 2.6.32-358.el6.x86_64 | ||
| + | $ ls | ||
| + | 2.6.32-358.11.1.el6.x86_64 | ||
| + | ln -s 2.6.32-358.11.1.el6.x86_64/ `uname -r` | ||
| + | $ ls | ||
| + | 2.6.32-358.11.1.el6.x86_64 2.6.32-358.el6.x86_64 | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | * Then either build the kernel manually or start the service | ||
| + | <syntaxhighlight> | ||
| + | $ /etc/init.d/fhgfs-client rebuild | ||
| + | - FhGFS module autobuild | ||
| + | Building fhgfs-client-opentk module | ||
| + | Building fhgfs client module | ||
| + | $ modinfo fhgfs | ||
| + | filename: /lib/modules/2.6.32-358.el6.x86_64/updates/fs/fhgfs_autobuild/fhgfs.ko | ||
| + | author: Fraunhofer ITWM, CC-HPC | ||
| + | description: FhGFS parallel file system client (http://www.fhgfs.com) | ||
| + | license: GPL v2 | ||
| + | srcversion: D15ED391FD2704BA5D65B86 | ||
| + | depends: fhgfs-client-opentk | ||
| + | vermagic: 2.6.32-358.11.1.el6.x86_64 SMP mod_unload modversions | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | * '''Note: The standard module only build for ethernet only, no IB support''' | ||
| + | * Rebuild client for IB (OFED) | ||
Revision as of 13:22, 30 June 2013
- Steps taken from: http://www.fhgfs.com/wiki/wikka.php?wakka=DownloadInstallationPackages
- Installation on Centos 6 servers with IB
System Architecture
http://www.fhgfs.com/wiki/images/sysarch2.png
Prepare Hosts
- Add the FhGFS repo to your environment
cd /etc/yum.repos.d/
wget http://www.fhgfs.com/release/latest-stable/dists/fhgfs-rhel6.repo- Repository Signature Keys
rpm --import http://www.fhgfs.com/release/latest-stable/gpg/RPM-GPG-KEY-fhgfsSetup the Hosts
- In this configuration we will be running an 8 node setup
- Node1, Management
- Node2, Metadata
- Node3, Storage Server
- Node4, Storage Server
- Nodes5-8, Clients
Management Server
- Install the management server
yum install fhgfs-mgmtdMetaData Server
- Install the metadata server package
yum install fhgfs-metaStorage Server(s)
- On each storage server
yum install fhgfs-storageClients
- On each client
yum install fhgfs-client- For each client, there will need to be a kernel built.
- This happens automatically when the service starts
- Ensure 'Development tools' and 'kernel-devel' are installed
- Verify the kernel in /usr/src/kernels matches the output of uname -r
$ pwd
/usr/src/kernels
$ uname -r
2.6.32-358.el6.x86_64
$ ls
2.6.32-358.11.1.el6.x86_64
ln -s 2.6.32-358.11.1.el6.x86_64/ `uname -r`
$ ls
2.6.32-358.11.1.el6.x86_64 2.6.32-358.el6.x86_64- Then either build the kernel manually or start the service
$ /etc/init.d/fhgfs-client rebuild
- FhGFS module autobuild
Building fhgfs-client-opentk module
Building fhgfs client module
$ modinfo fhgfs
filename: /lib/modules/2.6.32-358.el6.x86_64/updates/fs/fhgfs_autobuild/fhgfs.ko
author: Fraunhofer ITWM, CC-HPC
description: FhGFS parallel file system client (http://www.fhgfs.com)
license: GPL v2
srcversion: D15ED391FD2704BA5D65B86
depends: fhgfs-client-opentk
vermagic: 2.6.32-358.11.1.el6.x86_64 SMP mod_unload modversions- Note: The standard module only build for ethernet only, no IB support
- Rebuild client for IB (OFED)