Oracle® Linux 8

Managing Shared File Systems

F29522-03June 2020

Oracle Legal Notices

Copyright © 2020 Oracle and/or its affiliates.

This software and related documentation are provided under a license agreement containing restrictions on use anddisclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreementor allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute,exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, ordecompilation of this software, unless required by law for interoperability, is prohibited.

The information contained herein is subject to change without notice and is not warranted to be error-free. If you findany errors, please report them to us in writing.

If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf ofthe U.S. Government, then the following notice is applicable:

U.S. GOVERNMENT END USERS: Oracle programs (including any operating system, integrated software, anyprograms embedded, installed or activated on delivered hardware, and modifications of such programs) andOracle computer documentation or other Oracle data delivered to or accessed by U.S. Government end users are"commercial computer software" or "commercial computer software documentation" pursuant to the applicableFederal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, reproduction,duplication, release, display, disclosure, modification, preparation of derivative works, and/or adaptation of i) Oracleprograms (including any operating system, integrated software, any programs embedded, installed or activatedon delivered hardware, and modifications of such programs), ii) Oracle computer documentation and/or iii) otherOracle data, is subject to the rights and limitations specified in the license contained in the applicable contract. Theterms governing the U.S. Government's use of Oracle cloud services are defined by the applicable contract for suchservices. No other rights are granted to the U.S. Government.

This software or hardware is developed for general use in a variety of information management applications. It is notdeveloped or intended for use in any inherently dangerous applications, including applications that may create a riskof personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible totake all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporationand its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerousapplications.

Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of theirrespective owners.

Intel and Intel Inside are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are usedunder license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Epyc, and the AMDlogo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of TheOpen Group.

This software or hardware and documentation may provide access to or information about content, products, andservices from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim allwarranties of any kind with respect to third-party content, products, and services unless otherwise set forth in anapplicable agreement between you and Oracle. Oracle Corporation and its affiliates will not be responsible for anyloss, costs, or damages incurred due to your access to or use of third-party content, products, or services, except asset forth in an applicable agreement between you and Oracle.

Abstract

Oracle® Linux 8: Managing Shared File Systems provides tasks for managing shared file systems in Oracle Linux 8.

Document generated on: 2020-06-08 (revision: 10129)

Table of ContentsPreface .............................................................................................................................................. v1 About Shared File System Management in Oracle Linux ................................................................... 12 Managing the Network File System in Oracle Linux .......................................................................... 3

2.1 About NFS ........................................................................................................................... 32.1.1 Supported Versions of NFS ........................................................................................ 32.1.2 About NFS Services .................................................................................................. 4

2.2 Configuring an NFS Server ................................................................................................... 52.2.1 Configuring an NFS Server by Editing the /etc/exports File ........................................... 52.2.2 Configuring an NFS Server by Using the exportfs Command ........................................ 7

2.3 Mounting an NFS File System .............................................................................................. 83 Managing the Oracle Cluster File System Version 2 in Oracle Linux .................................................. 9

3.1 About OCFS2 ....................................................................................................................... 93.2 Maximum File System Size Requirements for OCFS2 .......................................................... 113.3 Installing and Configuring an OCFS2 Cluster ....................................................................... 11

3.3.1 Preparing a Cluster for OCFS2 ................................................................................ 113.3.2 Configuring the Firewall for the Cluster ..................................................................... 123.3.3 Configuring the Cluster Software .............................................................................. 123.3.4 Creating the Configuration File for the Cluster Stack .................................................. 133.3.5 Configuring the Cluster Stack ................................................................................... 153.3.6 Configuring the Kernel for Cluster Operation ............................................................. 173.3.7 Commands for Administering the Cluster Stack ......................................................... 17

3.4 Administering OCFS2 Volumes ........................................................................................... 183.4.1 Commands for Creating OCFS2 Volumes ................................................................. 183.4.2 Suggested Cluster Size Settings ............................................................................... 193.4.3 Creating OCFS2 Volumes ........................................................................................ 193.4.4 Mounting OCFS2 Volumes ....................................................................................... 203.4.5 Querying and Changing Volume Parameters ............................................................. 20

3.5 Creating a Local OCFS2 File System .................................................................................. 203.6 Troubleshooting OCFS2 Issues ........................................................................................... 21

3.6.1 Recommended Debugging Tools and Practices ......................................................... 213.6.2 Mounting the debugfs File System ............................................................................ 213.6.3 Configuring OCFS2 Tracing ..................................................................................... 223.6.4 Debugging File System Locks .................................................................................. 233.6.5 Configuring the Behavior of Fenced Nodes ............................................................... 24

3.7 OCFS2 Use Cases ............................................................................................................. 253.7.1 Load Balancing Use Case ........................................................................................ 253.7.2 Oracle Real Application Cluster Use Case ................................................................ 253.7.3 Oracle Database Use Case ...................................................................................... 25

4 Managing Samba in Oracle Linux .................................................................................................. 274.1 About Samba ..................................................................................................................... 27

4.1.1 About Samba Services ............................................................................................. 274.1.2 About the Samba Configuration File ......................................................................... 28

4.2 Configuring and Using Samba ............................................................................................. 294.2.1 Testing Samba Configuration by Using the testparm Command .................................. 294.2.2 Configuring a Samba Stand-Alone Server ................................................................. 294.2.3 Configuring a Samba Stand-Alone Server Within a Workgroup ................................... 304.2.4 Adding a Samba Server as a Member of an AD Domain ............................................ 31

4.3 Accessing Samba Shares ................................................................................................... 324.3.1 Accessing Samba Shares From a Windows Client ..................................................... 324.3.2 Accessing Samba Shares From an Oracle Linux Client .............................................. 33

iii

iv

PrefaceOracle® Linux 8: Managing Shared File Systems provides information about managing shared file systemsin Oracle Linux 8.

Audience

This document is intended for administrators who need to configure and administer Oracle Linux. It isassumed that readers are familiar with web technologies and have a general understanding of using theLinux operating system, including knowledge of how to use a text editor such as emacs or vim, essentialcommands such as cd, chmod, chown, ls, mkdir, mv, ps, pwd, and rm, and using the man command toview manual pages.

Document Organization

The document is organized into the following chapters:

• Chapter 1, About Shared File System Management in Oracle Linux

• Chapter 2, Managing the Network File System in Oracle Linux describes tasks for the distributedNetwork File System (NFS), including instructions for setting up NFS servers and clients.

• Chapter 3, Managing the Oracle Cluster File System Version 2 in Oracle Linux describes how toconfigure and use the Oracle Cluster File System Version 2 (OCFS2) file system in Oracle Linux.

• Chapter 4, Managing Samba in Oracle Linux describes tasks for the Samba shared file system, includinginstructions for setting up Samba servers.

Related Documents

The documentation for this product is available at:

https://docs.oracle.com/en/operating-systems/linux.html.

Conventions

The following text conventions are used in this document:

Convention Meaning

boldface Boldface type indicates graphical user interface elements associated with anaction, or terms defined in text or the glossary.

italic Italic type indicates book titles, emphasis, or placeholder variables for whichyou supply particular values.

monospace Monospace type indicates commands within a paragraph, URLs, code inexamples, text that appears on the screen, or text that you enter.

Documentation Accessibility

For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program websiteathttps://www.oracle.com/corporate/accessibility/.

v

Access to Oracle Support

Access to Oracle Support

Oracle customers that have purchased support have access to electronic support through My OracleSupport. For information, visithttps://www.oracle.com/corporate/accessibility/learning-support.html#support-tab.

vi

Chapter 1 About Shared File System Management in OracleLinux

This chapter includes a brief description of shared files systems and includes information about thetypes of shared file systems that are provided in Oracle Linux 8 and how they are used. The subsequentchapters of this guide describe how to manage each of the shared file systems types that are described inthis chapter.

For information about managing local file systems in Oracle Linux, see Oracle® Linux 8: Managing LocalFile Systems.

A shared file system is a type of file system that enables multiple users to access the same files acrossdifferent operating systems or over a network at the same time. The shared file system approach providesmany benefits. Most notably, using shared file systems can improve performance and scalability andgreatly reduce the time that administrators spend managing data.

Oracle Linux 8 includes support for several file systems types, including the following distributed andshared file systems:

Network File System (NFS) Is a distributed file system that enables users and client systems toaccess files over a network, as though the files were on local storage.An NFS server can share directory hierarchies in its local file systemswith remote client systems over an IP-based network.

For more information about NFS, see Chapter 2, Managing the NetworkFile System in Oracle Linux.

Oracle Cluster File SystemVersion 2 (OCFS2)

Is a general-purpose, shared-disk file system that is intended foruse with clusters. OCFS2 offers high performance, as well as highavailability. When you use clustered file systems, all of the nodesunderstand the file system structure and the full file system is sharedacross all of the nodes.

For more information about OCFS2, see Chapter 3, Managing theOracle Cluster File System Version 2 in Oracle Linux.

Samba Is an open-source implementation of the Server Message Block(SMB) protocol that provides the capability for Oracle Linux systemsto interoperate with Microsoft Windows systems, as both a server anda client. You can use Samba for file sharing across different operatingsystems over a network.

For more information about Samba, see Chapter 4, Managing Samba inOracle Linux.

1

2

Chapter 2 Managing the Network File System in Oracle Linux

Table of Contents2.1 About NFS ................................................................................................................................... 3

2.1.1 Supported Versions of NFS ............................................................................................... 32.1.2 About NFS Services .......................................................................................................... 4

2.2 Configuring an NFS Server ........................................................................................................... 52.2.1 Configuring an NFS Server by Editing the /etc/exports File .................................................. 52.2.2 Configuring an NFS Server by Using the exportfs Command ............................................... 7

2.3 Mounting an NFS File System ...................................................................................................... 8

This chapter includes information about managing the Network File System (NFS) in Oracle Linux 8,including tasks for configuring, administering, and using NFS.

For information about local file system management in Oracle Linux, see Oracle® Linux 8: Managing LocalFile Systems.

2.1 About NFSNFS (Network File System) is a distributed file system that enables a client system to access files over anetwork, as though the files were on local storage.

An NFS server can share directory hierarchies in its local file systems with remote client systems over anIP-based network. After an NFS server exports a directory, and then NFS clients mount this directory, ifthey have been granted the appropriate permissions. To the client systems, the directory appears as ifit were a local directory. Benefits of using NFS include centralized storage provisioning, improved dataconsistency , and reliability.

2.1.1 Supported Versions of NFS

The following versions of NFS are supported in Oracle Linux 8:

• NFS version 3 (NFSv3), specified in RFC 1813.

• NFS version 4 (NFSv4), specified in RFC 7530.

• NFS version 4 minor version 1 (NFSv4.1), specified in RFC 5661.

• NFS version 4 minor version 2 (NFSv4.2), specified in RFC 7862 .

Note

NFSv2 is no longer supported.

NFSv3 provides safe, asynchronous writes and efficient error handling. NFSv3 also supports 64-bit filesizes and offsets, which enable clients to access more than 2 GB of file data.

NFSv3 relies on Remote Procedure Call (RPC) services, which are controlled by the rpcbind service.The rpcbind service responds to requests for an RPC service and then sets up connections for therequested service. In addition, separate services are used to handle locking and mounting protocols, asconfiguring a firewall to cope with the various ports that are used by all these services can be complex anderror-prone.

3

About NFS Services

Note

In previous Oracle Linux releases, NFSv3 was able to also use the User DatagramProtocol (UDP). However, in Oracle Linux 8, NFS over UDP is no longer supported.Further, UDP is disabled in the NFS server by default in this release.

NFSv4 is capable of working through firewalls, as well as the Internet. Also, NFSv4 does not require therpcbind service. In addition, NFSv4 supports access Control Lists (ACLs), and uses stateful operations.

NFSv4 requires the Transmission Control Protocol (TCP) running over an IP network. As mentioned,NFSv4 does not use rpcbind; as such, the NFS server listens on TCP port 2049 for service requests. Themounting and locking protocols are also integrated into the NFSv4 protocol, which means that separateservices are also not required for these protocols. These refinements make firewall configuration for NFSv4no more difficult than for a service such as HTTP.

Note that in Oracle Linux 8, NFS clients attempt to mount by using NFSv4.2 (the default version), but fallback to NFSv4.1 when the server does not support NFSv4.2. The mount later falls back to NFSv4.0 andthen to NFSv3.

2.1.2 About NFS Services

In Oracle Linux 8, NFS versions rely on Remote Procedure Calls (RPC) between clients and servers.To share or mount NFS file systems, the following required services work together, depending on whichversion of NFS is implemented. Note that all of these services are started automatically:

nfsd Is the server kernel module that services requests for shared NFS filesystems.

rpcbind Is a service that accepts port reservations from local RPC services,which are made available or advertised so that the correspondingremote RPC services can access them and also hat the client is allowedto access it.

rpc.mountd Is a process that is used by an NFS server to process mount requestsfrom NFSv3 clients. The service checks that the requested NFS share iscurrently exported by the NFS server.

rpc.nfsd Is a process that enables explicit NFS versions and protocols the serveradvertises to be defined.

lockd Is a kernel thread that runs on both clients and servers. The lockdprocess implements the Network Lock Manager (NLM) protocol, whichenables NFSv3 clients to lock files on the server. The daemon is startedautomatically whenever the NFS server is run and whenever an NFS filesystem is mounted.

rpc-statd Is a process that implements the Network Status Monitor (NSM) RPCprotocol, which notifies NFS clients when an NFS server is restartedwithout being gracefully brought down. The rpc-statd service isautomatically started by the nfs-server service. This service does notrequire configuration by the user and is not used with NFSv4.

rpc-idmapd Is a process that provides NFSv4 client and server upcalls, whichmap between on-the-wire NFSv4 names (strings in the form ofuser@domain) and local UIDs and GIDs. Note that for the idmapdprocess to function with NFSv4, you must configure the /etc/

4

Configuring an NFS Server

idmapd.conf file. Note that only NFSv4 uses the rpc-idmapdprocess.

Note

The mounting and locking protocols are incorporated into the NFSv4 protocol.Also, the server listens on TCP port 2049. For this reason, NFSv4 does not need tointeract with the rpcbind, lockd, and rpc-statd services. However, the nfs-mountd service is still required to set up exports on the NFS server; but, the serviceis not involved in any over-the-wire operations.

The rpc-idmapd service only handles upcalls from the kernel and is not itselfdirectly involved in any over-the-wire operations. The service, however, might makenaming service calls, which do result in over-the-wire lookups.

2.2 Configuring an NFS Server

There are two ways in which you can configure an NFS server in Oracle Linux 8:

• By editing the /etc/exports file manually.

Exports can also be added to files that you create in the /etc/exports.d directory.

• By using the exportfs command.

The following procedures describe both of these methods.

2.2.1 Configuring an NFS Server by Editing the /etc/exports File

The following steps describe how to configure an NFS server by editing the /etc/exports file.

Note

You can also add exports to files that you create in the /etc/exports.d directoryin a similar fashion.

1. Check that the nfs-utils package is installed on the system. If necessary, install the package asfollows:

# dnf install nfs-utils

2. Edit the /etc/exports file to define the directories that the server will make available for clients tomount, for example:

/var/folder 192.0.2.102(rw,async)/usr/local/apps *(all-squash,anonuid=501,anongid=501,ro)/var/projects/proj1 192.168.1.0/24(ro) mgmtpc(rw)

Each entry includes the local path to the exported directory, followed by a list of clients that can mountthe directory, with client-specific exports options in parentheses.

The information in the previous example is as follows:

• The client system with the IP address 192.0.2.102 can mount the /var/folder directory withread and write permissions. All writes to the disk are asynchronous, which means that the serverdoes not wait for write requests to be written to disk before responding to further requests from theclient.

5

Configuring an NFS Server by Editing the /etc/exports File

Note

No other clients are allowed to mount the /var/folder directory.

• All of the clients can mount the /usr/local/apps directory as read-only. All connecting users,including root users, are mapped to the local, unprivileged user with UID 501 and GID 501.

• All of the clients on the 192.168.1.0/24 subnet can mount the /var/projects/proj1directory as read-only and the client system named mgmtpc can mount the directory with read-writepermissions.

Note

There is no space between a client specifier and the parenthesized list ofoptions that apply to that client.

For more information, see the exports(5) manual page.

3. If the server will serve NFSv4 clients, edit the /etc/idmapd.conf file's definition for the Domainparameter to specify the DNS domain name of the server:

Domain = mydom.com

This setting prevents the owner and group from being unexpectedly listed as the anonymous useror group (nobody or nogroup) on NFS clients when the all_squash mount option has not beenspecified.

4. If you need to enable access through the firewall for NFSv4 clients only, use the following commands:

# firewall-cmd --zone=zone --add-service=nfs# firewall-cmd --permanent --zone=zone --add-service=nfs

This configuration assumes that rpc.nfsd listens for client requests on TCP port 2049, which it doesby default.

5. If you need to enable access through the firewall for NFSv3 and NFSv4 clients, do the following:

a. Edit the /etc/nfs.conf file to create port settings for handling network mount requests andstatus monitoring, as well as set the TCP port on which the network lock manager should listen, forexample:

# Ports that various services should listen on.

[mountd]port = 892

[statd]port = 662

[lockd]port = 32803

If any port is in use, use the lsof -i command to locate an unused port and then amend thesetting in the /etc/nfs.conf file, as appropriate.

b. Restart the firewall service and configure the firewall to allow NFSv3 connections:

# firewall-cmd --permanent --zone=zone \ --add-port=2049/tcp

6

Configuring an NFS Server by Using the exportfs Command

--add-port=111/tcp --add-port=32803/tcp --add-port=892/tcp --add-port=662/tcp

c. Shut down and then reboot the server.

# systemctl reboot

Note that NFS fails to start if one of the specified ports is in use and reports an error in /var/log/messages. Edit the /etc/nfs.conf file to use a different port number for the service that wasunable to start, and then attempt to restart the nfs-server service. You can use the rpcinfo -pcommand to confirm on which ports RPC services are listening.

6. Start the nfs-server service and configure the service to start following a system reboot:

# systemctl enable --now nfs-server

7. Display a list of the exported file systems by running the showmount -e command, for example:

# showmount -eExport list for host01.mydom.com/var/folder 192.0.2.102/usr/local/apps */var/projects/proj1 192.168.1.0/24 mgmtpc

You can also use the exportfs command on the server to display this information, for example:

# /usr/sbin/exportfs -v

Use the showmount -a command to display all of the current clients, as well as all of the file systemsthat they have mounted:

# showmount -amgmtpc.mydom.com:/var/projects/proj1

Note

To enable use of the showmount command from NFSv4 clients, MOUNTD_PORTmust be defined in the /etc/nfs.conf file and a firewall rule must enableaccess on this TCP port.

2.2.2 Configuring an NFS Server by Using the exportfs Command

You can also export or unexport directories by using the exportfs command. Using the exportfscommand enables the root user to export or unexport directories selectively, and eliminates the need torestart the NFS service. By providing the appropriate options, the exportfs command writes the exportedfile systems to the /var/lib/nfs/etab file. Changes to the list of exported file systems are effectiveimmediately because the nfs-mountd service refers to the etab file for determining access privileges toa file system.

The following are some of the options that you can specify with the exportfs command. Note that usingthe exportfs without any options displays a list of currently exported file systems:

-r Enables all of the directories listed in the /etc/exports file that areto be exported by constructing a new export list in the /var/lib/nfs/etab file. The -r option refreshes the export list with changes that aremade to the /etc/exports file.

7

Mounting an NFS File System

-a Enables all directories to be exported or unexported, which isdependent on other options that are passed to the exportfscommand. If no other options are specified, the command exports all ofthe file systems that are specified in the /etc/exports file.

-u Unexports all of the shared directories.

Note

The exportfs -ua command suspends NFSfile sharing, but keeps all NFS services running.To re-enable NFS sharing, use the exportfs -r command.

-v Specifies a verbose operation of the exportfs command, whichdisplays information about the file systems that are being exported orunexported in greater detail.

For more information, see the exportfs(8), exports(5), and showmount(8) manual pages.

2.3 Mounting an NFS File SystemTo mount an NFS file system on an Oracle Linux 8 client:

1. Check whether the nfs-utils package is installed on the system. If necessary, install the package asfollows:

# dnf install nfs-utils

2. Use the showmount -e command to display the file systems that the NFS server exports, forexample:

# showmount -e host01.mydom.comExport list for host01.mydom.com/var/folder 192.0.2.102/usr/local/apps */var/projects/proj1 192.168.1.0/24 mgmtpc

Note

Be aware that some servers do not allow querying of this information, but theserver may still be exporting NFS file systems.

3. Use the mount command to mount an exported NFS file system on an available mount point:

# mount -r -o nosuid host01.mydoc.com:/usr/local/apps /apps

Note that in most cases, when mounting an NFS file system, the -t nfs option can be omitted.

This example mounts the /usr/local/apps directory that is exported by host01.mydoc.comwith read-only permissions on /apps. The nosuid option prevents remote users from gaining higherprivileges by running a setuid program.

4. To configure the system to mount an NFS file system at boot time, add an entry for the file system tothe /etc/fstab file, as shown in the following example:

host01.mydoc.com:/usr/local/apps /apps nfs ro,nosuid 0 0

For more information, see the mount(8), nfs(5), and showmount(8) manual pages.

8

Chapter 3 Managing the Oracle Cluster File System Version 2 inOracle Linux

Table of Contents3.1 About OCFS2 ............................................................................................................................... 93.2 Maximum File System Size Requirements for OCFS2 .................................................................. 113.3 Installing and Configuring an OCFS2 Cluster ............................................................................... 11

3.3.1 Preparing a Cluster for OCFS2 ........................................................................................ 113.3.2 Configuring the Firewall for the Cluster ............................................................................. 123.3.3 Configuring the Cluster Software ...................................................................................... 123.3.4 Creating the Configuration File for the Cluster Stack .......................................................... 133.3.5 Configuring the Cluster Stack ........................................................................................... 153.3.6 Configuring the Kernel for Cluster Operation ..................................................................... 173.3.7 Commands for Administering the Cluster Stack ................................................................. 17

3.4 Administering OCFS2 Volumes ................................................................................................... 183.4.1 Commands for Creating OCFS2 Volumes ......................................................................... 183.4.2 Suggested Cluster Size Settings ....................................................................................... 193.4.3 Creating OCFS2 Volumes ................................................................................................ 193.4.4 Mounting OCFS2 Volumes ............................................................................................... 203.4.5 Querying and Changing Volume Parameters ..................................................................... 20

3.5 Creating a Local OCFS2 File System .......................................................................................... 203.6 Troubleshooting OCFS2 Issues ................................................................................................... 21

3.6.1 Recommended Debugging Tools and Practices ................................................................. 213.6.2 Mounting the debugfs File System .................................................................................... 213.6.3 Configuring OCFS2 Tracing ............................................................................................. 223.6.4 Debugging File System Locks .......................................................................................... 233.6.5 Configuring the Behavior of Fenced Nodes ....................................................................... 24

3.7 OCFS2 Use Cases ..................................................................................................................... 253.7.1 Load Balancing Use Case ................................................................................................ 253.7.2 Oracle Real Application Cluster Use Case ........................................................................ 253.7.3 Oracle Database Use Case .............................................................................................. 25

This chapter includes information about managing the Oracle Cluster File System Version 2 (OCFS2) inOracle Linux 8. The chapter includes tasks for configuring, administering, and troubleshooting OCFS2.

Note

In Oracle Linux 8, the OCFS2 file system type is supported on UnbreakableEnterprise Kernel (UEK) releases only, starting with Unbreakable Enterprise KernelRelease 6 (UEK R6).

For information about local file system management in Oracle Linux, see Oracle® Linux 8: Managing LocalFile Systems.

3.1 About OCFS2

OCFS2 (Oracle Cluster File System Version 2) is a general-purpose shared-disk file system that isintended for use with clusters. OCFS2 offers high performance, as well as high availability. It is alsopossible to mount an OCFS2 volume on a standalone, non-clustered system.

9

About OCFS2

Although it might appear that the ability to mount an ocfs2 file system locally has no benefits, whencompared to alternative file systems such as Ext4 or Btrfs, you can use the reflink command withOCFS2 to create copy-on-write clones of individual files. You can also use the cp --reflink commandin a similar way that you would on a Btrfs file system. Typically, such clones enable you to save disk spacewhen storing multiple copies of very similar files, such as virtual machine (VM) images or Linux Containers.In addition, mounting a local OCFS2 file system enables you to subsequently migrate it to a cluster filesystem without requiring any conversion. Note that when using the reflink command, the resulting filesystem behaves like a clone of the original files ystem, which means that their UUIDs are identical. Whenusing the reflink command to create a clone, you must change the UUID by using the tunefs.ocfs2command. See Section 3.4.5, “Querying and Changing Volume Parameters”.

Almost all applications can use OCFS2 as it provides local file-system semantics. Applications that arecluster-aware can use cache-coherent parallel I/O from multiple cluster nodes to balance activity acrossthe cluster, or they can use of the available file-system functionality to fail over and run on another node inthe event that a node fails.

The following are examples of some typical use cases for OCFS2:

• Oracle VM to host shared access to virtual machine images.

• Oracle VM and VirtualBox to enable Linux guest machines to share a file system.

• Oracle Real Application Cluster (RAC) in database clusters.

• Oracle E-Business Suite in middleware clusters.

The following OCFS2 features that make it a suitable choice for deployment in an enterprise-levelcomputing environment:

• Support for ordered and write-back data journaling that provides file system consistency in the event ofpower failure or system crash.

• Block sizes ranging from 512 bytes to 4 KB, and file-system cluster sizes ranging from 4 KB to 1 MB(both in increments of powers of 2). The maximum supported volume size is 16 TB, which correspondsto a cluster size of 4 KB. A volume size as large as 4 PB is theoretically possible for a cluster size of 1MB, although this limit has not been tested.

• Inclusion of nowait support for OCFS2 in Unbreakable Enterprise Kernel Release 6 (UEK R6).

When the nowait flag is specified, -EAGAIN is returned if the following checks fail for direct I/O: cannotget related locks immediately or blocks are not allocated at the write location, which triggers blockallocation and subsequently blocks I/O operations.

• Extent-based allocations for efficient storage of very large files.

• Optimized allocation support for sparse files, inline-data, unwritten extents, hole punching, reflinks, andallocation reservation for high performance and efficient storage.

• Indexing of directories to allow efficient access to a directory even if it contains millions of objects.

• Metadata checksums for the detection of corrupted inodes and directories.

• Extended attributes to allow an unlimited number of name:value pairs to be attached to file systemobjects such as regular files, directories, and symbolic links.

• Advanced security support for POSIX ACLs and SELinux in addition to the traditional file-accesspermission model.

10

Maximum File System Size Requirements for OCFS2

• Support for user and group quotas.

• Support for heterogeneous clusters of nodes with a mixture of 32-bit and 64-bit, little-endian (x86,x86_64, ia64) and big-endian (ppc64), architectures.

• An easy-to-configure, in-kernel cluster-stack (O2CB) with the Linux Distributed Lock Manager (DLM) formanaging concurrent access from the cluster nodes.

• Support for buffered, direct, asynchronous, splice, and memory-mapped I/O.

• A toolset that uses similar parameters as the ext3 file system.

For more information about OCFS2, visit https://oss.oracle.com/projects/ocfs2/documentation/.

3.2 Maximum File System Size Requirements for OCFS2

Starting with the Oracle Linux 8 Update 2 release, the OCFS2 file system is supported on systems that arerunning the Unbreakable Enterprise Kernel Release 6 (UEK R6) kernel.

The maximum file size and maximum file system size requirements for OCFS2 are as follows:

• Maximum file size:

4 PiB

• Maximum file system size:

4 PiB

3.3 Installing and Configuring an OCFS2 Cluster

The following procedures describe how to set up a cluster to use OCFS2.

3.3.1 Preparing a Cluster for OCFS2

For the best performance, each node in the cluster should have at least two network interfaces. Thefirst interface is connected to a public network to allow general access to the systems, while the secondinterface is used for private communications between the nodes and the cluster heartbeat. This secondinterface determines how the cluster nodes coordinate their access to shared resources and how theymonitor each other's state.

Note

Both network interfaces must be connected through a network switch. Additionally,you must ensure that all of the network interfaces are configured and workingbefore configuring the cluster.

You can choose from the following two cluster heartbeat configurations:

• Local heartbeat thread for each shared device (default heartbeat mode).

In this mode, a node starts a heartbeat thread when it mounts an OCFS2 volume and stops the threadwhen it unmounts the volume. There is a large CPU overhead on nodes that mount a large numberof OCFS2 volumes as each mount requires a separate heartbeat thread. Note that a large number ofmounts also increases the risk of a node fencing itself out of the cluster due to a heartbeat I/O timeout ona single mount.

11

Configuring the Firewall for the Cluster

• Global heartbeat on specific shared devices.

This mode enables you to configure any OCFS2 volume as a global heartbeat device, provided thatit occupies a whole disk device and not a partition. In this mode, the heartbeat to the device startswhen the cluster comes online and stops when the cluster goes offline. This mode is recommendedfor clusters that mount a large number of OCFS2 volumes. A node fences itself out of the cluster if aheartbeat I/O timeout occurs on more than half of the global heartbeat devices. To provide redundancyagainst failure of one of the devices, you should configure at least three global heartbeat devices.

The following figure shows a cluster of four nodes that are connected by using a network switch to a LANand a network storage server. The nodes and storage server are also connected by using a switch to aprivate network that is used for the local cluster heartbeat.

Figure 3.1 Cluster Configuration by Using a Private Network

Although it is possible to configure and use OCFS2 without using a private network, note that such aconfiguration increases the probability of a node fencing itself out of the cluster due to an I/O heartbeattimeout.

3.3.2 Configuring the Firewall for the Cluster

Configure or disable the firewall on each node to allow access on the interface that the cluster will use forprivate cluster communication. By default, the cluster uses both TCP and UDP over port 7777.

For example, to allow incoming TCP connections and UDP datagrams on port 7777, you would use thefollowing command:

# firewall-cmd --zone=zone --add-port=7777/tcp --add-port=7777/udp# firewall-cmd --permanent --zone=zone --add-port=7777/tcp --add-port=7777/udp

3.3.3 Configuring the Cluster Software

Ideally, each node should be running the same version of OCFS2 software and a compatible UEK release.It is possible for a cluster to run with mixed versions of the OCFS2 and UEK software; for example, whileyou are performing a rolling update of a cluster. The cluster node that is running the lowest version of thesoftware determines the set of usable features.

12

Creating the Configuration File for the Cluster Stack

Use the dnf command to install or upgrade the following packages to the same version on each node:

• kernel-uek

• ocfs2-tools

Note

If you want to use the global heartbeat feature, you need to install the ocfs2-tools-1.8.0-11 or later package.

3.3.4 Creating the Configuration File for the Cluster Stack

You create the configuration file by using the o2cb command or by using a text editor.

To configure the cluster stack by using the o2cb command:

1. Create a cluster definition by using the following command:

# o2cb add-cluster cluster_name

For example, you would define a cluster named mycluster with four nodes as follows:

# o2cb add-cluster mycluster

The previous command creates the /etc/ocfs2/cluster.conf configuration file, if it does notalready exist.

2. For each node, define the node as follows:

# o2cb add-node cluster_name node_name --ip ip_address

The name of the node must be same as the value of system's HOSTNAME that is configured in the /etc/sysconfig/network file. The IP address will be used by the node for private communication inthe cluster.

For example, you would use the following command to define a node named node0, with the IPaddress 10.1.0.100, in the cluster mycluster:

# o2cb add-node mycluster node0 --ip 10.1.0.100

Note

OCFS2 only supports IPv4 addresses.

3. If you want the cluster to use global heartbeat devices, run the following commands:

# o2cb add-heartbeat cluster_name device1...# o2cb heartbeat-mode cluster_name global

Important

You must configure the global heartbeat feature to use whole disk devices. Youcannot configure a global heartbeat device on a disk partition.

For example, you would use /dev/sdd, /dev/sdg, and /dev/sdj as global heartbeat devices bytyping the following commands:

13

Creating the Configuration File for the Cluster Stack

# o2cb add-heartbeat mycluster /dev/sdd# o2cb add-heartbeat mycluster /dev/sdg# o2cb add-heartbeat mycluster /dev/sdj# o2cb heartbeat-mode mycluster global

4. Copy the cluster /etc/ocfs2/cluster.conf file to each node in the cluster.

5. Restart the cluster stack for the changes you made to the cluster configuration file to take effect.

The following is a typical example of the /etc/ocfs2/cluster.conf file. This particular configurationdefines a 4-node cluster named mycluster, with a local heartbeat:

node: name = node0 cluster = mycluster number = 0 ip_address = 10.1.0.100 ip_port = 7777

node: name = node1 cluster = mycluster number = 1 ip_address = 10.1.0.101 ip_port = 7777

node: name = node2 cluster = mycluster number = 2 ip_address = 10.1.0.102 ip_port = 7777

node: name = node3 cluster = mycluster number = 3 ip_address = 10.1.0.103 ip_port = 7777

cluster: name = mycluster heartbeat_mode = local node_count = 4

If you configure your cluster to use a global heartbeat, the file also include entries for the global heartbeatdevices, as shown in the following example:

node: name = node0 cluster = mycluster number = 0 ip_address = 10.1.0.100 ip_port = 7777

node: name = node1 cluster = mycluster number = 1 ip_address = 10.1.0.101 ip_port = 7777

node: name = node2 cluster = mycluster

14

Configuring the Cluster Stack

number = 2 ip_address = 10.1.0.102 ip_port = 7777

node: name = node3 cluster = mycluster number = 3 ip_address = 10.1.0.103 ip_port = 7777

cluster: name = mycluster heartbeat_mode = global node_count = 4

heartbeat: cluster = mycluster region = 7DA5015346C245E6A41AA85E2E7EA3CF

heartbeat: cluster = mycluster region = 4F9FBB0D9B6341729F21A8891B9A05BD

heartbeat: cluster = mycluster region = B423C7EEE9FC426790FC411972C91CC3

The cluster heartbeat mode is now shown as global and the heartbeat regions are represented by theUUIDs of their block devices.

If you edit the configuration file manually, ensure that you use the following layout:

• The cluster:, heartbeat:, and node: headings must start in the first column.

• Each parameter entry must be indented by one tab space.

• A blank line must separate each section that defines the cluster, a heartbeat device, or a node.

3.3.5 Configuring the Cluster Stack

When configuring a cluster stack, there are several values for which you are prompted. Refer to thefollowing table for the values that need to provide.

Prompt Description

Load O2CB driver on boot (y/n) Specify whether the cluster stack driver should be loaded atboot time. The default response is n.

Cluster stack backing O2CB Name of the cluster stack service. The default and usualresponse is o2cb.

Cluster to start at boot (Enter"none" to clear)

Enter the name of your cluster that you defined in the clusterconfiguration file, /etc/ocfs2/cluster.conf.

Specify heartbeat dead threshold(>=7)

Number of 2-second heartbeats that must elapse withoutresponse before a node is considered dead. To calculatethe value to enter, divide the required threshold time periodby 2 and then add 1. For example, to set the threshold timeperiod to 120 seconds, enter a value of 61. The defaultvalue is 31, which corresponds to a threshold time period of60 seconds.

15

Configuring the Cluster Stack

Prompt DescriptionNote

If your system uses multipathedstorage, the recommended value is61 or greater.

Specify network idle timeout inms (>=5000)

Time in milliseconds that must elapse before a networkconnection is considered dead. The default value is 30,000milliseconds.

Note

For bonded network interfaces,the recommended value is 30,000milliseconds or greater.

Specify network keepalive delayin ms (>=1000)

Maximum delay in milliseconds between sending keepalivepackets to another node. The default and recommendedvalue is 2,000 milliseconds.

Specify network reconnect delayin ms (>=2000)

Minimum delay in milliseconds between reconnectionattempts if a network connection goes down. The defaultand recommended value is 2,000 milliseconds.

Follow these steps to configure the cluster stack:

1. On each node of the cluster, run the following command:

# /sbin/o2cb.init configure

Verify the settings for the cluster stack.

# /sbin/o2cb.init statusDriver for "configfs": LoadedFilesystem "configfs": MountedStack glue driver: LoadedStack plugin "o2cb": LoadedDriver for "ocfs2_dlmfs": LoadedFilesystem "ocfs2_dlmfs": MountedChecking O2CB cluster "mycluster": Online Heartbeat dead threshold: 61 Network idle timeout: 30000 Network keepalive delay: 2000 Network reconnect delay: 2000 Heartbeat mode: LocalChecking O2CB heartbeat: Active

In the previous example, the cluster is online and is using the local heartbeat mode. If no volumes havebeen configured, the O2CB heartbeat is shown as Not active, rather than Active.

The following example shows the command output for an online cluster that is using three globalheartbeat devices:

# /sbin/o2cb.init statusDriver for "configfs": LoadedFilesystem "configfs": MountedStack glue driver: LoadedStack plugin "o2cb": LoadedDriver for "ocfs2_dlmfs": LoadedFilesystem "ocfs2_dlmfs": MountedChecking O2CB cluster "mycluster": Online

16

Configuring the Kernel for Cluster Operation

Heartbeat dead threshold: 61 Network idle timeout: 30000 Network keepalive delay: 2000 Network reconnect delay: 2000 Heartbeat mode: GlobalChecking O2CB heartbeat: Active 7DA5015346C245E6A41AA85E2E7EA3CF /dev/sdd 4F9FBB0D9B6341729F21A8891B9A05BD /dev/sdg B423C7EEE9FC426790FC411972C91CC3 /dev/sdj

2. Configure the o2cb and ocfs2 services so that they start at boot time after networking is enabled.

# systemctl enable o2cb# systemctl enable ocfs2

These settings enable the node to mount OCFS2 volumes automatically when the system starts.

3.3.6 Configuring the Kernel for Cluster Operation

To ensure the correct operation of a cluster, you must configure the required kernel settings, as describedin the following table.

Kernel Setting Description

panic STOPPED HERE. Specifies the number of seconds after a panic before a systemautomatically resets itself.

If the value is 0, the system hangs, which allows you to collect detailed informationabout the panic for troubleshooting. This is the default value.

To enable automatic reset, set a non-zero value. If you require a memory image(vmcore), allow enough time for Kdump to create this image. The suggested value is30 seconds, although large systems will require a longer time.

panic_on_oops Specifies that a system must panic if a kernel oops occurs. If a kernel thread requiredfor cluster operation crashes, the system must reset itself. Otherwise, another nodemight not be able to tell whether a node is slow to respond or unable to respond,causing cluster operations to hang.

1. On each node, set the recommended values for panic and panic_on_oops, for example:

# sysctl kernel.panic = 30# sysctl kernel.panic_on_oops = 1

2. To make the change persist across reboots, add the following entries to the /etc/sysctl.conf file:

# Define panic and panic_on_oops for cluster operationkernel.panic = 30kernel.panic_on_oops = 1

3.3.7 Commands for Administering the Cluster Stack

There are several commands that you can use to administer the cluster stack. The following tabledescribes the commands for performing various operations on the cluster stack.

Command Description

/sbin/o2cb.init status Check the status of the cluster stack.

/sbin/o2cb.init online Start the cluster stack.

/sbin/o2cb.init offline Stop the cluster stack.

17

Administering OCFS2 Volumes

Command Description

/sbin/o2cb.init unload Unload the cluster stack.

3.4 Administering OCFS2 Volumes

The following tasks describe how to administer OCFS2 volumes.

3.4.1 Commands for Creating OCFS2 Volumes

You use the mkfs.ocfs2 command to create an OCFS2 volume on a device. If you want to label thevolume and mount it by specifying the label, the device must correspond to a partition. You cannot mountan unpartitioned disk device by specifying a label.

The following table describes some useful options that you can use when creating an OCFS2 volume.

Command Option Description

-b block-size

--block-size block-size

Specifies the unit size for I/O transactions to and from the file system,and the size of inode and extent blocks. The supported block sizes are512 (512 bytes), 1K, 2K, and 4K. The default and recommended blocksize is 4K (4 kilobytes).

-C cluster-size

--cluster-size cluster-size

Specifies the unit size for space used to allocate file data. Thesupported cluster sizes are 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K,and 1M (1 megabyte). The default cluster size is 4K (4 kilobytes).

--fs-feature-level=feature-level

Enables you select a set of file-system features:

default Enables support for the sparsefiles, unwritten extents, and inlinedata features.

max-compat Enables only those features thatare understood by older versions ofOCFS2.

max-features Enables all features that OCFS2currently supports.

--fs_features=feature Enables you to enable or disable individual features such as supportfor sparse files, unwritten extents, and backup superblocks. For moreinformation, see the mkfs.ocfs2(8) manual page.

-J size=journal-size

--journal-optionssize=journal-size

Specifies the size of the write-ahead journal. If not specified, the sizeis determined from the file system usage type that you specify to the -T option, and, otherwise, from the volume size. The default size of thejournal is 64M (64 MB) for datafiles, 256M (256 MB) for mail, and128M (128 MB) for vmstore.

-L volume-label

--label volume-label

Specifies a descriptive name for the volume that allows you to identifyit easily on different cluster nodes.

-N number

--node-slots number

Determines the maximum number of nodes that can concurrentlyaccess a volume, which is limited by the number of node slots forsystem files such as the file-system journal. For best performance,set the number of node slots to at least twice the number of nodes. If

18

Suggested Cluster Size Settings

Command Option Descriptionyou subsequently increase the number of node slots, performance cansuffer because the journal will no longer be contiguously laid out onthe outer edge of the disk platter.

-T file-system-usage-type Specifies the type of usage for the file system:

datafiles Database files are typically fewin number, fully allocated, andrelatively large. Such files requirefew metadata changes, and do notbenefit from having a large journal.

mail Mail server files are typically manyin number, and relatively small.Such files require many metadatachanges, and benefit from having alarge journal.

vmstore Virtual machine image files aretypically few in number, sparselyallocated, and relatively large. Suchfiles require a moderate number ofmetadata changes and a mediumsized journal.

3.4.2 Suggested Cluster Size Settings

The following table provides suggested recommendations for minimum cluster size settings for different filesystem size ranges.

File System Size Suggested Minimum Cluster Size

1 GB - 10 GB 8K

10GB - 100 GB 16K

100 GB - 1 TB 32K

1 TB - 10 TB 64K

10 TB - 16 TB 128K

3.4.3 Creating OCFS2 Volumes

When creating OCFS2 volumes, keep the following additional points in mind:

• Do not create an OCFS2 volume on an LVM logical volume, as LVM is not cluster-aware.

• You cannot change the block and cluster size of an OCFS2 volume after you have created it. You canuse the tunefs.ocfs2 command to modify other settings for the file system, with certain restrictions.For more information, see the tunefs.ocfs2(8) manual page.

• If you intend that the volume store database files, do not specify a cluster size that is smaller than theblock size of the database.

• The default cluster size of 4 KB is not suitable if the file system is larger than a few gigabytes.

The following examples show some of the ways in which you can create an OCFS2 volume.

19

Mounting OCFS2 Volumes

Create an OCFS2 volume on /dev/sdc1 labeled myvol using all of the default settings for generic usageon file systems that are no larger than a few gigabytes. The default values are a 4 KB block and clustersize, eight node slots, a 256 MB journal, and support for default file-system features:

# mkfs.ocfs2 -L "myvol" /dev/sdc1

Create an OCFS2 volume on /dev/sdd2 labeled as dbvol for use with database files. In this case, thecluster size is set to 128 KB and the journal size to 32 MB.

# mkfs.ocfs2 -L "dbvol" -T datafiles /dev/sdd2

Create an OCFS2 volume on /dev/sde1, with a 16 KB cluster size, a 128 MB journal, 16 node slots, andsupport enabled for all features except refcount trees.

# mkfs.ocfs2 -C 16K -J size=128M -N 16 --fs-feature-level=max-features \ --fs-features=norefcount /dev/sde1

3.4.4 Mounting OCFS2 Volumes

Specify the _netdev option in the /etc/fstab file if you want the system to mount an OCFS2 volume atboot time after networking is started and unmount the file system before networking is stopped, as shownin the following example:

myocfs2vol /dbvol1 ocfs2 _netdev,defaults 0 0

Note

For the file system to mount, you must enable the o2cb and ocfs2 services to startafter networking is started. See Section 3.3.5, “Configuring the Cluster Stack”.

3.4.5 Querying and Changing Volume Parameters

Use the tunefs.ocfs2 command to query or change volume parameters.

For example, to find out the label, UUID, and number of node slots for a volume, you would use thefollowing command:

# tunefs.ocfs2 -Q "Label = %V\nUUID = %U\nNumSlots =%N\n" /dev/sdbLabel = myvolUUID = CBB8D5E0C169497C8B52A0FD555C7A3ENumSlots = 4

You would generate a new UUID for a volume by using the following command:

# tunefs.ocfs2 -U /dev/sda# tunefs.ocfs2 -Q "Label = %V\nUUID = %U\nNumSlots =%N\n" /dev/sdbLabel = myvolUUID = 48E56A2BBAB34A9EB1BE832B3C36AB5CNumSlots = 4

3.5 Creating a Local OCFS2 File System

Note

The OCFS2 file system type is supported on the Unbreakable Enterprise Kernel(UEK) release only.

The following procedure describes how to create an OCFS2 file system to be mounted locally, which is notassociated with a cluster.

20

Troubleshooting OCFS2 Issues

To create an OCFS2 file system that is to be mounted locally, use the following command syntax:

# mkfs.ocfs2 -M local --fs-features=local -N 1 [options] device

For example, you would create a locally mountable OCFS2 volume on /dev/sdc1, with one node slot andthe label localvol, as follows:

# mkfs.ocfs2 -M local --fs-features=local -N 1 -L "localvol" /dev/sdc1

You can use the tunefs.ocfs2 utility to convert a local OCTFS2 file system to cluster use, as follows:

# umount /dev/sdc1# tunefs.ocfs2 -M cluster --fs-features=cluster -N 8 /dev/sdc1

The previous example also increases the number of node slots from 1 to 8, to allow up to eight nodes tomount the file system.

3.6 Troubleshooting OCFS2 IssuesRefer to the following information when investigating how to resolve issues that you might encounter whenadministering OCFS2.

3.6.1 Recommended Debugging Tools and Practices

You can use the following tools to troubleshoot OCFS2 issues:

• It is recommended that you set up netconsole on the nodes to capture an oops trace.

• You can use the tcpdump command to capture the DLM's network traffic between nodes. For example,to capture TCP traffic on port 7777 for the private network interface em2, you could use the followingcommand:

# tcpdump -i em2 -C 10 -W 15 -s 10000 -Sw /tmp/`hostname -s`_tcpdump.log \ -ttt 'port 7777' &

• The debugfs.ocfs2 command tarcesevents in the OCFS2 driver, determine lock statuses, walkdirectory structures, examine inodes, and so on. This command is similar in behavior to the debugfscommand that is used for the ext3 file system.

For more information, see the debugfs.ocfs2(8) manual page.

• Use the o2image command to save an OCFS2 file system's metadata, including information aboutinodes, file names, and directory names, to an image file on another file system. Because theimage file contains only metadata, it is much smaller than the original file system. You can use thedebugfs.ocfs2 command to open the image file and analyze the file system layout to determine thecause of a file system corruption or performance problem.

For example, to create the image /tmp/sda2.img from the OCFS2 file system on the device /dev/sda2, you would use the following command:

# o2image /dev/sda2 /tmp/sda2.img

For more information, see the o2image(8) manual page.

3.6.2 Mounting the debugfs File System

OCFS2 uses the debugfs file system to enable userspace access to information about its in-kernel state.Note that you must mount the debugfs file system to use the debugfs.ocfs2 command.

21

Configuring OCFS2 Tracing

For example, to mount the debugfs file system, add the following line to the /etc/fstab file:

debugfs /sys/kernel/debug debugfs defaults 0 0

Then, run the mount -a command.

3.6.3 Configuring OCFS2 Tracing

You can use the following commands and methods to trace issues in OCFS2.

3.6.3.1 Commands for Tracing OCFS2 Issues

The following table describes several commands that are useful for tracing issues.

Command Description

debugfs.ocfs2 -l List all of the trace bits and their statuses.

debugfs.ocfs2 -l SUPER allow Enable tracing for the superblock.

debugfs.ocfs2 -l SUPER off Disable tracing for the superblock.

debugfs.ocfs2 -l SUPER deny Disallow tracing for the superblock, even if it isimplicitly enabled by another tracing mode setting.

debugfs.ocfs2 -l HEARTBEAT \

ENTRY EXIT allow

Enable heartbeat tracing.

debugfs.ocfs2 -l HEARTBEAT off \

ENTRY EXIT deny

Disable heartbeat tracing. Note that the ENTRY andEXIT parameters are set to deny, as they exist inall trace paths.

debugfs.ocfs2 -l ENTRY EXIT \

NAMEI INODE allow

Enable tracing for the file system.

debugfs.ocfs2 -l ENTRY EXIT \

deny NAMEI INODE allow

Disable tracing for the file system.

debugfs.ocfs2 -l ENTRY EXIT \

DLM DLM_THREAD allow

Enable tracing for the DLM.

debugfs.ocfs2 -l ENTRY EXIT \

deny DLM DLM_THREAD allow

Disable tracing for the DLM.

3.6.3.2 OCFS2 Tracing Methods and Examples

One method that you can use to obtain a trace is to first enable the trace, sleep for a short while, and thendisable the trace. As shown in the following example, to avoid unnecessary output, you should reset thetrace bits to their default settings after you have finished tracing:

# debugfs.ocfs2 -l ENTRY EXIT NAMEI INODE allow && sleep 10 && \ debugfs.ocfs2 -l ENTRY EXIT deny NAMEI INODE off

To limit the amount of information that is displayed, enable only the trace bits that are relevant todiagnosing the problem.

If a specific file system command, such as mv, is causing an error, you might use commands such as thoseused in the following example to trace the error:

22

Debugging File System Locks

# debugfs.ocfs2 -l ENTRY EXIT NAMEI INODE allow# mv source destination & CMD_PID=$(jobs -p %-)# echo $CMD_PID# debugfs.ocfs2 -l ENTRY EXIT deny NAMEI INODE off

Because the trace is enabled for all mounted OCFS2 volumes, knowing the correct process ID can helpyou to interpret the trace.

For more information, see the debugfs.ocfs2(8) manual page.

3.6.4 Debugging File System Locks

If an OCFS2 volume hangs, you can use the following procedure to determine which locks are busy andwhich processes are likely to be holding the locks.

In the following procedure, the Lockres value refers to the lock name that is used by DLM, which is acombination of a lock-type identifier, inode number, and a generation number. The following table lists thevarious lock types and their associated identifier.

Table 3.1 DLM Lock Types

Identifier Lock Type

D File data

M Metadata

R Rename

S Superblock

W Read-write

1. Mount the debug file system.

# mount -t debugfs debugfs /sys/kernel/debug

2. Dump the lock statuses for the file system device, which is /dev/sdx1 in the following example:

# echo "fs_locks" | debugfs.ocfs2 /dev/sdx1 >/tmp/fslocks 62Lockres: M00000000000006672078b84822 Mode: Protected ReadFlags: Initialized AttachedRO Holders: 0 EX Holders: 0Pending Action: None Pending Unlock Action: NoneRequested Mode: Protected Read Blocking Mode: Invalid

3. Use the Lockres value from the output in the previous step to obtain the inode number and generationnumber for the lock.

# echo "stat <M00000000000006672078b84822>" | debugfs.ocfs2 -n /dev/sdx1Inode: 419616 Mode: 0666 Generation: 2025343010 (0x78b84822)...

4. Determine the file system object to which the inode number relates, for example:

# echo "locate <419616>" | debugfs.ocfs2 -n /dev/sdx1419616 /linux-2.6.15/arch/i386/kernel/semaphore.c

5. Obtain the lock names that are associated with the file system object.

# echo "encode /linux-2.6.15/arch/i386/kernel/semaphore.c" | \ debugfs.ocfs2 -n /dev/sdx1M00000000000006672078b84822 D00000000000006672078b84822 W00000000000006672078b84822

23

Configuring the Behavior of Fenced Nodes

In the previous example, a metadata lock, a file data lock, and a read-write lock are associated with thefile system object.

6. Determine the DLM domain of the file system.

# echo "stats" | debugfs.ocfs2 -n /dev/sdX1 | grep UUID: | while read a b ; do echo $b ; done82DA8137A49A47E4B187F74E09FBBB4B

7. Using the values of the DLM domain and the lock name that enables debugging for the DLM, run thefollowing command:

# echo R 82DA8137A49A47E4B187F74E09FBBB4B \ M00000000000006672078b84822 > /proc/fs/ocfs2_dlm/debug

8. Examine the debug messages.

# dmesg | tailstruct dlm_ctxt: 82DA8137A49A47E4B187F74E09FBBB4B, node=3, key=965960985 lockres: M00000000000006672078b84822, owner=1, state=0 last used: 0, on purge list: no granted queue: type=3, conv=-1, node=3, cookie=11673330234144325711, ast=(empty=y,pend=n), bast=(empty=y,pend=n) converting queue: blocked queue:

The DLM supports three lock modes: no lock (type=0), protected read (type=3), and exclusive(type=5). In the previous example, the lock is mastered by node 1 (owner=1) and node 3 has beengranted a protected-read lock on the file-system resource.

9. Use the following command to search for processes that are in an uninterruptable sleep state, whichare indicated by the D flag in the STAT column:

# ps -e -o pid,stat,comm,wchan=WIDE-WCHAN-COLUMN

Note that at least one of the processes that are in the uninterruptable sleep state is responsible for thehang on the other node.

If a process is waiting for I/O to complete, the problem could be anywhere in the I/O subsystem, from theblock device layer through the drivers, to the disk array. If the hang concerns a user lock (flock()), theproblem could lie with the application. If possible, kill the holder of the lock. If the hang is due to lack ofmemory or fragmented memory, you can free up memory by killing non-essential processes. The mostimmediate solution is to reset the node that is holding the lock. The DLM recovery process can then clearall of the locks owned by the dead node owned; thus, enabling the cluster to continue to operate.

3.6.5 Configuring the Behavior of Fenced Nodes

If a node with a mounted OCFS2 volume assumes that it is no longer in contact with the other clusternodes, it removes itself from the cluster. This process is called fencing. Fencing prevents other nodes fromhanging when attempting to access resources that are held by the fenced node. By default, a fenced noderestarts instead of panicking so that it can quickly rejoin the cluster. Under some circumstances, you mightwant a fenced node to panic instead of restarting. For example, you might want to use the netconsolecommand to view the oops stack trace or diagnose the cause of frequent reboots.

To configure a node to panic when it next fences, run the following command on the node after the clusterstarts:

# echo panic > /sys/kernel/config/cluster/cluster_name/fence_method

In the previous command, cluster_name is the name of the cluster.

24

OCFS2 Use Cases

To set the value after each system reboot, add this line to the /etc/rc.local file. To restore the defaultbehavior, use the reset value instead of the panic value.

3.7 OCFS2 Use Cases

The following are some typical use cases for OCFS2.

3.7.1 Load Balancing Use Case

You can use OCFS2 nodes to share resources between client systems. For example, the nodes couldexport a shared file system by using Samba or NFS. To distribute service requests between the nodes,you can use round-robin DNS, a network load balancer; or, you can specify which node should be used oneach client.

3.7.2 Oracle Real Application Cluster Use Case

Oracle Real Application Cluster (RAC) uses its own cluster stack, Cluster Synchronization Services (CSS).You can use O2CB in conjunction with CSS, but note that each stack is configured independently fortimeouts, nodes, and other cluster settings. You can use OCFS2 to host the voting disk files and the Oraclecluster registry (OCR), but not the grid infrastructure user's home, which must exist on a local file systemon each node.

Because both CSS and O2CB use the lowest node number as a tie breaker in quorum calculations, ensurethat the node numbers are the same in both clusters. If necessary, edit the O2CB configuration file, /etc/ocfs2/cluster.conf, to make the node numbering consistent. Then, update this file on all of thenodes. The change takes effect when the cluster is restarted.

3.7.3 Oracle Database Use Case

Specify the noatime option when mounting volumes that host Oracle datafiles, control files, redo logs,voting disk, and OCR. The noatime option disables unnecessary updates to the access time on theinodes.

Specify the nointr mount option to prevent signals interrupting I/O transactions that are in progress.

By default, the init.ora parameter filesystemio_options directs the database to perform directI/O to the Oracle datafiles, control files, and redo logs. You should also specify the datavolume mountoption for volumes that contain the voting disk and OCR. Do not specify this option for volumes that hostthe Oracle user's home directory or Oracle E-Business Suite.

To prevent database blocks from becoming fragmented across a disk, ensure that the file system clustersize is at minimum as large as the database block size, which is typically 8KB. If you specify the file systemusage type as datafiles when using the mkfs.ocfs2 command, the file system cluster size is set to128KB.

To enable multiple nodes to maximize throughput by concurrently streaming data to an Oracle datafile,OCFS2 deviates from the POSIX standard by not updating the modification time (mtime) on the diskwhen performing non-extending direct I/O writes. The value of mtime is updated in memory. However,OCFS2 does not write the value to disk unless an application extends or truncates the file or performs aoperation to change the file metadata, such as using the touch command. This behavior leads to resultswith different nodes reporting different time stamps for the same file. Use the following command to viewthe on-disk timestamp of a file:

# debugfs.ocfs2 -R "stat /file_path" device | grep "mtime:"

25

26

Chapter 4 Managing Samba in Oracle Linux

Table of Contents4.1 About Samba ............................................................................................................................. 27

4.1.1 About Samba Services ..................................................................................................... 274.1.2 About the Samba Configuration File ................................................................................. 28

4.2 Configuring and Using Samba ..................................................................................................... 294.2.1 Testing Samba Configuration by Using the testparm Command .......................................... 294.2.2 Configuring a Samba Stand-Alone Server ......................................................................... 294.2.3 Configuring a Samba Stand-Alone Server Within a Workgroup ........................................... 304.2.4 Adding a Samba Server as a Member of an AD Domain .................................................... 31

4.3 Accessing Samba Shares ........................................................................................................... 324.3.1 Accessing Samba Shares From a Windows Client ............................................................. 324.3.2 Accessing Samba Shares From an Oracle Linux Client ..................................................... 33

This chapter including information about managing Samba in Oracle Linux 8, including tasks for configuringSamba and accessing Samba shares on different platforms.

For information about local file system management in Oracle Linux, see Oracle® Linux 8: Managing LocalFile Systems.

4.1 About SambaSamab is an open-source implementation of the Server Message Block (SMB) protocol that enablesOracle Linux to interoperate with Microsoft Windows systems, as both a server and a client.

Samba implements the Distributed Computing Environment Remote Procedure Call (DCE RPC) protocolthat is used by Microsoft Windows to provision file and print services for Windows clients. Samba alsoenables Oracle Linux users to access files on Windows systems and includes capability for integrating witha Windows workgroup, NT4 domain, and an Active Directory (AD) domain.

Samba uses the NetBIOS over TCP/IP protocol, which allows computer applications that depend on theNetBIOS API to work on TCP/IP networks.

4.1.1 About Samba Services

The Samba server consists of three important daemons: smbd, nmbd, and winbindd.

• The smb service enables file sharing and printing services by using the SMB protocol. This serviceis also responsible for resource locking and for authenticating connecting users. The smb systemdservice starts and stops the smbd daemon.

• The nmbd service provides host name and IP resolution by using the NetBIOS over IPv4 protocol. Thenmbd service also enables browsing of the SMB network to locate domains, workgroups, hosts, fileshares, and printers. The nmb systemd service starts and stops the nmbd daemon.

• The winbindd daemon is a Name Service Switch (NSS) daemon for resolving AD Users and Groups.The daemon enables AD Users to securely access services that are hosted on the Samba server. Thewinbind systemd service starts and stops the winbindd daemon.

To use the smbd and nmbd services, you need to install the samba package on your system. To use thewinbindd service, install the samba-winbind package.

27

About the Samba Configuration File

Note that if you are setting up Samba as a domain member, you must start the winbindd service beforestarting the smbd service. Otherwise, domain users and groups are not available to the local system.

4.1.2 About the Samba Configuration File

Samba uses the /etc/samba/smb.conf file to manage Samba configuration. This file consists ofseveral sections that you can configure to support the required services for a specific Samba configuration,for example:

[global]security = ADSrealm = MYDOM.REALMpassword server = krbsvr.mydom.comload printers = yesprinting = cupsprintcap name = cups

[printers]comment = All Printerspath = /var/spool/sambabrowseable = noguest ok = yeswritable = noprintable = yesprinter admin = root, @ntadmins, @smbprintadm

[homes]comment = User home directoriesvalid users = @smbusersbrowsable = nowritable = yesguest ok = no

[apps]comment = Shared /usr/local/apps directorypath = /usr/local/appsbrowsable = yeswritable = noguest ok = yes

• [global]

Contains settings for the Samba server. In the previous example, the server is assumed to be a memberof an AD domain that is running in native mode. Samba relies on tickets issued by the Kerberos server toauthenticate clients who want to access local services.

• [printers]

Specifies support for print services. The path parameter specifies the location of a spooling directorythat receives print jobs from Windows clients before submitting them to the local print spooler. Sambaadvertises all locally configured printers on the server.

• [homes]

Provides a personal share for each user in the smbusers group. The settings for browsable andwritable prevent other users from browsing home directories, while allowing full access to valid users.

• [apps]

Specifies a share named apps, which grants Windows users browsing and read-only permission to the /usr/local/apps directory.

28

Configuring and Using Samba

4.2 Configuring and Using SambaThe following tasks describe how to configure and use Samba in Oracle Linux 8.

4.2.1 Testing Samba Configuration by Using the testparm Command

After configuring the /etc/samba/smb.conf file, per the information that is provided in Section 4.1.2,“About the Samba Configuration File”, you can verify your Samba configuration by using the testparmcommand. The testparm command detects invalid parameters and values, as well as any incorrectsettings such as incorrect ID mapping. If the testparm command does not report any problems, theSamba services successfully load the configuration that is specified in the /etc/samba/smb.conf file.Note that the testparm command is not capable of testing whether configured services will be availableor work as expected. You should use the testparm command every time you make a change to yourSamba configuration.

The following example shows the type of output that might be displayed when you run the testparmcommand:

# testparmLoad smb config files from /etc/samba/smb.confLoaded services file OK.WARNING: The 'netbios name' is too long (max. 15 chars).

Server role: ROLE_STANDALONE

Press enter to see a dump of your service definitions...

If the testparm command reports any errors or misconfiguration in the /etc/samba/smb.conf file, fixthe problem, then re-run the command.

For more information, see the testparm(1) manual page.

4.2.2 Configuring a Samba Stand-Alone Server

To configure a Samba stand-alone server:

1. Install the samba and samba-winbind packages:

# dnf install samba samba-winbind

2. Edit the /etc/samba/smb.conf file and configure the various sections to support the services thatare required for your specific configuration.

For general information, see Section 4.1.2, “About the Samba Configuration File”

For specific instructions on configuring a Samba stand-alone server within a workgroup, seeSection 4.2.3, “Configuring a Samba Stand-Alone Server Within a Workgroup”.

For specific instructions on adding a Samba server as a member of an AD domain, see Section 4.2.4,“Adding a Samba Server as a Member of an AD Domain”.

3. (Optional) Configure file system sharing, as needed.

See Section 4.3.1, “Accessing Samba Shares From a Windows Client” and Section 4.3.2, “AccessingSamba Shares From an Oracle Linux Client” for instructions.

4. Test the configuration by running the testparm command as follows:

29

Configuring a Samba Stand-Alone Server Within a Workgroup

# testparm

If the command returns any errors or reports a misconfiguration, manually fix the errors in the /etc/samba/smb.conf file and then re-run the command. See Section 4.2.1, “Testing Samba Configurationby Using the testparm Command” for more information.

5. Configure the system firewall to enable incoming TCP connections to ports 139 and 445 and incomingUDP datagrams to ports 137 and 138:

# firewall-cmd --zone=zone \ --add-port=139/tcp --add-port=445/tcp --add-port=137-138/udp# firewall-cmd --permanent --zone=zone \ --add-port=139/tcp --add-port=445/tcp --add-port=137-138/udp

The nmdb daemon services NetBIOS Name Service requests on UDP port 137 and NetBIOS DatagramService requests on UDP port 138.

The default ports are 139 (used for SMB over NetBIOS over TCP) and port 445 (used for plain SMBover TCP).

6. (Optional) Add similar rules for other networks from which Samba clients can connect, as required.

7. Start and enable the smb service so the service starts following a system reboot:

# systemctl start smb# systemctl enable smb

Note

If you make changes to the /etc/samba/smb.conf file and any files that this filereferences, the smb service reloads the configuration automatically, after a delayof up to one minute. If necessary, you can force the smb service to reload the newconfiguration by sending a SIGHUP signal to the service daemon:

# killall -SIGHUP smbd

Making the smb service reload its configuration has no effect on any establishedconnections. You must restart the smb service. Otherwise, any existing users of theservice must disconnect and then reconnect.

For more information, see the smb.conf(5) and smbd(8) manual pages. See also http://www.samba.org/samba/docs/.

4.2.3 Configuring a Samba Stand-Alone Server Within a Workgroup

Windows systems that are on an enterprise network usually belong to either a workgroup or to a domain.

Workgroups are usually only configured on networks that connect a small number of computers. Aworkgroup environment is a peer-to-peer network, where the systems do not rely on each other forservices and there is no centralized management. User accounts, access control, and system resourcesare configured independently of each system. Note that such systems can share resources only if they areconfigured to do so.

A Samba server can act as a stand-alone server within a workgroup. To configure a stand-alone Sambaserver within a workgroup, follow the instructions inSection 4.2.2, “Configuring a Samba Stand-AloneServer”. For Step 2 of the procedure, configure the settings in the /etc/samba/smb.conf file as follows:

Configure the [global] section by using share-level security, as follows:

30

Adding a Samba Server as a Member of an AD Domain

[global]security = shareworkgroup = workgroup_namenetbios name = netbios_name

The client provides only a password to the server, and not a user name. Typically, each share isassociated with a valid users parameter. The server validates the password against the hashedpasswords that are stored in the /etc/passwd and /etc/shadow files, NIS, or LDAP for the listed users.Note that user-level security is preferred over share-level security, as shown in the following example:

[global]security = userworkgroup = workgroup_namenetbios name = netbios_name

In the user security model, a client must supply a valid user name and a password. This model supportsencrypted passwords. If the server successfully validates the client's user name and password, the clientcan mount multiple shares without being required to specify a password.

Use the smbpasswd command to create an entry for a user in the Samba password file, for example:

# smbpasswd -a guestNew SMB password: passwordRetype new SMB password: passwordAdded user guest.

Note

The user must already exist on the system. If permitted to log in to the server, theuser can use the smbpasswd command to change his or her password.

If a Windows user has a user name that is different from the user name on the Samba server, create amapping between these names in the /etc/samba/smbusers file, for example:

root = admin administrator rootnobody = guest nobody pcguest smbguesteddie = ejonesfiona = fchau

In the previous example, the first entry for each line is the user name on the Samba server. The entriesthat appear after the equal sign (=) are the equivalent to Windows user names.

Note

Only the user security model uses Samba passwords.

The server security model, where the Samba server relies on another server to authenticate user namesand passwords, is deprecated. This model has numerous security and interoperability issues.

4.2.4 Adding a Samba Server as a Member of an AD Domain

Typically, corporate networks configure domains to enable large numbers of networked systems to beadministered centrally. A domain is a group of trusted computers that share security and access control.Systems that are known as domain controllers provide centralized management and security. Windowsdomains are usually configured to use AD, which uses the Lightweight Directory Access Protocol (LDAP)to implement versions of Kerberos and DNS by providing authentication, access control to domainresources, and name services. Some Windows domains use Windows NT4 security, which does not useKerberos to perform authentication.

31

Accessing Samba Shares

Note

A Samba server can be a member of an AD or NT4 security domain, but it cannotoperate as a domain controller. As a domain member, a Samba server mustauthenticate itself with a domain controller; thus, it is controlled by the security rulesof the domain. The domain controller authenticates clients, while the Samba servercontrols access to printers and network shares.

In the Activity Directory Server (ADS) security model, Samba acts as a domain member server in an ADSrealm. Clients use Kerberos tickets for AD authentication. You must first configure Kerberos and then jointhe server to the domain, which creates a machine account for your server on the domain controller.

To add a Samba server to an AD domain:

1. Edit /etc/samba/smb.conf and configure the [global] section to use ADS:

[global]security = ADSrealm = KERBEROS.REALM

You might also have to specify the password server explicitly if different servers support AD servicesand Kerberos authentication:

password server = kerberos_server.your_domain

2. Install the krb5-workstation package:

# dnf install krb5-workstation

3. Create a Kerberos ticket for the Administrator account in the Kerberos domain, for example:

# kinit Administrator@MYDOMAIN.COM

This command creates the Kerberos ticket that is required to join the server to the AD domain.

4. Join the server to the AD domain:

# net ads join -S winads.mydom.com -U Administrator%password

In the previous example, the AD server is winads.mydom.com and password is the password for theAdministrator account.

The command creates an machine account in Active Directory for the Samba server and enables it tojoin the domain.

5. Restart the smb service:

# systemctl restart smb

4.3 Accessing Samba Shares

The following tasks describe how to access Samba shares from a Windows client and an Oracle Linuxclient.

4.3.1 Accessing Samba Shares From a Windows Client

To access a share on a Samba server from Windows, open Computer or Windows Explorer, and enter thehost name of the Samba server and the share name using the following format:

32

Accessing Samba Shares From an Oracle Linux Client

\\server_name\share_name

If you enter \\server_name, Windows displays the directories and printers that the server is sharing. Youcan also use the same syntax to map a network drive to a share name.

4.3.2 Accessing Samba Shares From an Oracle Linux Client

Note

To use the following commands, the samba-client and cifs-utils packagesmust be installed on the system.

You can use the findsmb command to query a subnet for Samba servers. The command displays the IPaddress, NetBIOS name, workgroup, operating system and version for each server that it finds.

Alternatively, you can use the smbtree command, which is a text-based SMB network browser thatdisplays the hierarchy of known domains, the servers in those domains, and the shares on those servers.

The GNOME desktop provides browser-based file managers that you can use to view Windows shares onthe network. Enter smb: in the location bar of a file manager to browse network shares.

To connect to a Windows share by using the command line, use the smbclient command:

$ smbclient //server_name/share_name [-U username]

After logging in, enter help at the smb:\> prompt to display a list of available commands.

To mount a Samba share, use a command similar to the following:

# mount -t cifs //server_name/share_name mountpoint -o credentials=credfile

In the previous command, the credentials file contains settings for username, password, and domain:

username=eddiepassword=clydenwdomain=MYDOMWKG

The argument to domain can be the name of a domain or a workgroup.

Caution

Because the credentials file contains a plain-text password, use chmod to make itreadable by only you, for example:

# chmod 400 credfile

If the Samba server is a domain member server in an AD domain, and your current login session wasauthenticated by the Kerberos server in the domain, you can use your existing session credentials byspecifying the sec=krb5 option instead of a credentials file:

# mount -t cifs //server_name/share_name mountpoint -o sec=krb5

For more information, see the findsmb(1), mount.cifs(8), smbclient(1), and smbtree(1)manual pages.

33

34