Not So Stupid Cluster Tricks

Mark Gonnelly

ATT Engineer

Novell, Inc

Mitchell Smith

ATT Engineer

Novell, Inc

Agenda

Clusters without shared storage

Clustering LDAP

Creating Virtual Servers

Adding storage to a cluster post-install

Perl CLI management

Clusters Without Shared Storage

Hardware Requirements

The following list specifies hardware requirements for installing Novell Cluster Services. These requirements represent the minimum hardware configuration. Additional hardware might be necessary depending on how you intend to use Novell Cluster Services.

A minimum of two NetWare® servers

At least 512 MB of memory on all servers in the cluster.

At least one local disk device (not shared) for the SYS: volume on each server

Selecting the Shared Media

What Can You Run?

LDAP

SLP DA

DHCP

DNS

FTP and Web Services

(If you sync data between the servers)

LDAP

Does LDAP require shared storage? No

Do you need to start an application? No

What do you need? Just an IP address

So the resource load script is just

ADD SECONDARY IP ADDRESS a.b.c.d

Appnote

Cluster-Enabling DHCP, DNS, and SLP Services in NetWare® 6

Dr. Frieder Schmidt and Hylton Leigh

http://developer.novell.com/research/

appnotes/2003/october/a031002.html

Virtual Servers?

Can you create a virtual server without a shared pool to cluster-enable?

Yes!

CLUSTER CVSBIND ADD VSERVERNAME IPADDRESS

NUDP ADD VSERVERNAME IPADDRESS

Create a new NCP server object in DS and point to the virtual server.

Adding Storage

Split Brain Detector Partition (SBD) – A partition

where the servers mark ticks and keep track of the

epoch numbers.

Requires one cylinder of unpartitioned space on a shared LUN.

May be a dedicated LUN

Can be mirrored for fault tolerance

SAN Requirements

Creating A New SBD Partition

On any node's console type CLUSTER DOWN

Unload Cluster Services from each node (ULDNCS.NCF)

Load (in order) CLSTRLIB.NLM, VLL.NLM, and SBDLIB.NLM on any node

LOAD SBD INSTALL

Say Yes/No to mirror the Cluster Services Partition, and select the device(s) where the partition should reside.

Switching the NCF Files

_LDNCSS.NCF – loads clustering with support for shared storage

_ULDNCSS.NCF – unloads clustering with support for shared storage

_LDNCS.NCF – loads clustering without support for shared storage

_ULDNCS.NCF – unloads clustering without support for shared storage

So we need to...

Replace LDNCS.NCF with _LDNCSS.NCF

Replace ULDNCS.NCF with _ULDNCSS.NCF

Adding a LUN

Create the LUN

Function of the SAN

Grant all servers access to LUN

Configure LUN masking

Scan for new devices

Set sharable for clustering flag

Cluster scan for new devices

Scan for new devices on all nodes

Perl CLI

NetWare Virtual File Services

Introduced in NetWare 6

Simple, ubiquitous interface

File Open, Read, Write, Close

All Platforms of all times can do this

XML interface

Scriptable

Remoteable

Full NetWare Security enforced

Must be Admin user

Exposed as “_admin” volume on each server

The _admin volume

Persistent Storage

I/O Request

I/O Request

Virtual File System

Normal File System

NLM Content generator

Features

A new _ADMIN:\Manage_Cluster (XML) virtual file interface that enables script based cluster management.

It's now possible to write a script to migrate a cluster resource from one server to another, for example, in automated cluster volume backup by migrating volumes to the server that has a direct attached tape device, and then back again after the backup has completed.

Sample Perl script in SYS:PERL\SCRIPTS

Misc

Applying a Service Pack

Determine which “half” of the nodes to upgrade

Offline resources that are not required

Migrate required resources

Modify nodes list of active resources (preferred node should be from the “to be upgraded list” )

“Cluster leave” and upgrade OS/support pack

Down second half and restart first half

Upgraded half becomes active cluster and online resources on preferred node

Upgrade other half and restart

Changing the Heartbeat Interface

Configure and activate the new networking hardware

Make sure all nodes can ping through the new hardware

Using iManager, change the IP address of each node in the properties of the node object within the cluster object

Down the cluster by executing 'ULDNCS' on each node

Re-load the cluster by executing 'LDNCS' on the first node, waiting until it has loaded then on each node remaining

1

2

3

4

5

The SBD begins with a header field the includes the name of the cluster responsible for this partition. After this header, each node is allocated a 512 byte slot in the partition on which it updates its heartbeat counter. At the same time that it updates this counter, it reads the heartbeat counter of the other nodes on their allocated slots. In other words, a node has write access to the slot it owns, and read access to the slots of the other nodes in the cluster.

Regardless of the number of nodes included in the cluster, the NCS install program creates slots for thirty two nodes when the cluster is installed.

Each node updates its tick count once every four seconds by default. This value is half the LAN heartbeat timeout.

A new file called files.cmd was added to the Admin volume (Admin:manage_nss\files\files.cmd). Applications like NetStorage and other end-user browser-based tools use this file to communicate with the file server. The files.cmd file is a virtual file and generates dynamic information that lets users see and modify their files through a browser.

Every user on the file server will have access to this file and will thus inherit the following default rights: Read, Write, and Scan. This also means that the Admin volume will now be visible to end users. In previous versions of NetWare, this was not the case. End users cannot delete this file nor modify its properties.

There also exists a second option that will enable the upgrade to be performed. Using this method, the downtime window can be drastically reduced.

In this scenario the upgrade is performed in two stages and involves a bit of planning because it is performed on approximately half the number of nodes at a time. You decide what resources need to be kept active, and on which half of the nodes to apply the upgrade first. There will, however, be a short time during which the cluster must be downed.

It is worth considering the implementation of the NCP EXCLUDE set parameter on the cluster nodes for the private heartbeat interface address to avoid DS traffic attempting to sync to an address which is unreachable from the public network. It should also be noted that you will no longer be able to manage the cluster with ConsoleOne unless you place a PC with ConsoleOne on the heartbeat line.

2004-12-09T12:09:05