Configuring and Tuning Storage and Novell Storage Services™ (NSS) features on Novell Open Enterprise Server (OES)

David Mair

Novell Virtual Directory Services Engineer

Novell, Inc.

Randy Stokes

Storage Services Engineer

Novell, Inc.

Paul Taysom

Storage Services Engineer

Novell, Inc.

Steve Toner

Storage Services Engineer

Novell, Inc.

What we will be covering

Configuring physical storage for performance

David Mair

Configuring NSS on OES

Paul Taysom

Managing pools, volumes, and files on OES

Randy Stokes

Configuring and using the distributed file system (DFS)

Steve Toner

Configuring and using archive and versioning services

Randy Stokes

Configuring Physical Storage
for Performance

Plug it in and switch it on?

It's built to be fast

My host OS already has a very fast file system and storage drivers

I'll get more from optimizing the OS or applications

My users are only editing spreadsheets

YES!

Plug it in and switch it on?

It could be even faster!

It could be even faster!!

It could be even faster!!!

You do run backups...don't you

NO!

Carving out pools

Typical procedure

1

Identify a rack with enough storage

Create a logical device from it

Expose it to a server/cluster

Boot the server/cluster, create a pool and volume

2

3

4

Carving out pools

Alternatives

Identify multiple racks with enough storage to sum to the required capacity

Create a logical device grouped across the racks

Expose the logical device to a server/cluster

Boot the server/cluster, create a pool and volumes

1

2

3

4

Fan out

Storage controllers

Use available channels

Multiple controllers can do more than provide redundant paths

Bus

Avoid shared capacity

Software

Software Arrays

Optimized spanning

Use the high performance resources of the host

Distribute over more layers the modes you already planned to use

Validation

Methods

Prefer test cases that represent the most hostile applications you already run

Consider test cases that explore limits

Establish baselines

Server and client cases but understand the expected differences

Test alternative configurations

Consider what unknowns might influence test results

Validation

Tools

TSATest (now on Linux)

NetBench

DPMETERM

IOMeter

ReplSMS*

NTTBE*

CIFSRW*

*Contact dmair@novell.com for copies

How far to go

Be pragmatic

Test off-line

Use representative equipment and software

Only explore configurations you'd be prepared to implement and maintain

Devise a method of analysis that limits the number of test cases you need to run

Quit when you are far enough ahead

That's all very well...

...you don't have to support it

Correct, I don't

Customers do have unexpected storage performance problems

Understanding the scope of optimization for your storage system can save you a lot of time on support calls

Example

The problem

Poor GroupWise® client performance

NetWare® 6.5, GroupWise 6.5, EMC Symmetrix storage

The cause

Configuration

Why

PC-BIOS limits sectors per track to 63

Disks are partitioned

Partition tables must be compatible with BIOS limits

Partitions begin on 63 sector boundaries

Example

Why (continued)

NSS uses 8 sector blocks

The Symmetrix uses 64 sector alignment and custom drive firmware

63, 64 and 8 have only one common factor (1)

NSS blocks are mis-aligned with respect to the Symmetrix because the BIOS partitioning limits

Example

The solution

A shipping version of SCSIHD.CDM that aligns on 32 sector boundaries

Distributing the pool across multiple disk units was effective but not the best solution

Software RAID was effective but not the best solution

Configuring NSS on OES

NRM Statistics

Disk Requests (IOs/S) trend graph

LAN Packets (Packets/S)

Available Memory

Available Logical Space

Tuning NSS for NetWare

NSS Statistics

nss /CacheStats and /ResetStats

nss /ZLSSPoolIOStatistics=_summary

nss /ZLSSPoolIOReset=_summary

/_admin/Manage_NSS/LSS/ZLSS/PhysicalIO.xml

/_admin/Manage_NSS/Pool/<pool name>/ZLSS/PhysicalIO.xml

Tuning NSS for NetWare

Tuning NSS for NetWare

NSS Configuration (examples, not recommendations)

nss /minBufferCacheSize=400000 (/noCacheBalance)

nss /ClosedFileCacheSize=100000 (~1 kbyte per entry)

nss /BufferFlushTimer=30

nss /FileFlushTimer=30

Tuning NSS for Linux

NSS Configuration

No cache balance

1 Gig Memory Limit

nss /minBufferCacheSize=150000

nss /BufferFlushTimer=30 /FileFlushTimer=30

Tuning NSS for Linux

Case Insensitive

http://localhost:901/share Set case sensitive to yes

For mount, add to “-o” option “,ns=long”

For fstab, add “,ns=long” to options after name

Managing Pools, Volumes and Files

What's available for storage configuration?

NetWare

NSSMU

iManager

NRM

Linux

NSSMU

iManager

EVMS

YAST

Command line

NSSMU
(NetWare and Linux)

Runs locally on the server.

Allows management of devices, partitions, pools, volumes, and RAID devices.

On NetWare pool creation is partition based, on Linux it is device based.

Uses the _admin volume as its interface to NSS

Usually the fastest way to set up storage.

Can be done remotely using ssh on Linux.

Adding a volume using NSSMU

Adding a volume using NSSMU

iManager for NSS

Browser

iManager

Server

Storage

Server

Storage Plugin

_admin

HTML

CIMXML or

CIFS or

NCP

iManager for NSS

Usually able to display more information on a screen

Found under the “Storage” role

Uses iManager authentication and role management

Browser based interface

iManager for NSS

NRM

Browser based interface

NetWare only

Supports most frequent storage operations

Device-centric view

Adding a volume using NRM

Volume management on Linux

File systems can run on top of raw block devices

To get additional features you use a volume manager

NSS can run with or without a volume manager, but ...

To be able to move volumes between NetWare and Linux you must use EVMS

Clustering requires EVMS for full functionality

EVMS is open source and supported by IBM and Novell

Conflicts between volume managers

Under the covers

NSSMU, iManager and other _admin based utilities do more than just the basic storage allocation

Create and update the eDirectory objects for pool and volumes.

Interface with clustering

Enforce naming and sizing rules

Update the mount table (Linux – etc/fstab)

Configuring storage using Linux utilities

evmsgui and evmsn

YAST

Various command line utilities

Need to do mkfs for each pool and volume.

mkfs -t nsspool -n poolname /dev/....

mkfs -t nssvol -n volname poolname

Mounting pools and volumes on Linux

Pool mount command

mount -t nsspool -o name=poolname,barrier,shared

/dev/evms/sda1 /opt/novell/nss/mnt/.pools/POOLNAME

Volume mount command

mount -t nssvol -o name=volname,ns=long,norename

volname /media/nss/VOLNAME

Verify and Rebuild

In Linux they are implemented as a user space utility.

Example: ravsui verify pool1

Various option to control behavior

ravview allows you to look at the detailed results

File management

NetWare

Client (big red N)

Client command line utilities

NetStorage

NRM

Linux

Same as NetWare (except NRM)

rights command (trustees, IRF, effective rights)

attrib command for NSS attributes

chmod (sets read-only, execute)

chown (no chgrp)

NetStorage file management

Rights utiltity

Attrib utility

Clear happens before set

Will work on multiple file name

attrib -sro,ip -call foo* -- remove all attributes and then set read-only and immediate purge for all files starting with “foo” in the directory.

find /media/nss/vol -name “foo” | xargs attrib -cro,di,ri,ci – clear the read-only, delete inhibit, rename inhibit and copy inhibit attributes.

Examples:

Attrib utility

NSSCON

Allows you to see the output that would have gone to the NetWare console and logger screens.

Buffers up a limited amount.

Only NSS commands

Managing and Using
the Distributed File System

DFS Components

Junctions

Management Contexts

Volume Location Database (VLDB)

Volume Manager

Clients

Management Interface

Junctions

Look like subdirectory to end-user

Point to the root of another volume

Same server or different server

Let you create a virtual file system that spans devices/servers

Location transparency

Similar in some ways to Linux mount points

Distributed File System

Projects

acorn

sapling

mighty_oak

vol1

code

docs

links

oak_customers

vol2

vol3

vol4

vol17

•

••

•

••

code

docs

bug_db

•

••

•

••

code

docs

customer_list

•

••

•

••

•

••

•

••

•

••

Junctions vs. Mount Points

Junctions:

Allow target volume to move

Physical location is contained in VLDB, not in the junction itself

Allow for volume replication in the future

More than one location allowed!

Volume Manager: Split and Move

Split moves a subdirectory tree & replaces with a junction

Move moves an entire volume

Both update VLDB so the operation is transparent to the end user

Tracks file changes so volumes do not have to be taken offline

When to Use

Split a volume:

To free up space

To reduce load on a system

Move a volume:

To free up space or reduce load

To decommission a server or its storage

When it has junctions already pointing to it

Job Scheduling

Split & Move jobs can be scheduled to run at a later time

Jobs can be paused, rescheduled

Time of day/Day of week control

Not currently exposed in iManager

Management Interface

A mixed bag right now

ConsoleOne (or XML) to create Management Context, Junctions

iManager to move or split a volume

Everything can be done through XML

Documented in Virtual File Services for NetWare on developer.novell.com/ndk/doc.htm

# Main

#

{

if ($#ARGV == -1 || $#ARGV > 1 || $ARGV[0] eq "/?" || $ARGV[0] eq "-?")

{

print "USAGE: readjunc.pl <path> [server]\n".

"Example: readjunc VOL1:junc server1 \n".

"If running from a server don't enter a server name.";

exit;

}

# Open the command interface file for read and write

if ($#ARGV == 0)

{

open(NSSFILE, "+<_admin:Manage_NSS\\manage.cmd")

or die "Error opening admin file ($!) on server";

}

else

{

my $server = $ARGV[1];

open(NSSFILE, "+<\\\\$server\\_admin\\Manage_NSS\\manage.cmd")

or die "Error opening admin file ($!)";

}

ReadJunction($ARGV[0]);

close(NSSFILE);

}

Example Perl Script

# Read a DFS Junction

#

sub ReadJunction($)

{

my $cmdResult;

my $pathname = $_[0];

my $cmdResult = ExecuteCommand(NSSFILE, "<readLink><pathName>$pathname</pathName></readLink>");

print("Result: $cmdResult\n");

}

Example Perl Script cont...

# Execute an NSS command

#

sub ExecuteCommand(*$)

{

my $fh = $_[0];

my $command = $_[1];

my $result;

my $reply;

my $error;

seek $fh, 0, 0;

$command = "<virtualIO><datastream name=\"command\"/></virtualIO>".

"<nssRequest><dfs>".$command."</dfs></nssRequest>";

syswrite($fh, $command, length($command))

seek $fh, 0, 0;

my $readLen = 999;

while ($readLen)

{

$readLen = sysread($fh, $reply, 1000);

$result .= $reply;

}

return $result;

}

Example Perl Script cont...

Availability

Services only run on NetWare today

Junctions can only be created on NSS file system

But can point to non-NSS volume

(target must have Volume object in eDirectory)

Linux is coming

Availability:
Client Support

These clients are DFS-aware:

CIFS (Native Windows™ client)

NCP (Novell Client™)

NetStorage

Not DFS-aware:

AppleTalk

NFS

LibC

Samba

Integrating Linux into NetWare Tree

Projects

acorn

sapling

mighty_oak

vol1 (NetWare)

code

docs

links

oak_customers

vol2 (NetWare)

vol3 (Linux)

vol4 (NetWare)

vol17 (Linux)

•

••

•

••

code

docs

bug_db

•

••

•

••

code

docs

customer_list

•

••

•

••

•

••

•

••

•

••

Configuring and Using
Archive and Versioning Services

What is archive and versioning services?

Automated backup of changed files from specified servers to a central archive location.

Versions of files preserved

Automated rules for limiting the number of versions and how long to keep them

Selection criteria to determine what files will be archived

Archive/Versioning architecture

Versions:

Server A:Vol2:foo 10 Mar 2:00

Server A:Vol2:foo 10 Mar 2:15

Server A:Vol2:foo 12 Mar 10:32

Server B:Vol2:moo 8 Mar 6:12

Server A:Vol2:moo 11 Mar 4:56

Server A:Vol2:moo 12 Mar 10:32

File: foo

Archive

Server

Server A

VOL

1

VOL

2

VOL

3

VOL

2

VOL

2

VOL

1

VOL

1

Server B

Server C

File: moo

Configuration file

Authentication Information

Database location

Destination path

Directory tree information

Logging parameters

Default job information

Jobs

Name

Schedule

Filters

Delete policy

Checking job status

add screen shot of iManager page

Restoring versions with NetStorage

add screen shot

Restoring versions from Windows

Add screen shot

Archive/Versioning conclusions

Keeps versions of files for a defined length of time

Allows users to restore files or versions of files themselves

Runs unattended

Good for work space type applications (i.e. documents, spreadsheets, graphics)

Not suited for databases

The future
of archive and versioning services

The current product is just starting to get attention

The set of services will be expanded to allow restoring of files from snapshots and salvage.

We intend to make a straight archive that can be used as a warm backup, with no versioning database, just a current image.

TUT308

2003-12-08T12:43:39