Deploying Linux to the Data Center

Carter George

Vice President, Storage Software

PolyServe

Derk Meima

Data Center Product Manager

Novell, Inc.

Assertions about the Data Center

Applications drive data center strategies and direction

Customers are willing to pay for and implement proprietary solutions that meet/exceed their needs and improve application

Mixed environments will continue to exist for some time

Support remains the primary concern for data center customers

Business Perspective – Tier One Linux Data Center Requirements

Focused Effort

Novell has created a team to drive a focused, cross organizational data center strategy

Application Certification

Novell has commit resources to strategic LOB application vendors

Sales Execution

Novell is investing heavily in data center expertise within the sales organization

Pricing and licensing has been modified to meet customer needs

Hardened Linux

Appropriate white papers and best practices have been created to educate customers on Linux readiness

Business Drivers

What problem needs to be solved?

Unix is expensive and customers are looking for a low cost alternative

Where are the gaps?

1. Application Certification

2. Reliability, Availability, Scalability (business continuance)

3. Migration & Consolidation

Virtual Storage

Volume manager

High availability software

Cluster file system

Virtual Machines

Consolidation

Migration

Effective hardware utilization

Resource Management

Scheduling

Patch management

Deployment

Grid

Identity Management

Users, groups, resource association

Pervasive infrastructure that relates these services together

Utility Computing

SUSE Enterprise Linux

VS

VM

IDM

RM

Backup

Data Center Technical Perspective
Tier One Linux Data Center Requirements

Applications

The Linux Data Center

Data Center Linux is an Architecture

It is much more than just running Linux on some servers

The Linux Data Center is never just about one server

The Linux Data Center hosts mulitple mission critical applications

All servers are part of a scalable, fault tolerant cluster

There are three key goals for the Linux Data Center

Security

Manageability

Availability

This presentation is about creating, managing and deploying scalable fault-tolerant clusters to host mission-critical applications on Linux

Especially databases and enterprise file services….

Why consolidate to Linux in the Data Center ?

The potential benefits are obvious….

Two 16-way Sun Fire 6800

One Active

One Passive for Failover

Solaris

10 Linux Xeon Blade Servers

(all active, 2 active “spares”)

20 x 3+ Ghz CPUs

SUSE® LINUX SLES9

$97,000

$1,053,190

OR

The $1,043,190 question
Can the servers on the right-hand side do the job?

Scalability

Can the application scale across servers?

Availability

Can the same level of application availability be met?

Manageability

Can a cluster of ten servers be managed as easily as two?

Conclusion: Need a new data center Linux system software stack to deliver the new data center architecture

Legacy HA Clusters: Not Enough

Purpose: Application Availability

Protect against server failures

Protect against system hangs

Architecture: Active / Active / Passive

A primary for one application may be backup for another

No two nodes access the same data set at the same time

Ownership of application data is transferred at failover time

Shortcomings

Complicated to install

Hard to manage

Don’t help scalability

Don’t help manage a set of servers as though they were one

client

client

Failed

App Server A

App Server B

Server A Application Data

Server B Application Data

Heartbeat Signaling

Data Center Clustering with Shared Data

Legacy HA

Shared Data

Benefit

Scalability: Add capacity for applications without disrupting service levels, and without overtime

Seamless, modular, scale-out

Transition, scale-up

Availability: Absorb failures to minimize downtime

Server failures

Storage failures

Manageability: Deliver true server consolidation by compressing managed software entities

Single ‘Application Home’ for multiple servers

Single ‘Operating System Home’ for multiple servers

High Availability

Application Monitoring

Application Failover

Cluster File System

All nodes can read and write all storage

Data Center Linux
Tier One Linux Data Center Requirements

SUSE Enterprise Linux

CFS

CVM

IDM

HA

Backup

Applications

Cluster Volume Management

Storage can be virtualized across all nodes

Identity Management

View the cluster as a single entity for authentication and security

Data Center Cluster Offerings

Cluster Volume Management

Clustered FS

High Availability

Virtual Machine

Resource Mgmt

Advantages

Identity Mgmt

Designed for legacy Unix

Sold and Supported by Novell

Based on acquisitions: GFS and RHCS

HA only

Xen

PolyServe - Foundation for Shared Data Clustering for Data Center Linux

Single point of administration

Central console for unified

Multi-server view

Cluster File System

All servers can access common applications, data at full SAN speeds

Integrated high-availability

Monitoring of hardware, software health

Configurable failover policies

Cluster Volume Manager

Group physical storage into logical volumes

Stripe or concatenate across multiple arrays

Online file system growth

Up to 128 LUN’s per volume

Sold and Supported by Novell

PolyServe Matrix Server
Shared-Data Clustering Software

Software

Hardware

Scale-out Database

SUSE LINUX

Application Server

Web Server

Cluster File System

Programs on multiple servers can share files on SAN

Mount a file system on up to 16 nodes

Up to 512 shared file systems per cluster

Symmetric architecture – no single point of failure

Single system semantics – works just like any standard Linux file system

Supports databases, applications, file serving, and other uses

Distributed lock and metadata services

Single Data and Binary Pools

Matrix Server

Cluster Volume Manager

Array 1

Array n

Cluster-aware Volume

Create logical volumes from multiple physical chunks of storage

Up to 128 chunks per logical volume

Stripe or concatenate chunks to create the volume

Variable stripe width for performance optimization

Online file system growth – add or expand LUN’s any time with no user disruption

File system size: 16 Terabytes now, 256 Terabytes in June

High Availability Clustering

Matrix Server provides

High availability engine that monitors

Hardware health

OS health

Application health

Both whole server and per-application failover

Administratively driven rehosting (for planned updates)

Fast fail-over time with no disk transition or reboot delay

Easy set-up with concurrent server access to application files

Virtualization and Instant Provisioning

Boot images for each server

OS Images can be on the
blades’ local disk or SAN

PolyServe allows SHARED SuSE Linux root file system

Servers SHARE everything else

CFS supports sharing of everything else:

Applications

Databases

User Files

Instant Provisioning

Just mount the file system

No copying of large installations from one place to another

No propagation of multiple copies that have to be managed

Shared Data Clustering and the Server Farm

The Server Farm Approach:

Allocate nodes to applications as required

Ideal for 3 tier apps like Oracle11i or SAP

Replaces Unix systems as the data center architecture of choice

Virtual server consolidation

One blade farm can support both Linux and Windows

Use standard server technology, storage, interfaces for maximum compatibility, interoperability

Benefits:

Greater HW capacity utilization

Higher availability with lower costs

No fork-lift upgrades

Monitor

Action

Alert

Partners

Four Types of Cost Savings

Once you consolidate to a Linux Data Center server farm:

Lower Server Costs

Linux / Intel server farms cost less than legacy Unix boxes

Lower Lifecycle Costs

No need to pay hardware maintenance

Lower upgrade costs – just add servers when you need them

Lower cost of availability: 20% versus 100% on Unix

Better Utilization of Assets

Move servers to where they are needed dynamically

Rebalance capacity at any time

Move from 25% utlilization to 66% or more

Operational Efficiency

All the benefits of server consolidation – single, central point of mgmt

One set of file systems to update, manage, and backup

Everyone’s Doing It
PolyServe Matrix Server Customers

Consolidating Oracle on Data Center Linux

Using the PolyServe Cluster File System to Make Oracle More Reliable, Faster, and Easier to Manage

Oracle High Availability solution

With CFS multiple Oracle instances are flexibly deployed, managed and backed up from one location

Simplified management of a clustered environment

Adds N+1 failover capability to eliminate idle servers and achieve higher server and storage utilization

Improved server and storage utilization over current levels - using passive standby servers for availability, and over-provisioned

With virtualized Oracle instances configuration is easy and clients failover automatically and quickly

Fast reliable failover of clients

Adds High Availability to Oracle instances, makes management easier, and is a great stepping stone to RAC

Oracle databases being shifted from UNIX to Linux require fault tolerant infrastructure

PolyServe solution

Need

PolyServe Oracle HA solution

Easy, reliable HA for Oracle that works!

Replace active/passive pairs

Implement N+1 high availability

Virtualized Oracle instances

Consolidate management of multiple Oracle servers

Service monitoring of key Oracle processes

Centralize storage management

Centralized I/O monitoring

Easily add RAC for horizontal scaling when it is needed

Oracle HA solution benefits

Reliable high availability for business continuity

a. Transition versus failover

b. Fault tolerance and assured successful failover

c. High availability for Oracle related files

Simplified systems management saves time

a. Shared Oracle Home

b. Cluster wide I/0 monitoring

c. Single central Oracle administration

Lower costs

a. Lower CapEx and OpEx for Oracle systems

b. Manage to higher utilization

c. “One for all” back-up and recovery

Simple, reliable, successful

Oracle high-availability

1

2

3

Shared Oracle Home

Replaces multiple Oracle Home directories with a single shared location

Single management point for all files, tables, logs, patches etc

Reduced storage usage

Reduced installation time

Reduced patch work

Reduced downtime during upgrades

With PolyServe

Oracle HA Solution

Transition versus Failover

No data transition or hand-off

No manual intervention required

No single point of failure

Virtualized Oracle instances

Instances move automatically to other servers

Redeployment of instances in < 1 minute

Clients transition with ease

Eliminates the need for, SCSI reservation

Faster, safer, more successful failovers

When a server fails

Instances failover

Fault tolerance and assured failover

Integrates multi-path I/O capabilities for server-to-storage redundancy

Multiple switch support for fabric redundancy

Multiple network support for server-to-server redundancy

Backup target servers are always known to be good

Continuous health monitoring

Backup server may also be an active server

Journal File System

Failback support

SAN Fabric

SAN Fabric

HA for Oracle related files

All Oracle related files are centrally managed and continuously available

Single shared Oracle diagnostic information location

All instance logging files - alert logs, trace files core dumps, etc

No need to wait for a node to recover

Immediate knowledge of what a failed node was doing - Was it in a checkpoint, in a log switch, auto-extending?

Continuous access to archived redo logs

Makes Oracle "Log Mining“ highly available

Oracle Cluster

SAN Fabric

Oracle

Data

Shared

Oracle

Home

File

Based

Data

Single central Oracle administration

Easy, GUI-based configuration

Very few dialog boxes

Simple pick-list for HA nodes

Optional demonstration database setup for training

Optional SQL*Net setup

Single SQL*Net entry

Administrators can also choose to use netca/netmgr

No special sysadmin privileges required for management

Serviceguard

Market for Linux Oracle HA solutions

Matrix Server

Manageability

System Availability

VSF 4.0

CFS, HA & MPIO

Cluster HA Technology

GFS + CS

LifeKeeper

Virtual Machine Technology

VCS

“With PolyServe Matrix Server, we can chain together less expensive Intel architecture–based servers and get more processing power at a much lower cost … than previous proprietary hardware. The PolyServe environment is just as manageable as a large UNIX server.”

Bill Ubelacker Director of System Services

Customer Success:

Goals

Migrate entire data center from IBM/Sequent UNIX to Linux clusters

Migrate a variety of applications to Linux

40 mission-critical Oracle databases

Custom J2EE Java applications

Custom file-based applications

Solution

Multiple clusters of Oracle9i RAC

Oracle9i Application Server for J2EE

File-based applications scale out with shared data repositoryOracle9i RAC

Results

Lower TCO with better performance

Gained system management and availability comparable to UNIX

Scalability to keep pace with retail volume

Fault tolerance to guarantee uptime

Customer Success :

CPF is one of UK’s 200 largest companies, with over 2.7B in assets

Solution

Loan processing application

PolyServe Oracle HA Solution for Oracle9i

3 clusters of 5 nodes;

3 active nodes, 2 stand-by for HA

6 live Oracle database instances

Results

Rugged High Availability for 24x7 applications

Oracle HA at drastic cost savings compared to UNIX alternatives

A flexible, cost-efficient infrastructure that can grow with the business

Highlights

Oracle HA / PolyServe / SuSE

Three 5-node clusters

Hosting 6 Oracle 9i databases

24x7 Availability for one of the largest financial services companies in the UK

Drastic cost savings over UNIX-based alternatives

Deploying Oracle RAC Clusters on Data Center Linux

Using the PolyServe Cluster File System to Make Oracle More Reliable, Faster, and Easier to Manage

Oracle RAC on PolyServe

Shared Oracle Home – a single installation and maintenance point for Oracle binaries

Shared All – makes all Oracle deployment files, like log files, accessible to all servers

On-demand capacity allocation - adds and reallocates server and storage as needed

RAC Stats – provides a view of aggregate database performance across all servers

Parallel Extract Transform & Load (ETL) - concurrently applies all servers in the cluster to data warehousing tasks

Oracle RAC on PolyServe solution

Reduces implementation risk and increases failover success

Complexity jeopardizes service level agreements (SLAs)

Increases total cluster performance and utilization

Performance scaling past 2 nodes

Provides key operational tools to lower operating expenses

Complexity increases deployment and management costs

Decreases operational complexity by simplifying installations

RAC increases the complexity of server and storage administration

PolyServe solution

Need

World’s Largest Proof Point

16 x345 servers and 200 IBM disk spindles

SuSE SLES 8

Five principal areas of measurement

Complex (mixed OLTP and DSS running on same cluster) workloads

Dynamic server partitioning

Performance & bandwidth

Availability

Manageability improves TCO

Switch

Switch

16 Node Linux Cluster

PolyServe’s Flexible Database Cluster proof point is a “milestone in the evolution of Linux… the test indicates that Linux will soon be running more critical workloads… and Linux being able to handle workloads that only Unix or mainframes could handle before.”

Dan Kusnetzky, VP, Systems Software Research, IDC

The Flexible Database Cluster –
World’s Largest Oracle9i RAC Proofpoint

16 IBM x345 servers

SuSE Linux

Dual FC HBA’s

Dual switches

206 Disk Spindles

IBM Raleigh Test Center

FDC - Scaling & Performance

Results: OLTP

Scalability directly related to I/O throughput

No server scalability problems experienced

Peak cluster-wide CPU utilization < 82%

Near-linear scaling through 10 nodes

Adding disk drives relieves bottleneck

Results: DSS

Confirmed Oracle 9i RAC scales on demand

Sorted table had 1.6 billion rows

Sorting not performed entirely in memory

Adding nodes a non-intrusive effort

No scalability limits seen at software level

One node query = 101 minutes

Eight node query = 16 minutes !!

Scalability was 81% from 1 to 8 nodes

Competitive feature comparison

Distributed Lock Manager

Scales past 8 nodes

performance

Distributed Metadata Mgmt

Cluster wide I/0 monitoring

Updates – one place!

management

On demand capacity growth

Async I/O with ODM

Scales past 2 nodes

Increased

Lower costs

Simplified

Benefit

Parallel ETL

On line re-purposing of nodes

Shared Oracle Home

On-line Dynamic Data Redistribution

Feature

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Matrix Server

No

No

No

No

No

No

No

?

1 node

No

No

No

OCFS

NAS NetApp

No

No

No

No

No

Yes

No

No

No

No

No

No

Redhat GFS

No

No

Yes

No

Yes

Yes

No

Yes

Yes

?

Yes

?

VCF 4.0

No

No

No

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Raw

No

No

No

No

No

No

With ASM

Yes

1 node

No

No

With ASM

Landscape for Linux Oracle RAC solutions

CFS Only

OCFS

Manageability

System Availability

VSF 4.0

GFS + CS

Matrix Server

CFS, HA & MPIO

Customer Success: Putnam Investments

A global money management firm with over $2B in assets.

Solution

Professional online trading application

2 node, 8-cpu cluster on SuSE Linux

PolyServe Matrix Server Oracle RAC pack

Results

Reduce cost & improve asset utilization

Ousted incumbent eGenera & OCFS

MxS and Oracle RAC implemented and configured in under five hours

Improved system reliability and failover time

Highlights

Data & Server Consolidation

Replaced eGenera for significant cost savings

Fast and Easy implementation

Improved reliability

Customer Success: Open Solutions

Goals

Migrate from Windows to Linux infrastructure

Simplify storage management – major cause of downtime and inhibitor to growth

Solution

Hosted financial services application for local banks and credit unions

2 Oracle9i RAC/PolyServe clusters of 3 nodes

Results

Hosts 13 major financial institution clients (growing)

Saved 80% compared to UNIX (hardware & maintenance)

No downtime since production deployment

20% system-wide performance increase

Cut batch processing time by 50%

“PolyServe Matrix Server software made it possible to implement clusters of blades at a fifth of the cost of ‘Big Iron’ UNIX machines and provides a high performance and highly available data processing infrastructure for our clients.”

Charlie Beer

VP of Operations

Customer Success : Tractor Supply Company

Goals

Replace 8-way Windows SMP servers failover cluster with all active Linux cluster

Solution

Data warehouse for all POS transactions

6 node Oracle9i RAC/PolyServe cluster

Results

Improved system uptime compared to Windows

Reduced cost of on-going systems management

Facilitated non-disruptive modular expansion

Manage the cluster cost-effectively

“We put PolyServe through an exhaustive set of tests and benchmarks and compared it to other alternatives. Each time, PolyServe stood out from the crowd. It was the best product out there.”

Stevan Townsend

Project Manager

Customer Success : Novell, Inc.

CIO Mandate: Get every Novell data center running on Linux

Solution

All Novell financials—accounting, P&L, payroll, tax—on PolyServe & SuSE

Host Oracle Apps and RAC on single cluster

Multiple server vendors; Infrastructure re-use

Pilot project of 12-processors with significant increase for production and roll-out

Results

Huge cost savings from Unix to Linux migration

Improved management through simplified upgrade & patch process

Robust and easily scalable Oracle system

Highlights

HP-UX Migration

Oracle11i Financial Apps and 9i RAC on single cluster

Greater admin efficiency through improved upgrade and patch management

Storage System

Storage System

Shared Data

Allows all servers to “own” all data with speed & integrity

High Availability

Application Failover

Capacity Maintenance

Inserts & upgrades servers and storage as needed, with no service disruption

Application Management

Matches applications to required capacity - dynamically

Storage System

Storage System

Storage System

Storage System

Consolidated Storage SAN

High Availability

Application Management

Storage System

Storage System

Storage System

Storage System

Shared Data

Storage System

Storage System

Storage System

Storage System

Shared Data

Capacity Maintenance

The Clustered Data Center

Linux Data Center Applications

Oracle 8i, 9i, and 10g

Oracle RAC

File Services (NFS, NAS)

Sybase

DB2

Infrastructure

Dynamic Allocation of Servers to Applications

Dynamic Allocation of Storage

Shared SuSE Root

Shared Applications Binaries

Shared Config Files

Application Fault Tolerance

Storage System

Storage System

Storage System

Storage System

Consolidated Storage SAN

High Availability

Application Management

Storage System

Storage System

Storage System

Storage System

Shared Data

Storage System

Storage System

Storage System

Storage System

Shared Data

Capacity Allocation

File Serving

Web & App. Server

Oracle RAC

Messaging

Back

Up

Scale

Up

DB

Scale

Up

DB

The Clustered Data Center

Scale

Up

DB

X

Scale

Up

DB

Repeat the following slides for each product line.

Include these slides even for product lines you plan not to discuss.

1. Form a team to focus on the Data Center Linux

Need the focus to enter this new market

Reaches all parts of the organization (sales, marketing, PBUs, consulting)

Define the data center product that sits above the distribution

Alliances (Application Certification - ISV’s, IHV’s)

Certify SUSE Linux on all major line of business and vertical applications

Provide testing environment for application certification

Expand development resources with key partners (Oracle, DB2 and others) for on-site consultation and development

Marketing/Consulting

Publish detailed white papers, migration practices, and benchmarks

Coordinate the delivery and implementation of open source management tools

Unix to Linux, and Windows to Linux documentation

Positioning Novell in the Data Center as the market leader

Determine correct licensing/pricing structure

Neutralize RedHat market momentum

Determine correct support structure

4. Sales

Different sales pitch and structure than in work group space

Training within partner sales organizations is critical

5. Security

Publish policies and best practices

Default to Off

Need to Add Ron's Edit's here

We've spent months doing research on Linux in the data center. We've talked with customers all over the US and Europe.

We've validated our findings with reseller partners ISV's and IHV's then created a market response by articulating a data center solution strategy. We've socialized this strategy with the likes of:

- Ron Hosephian

- Bob Coutour

- Nancy Faigen

- Joe Wagner

- John Dragoon

- John Hogan

- Scott Lewis

- Brian Green

We talked with category specialists in the US and Europe. And, everyone agrees that this is the right strategy.

Includes: Server hardware, HBA’s

Does Not Include: File system or cluster software

Summary Slide: Recap evolution of computing…economics of Matrix computing too compelling to ignore

Customer cases: Morgan Stanley…2800 Mid range Sun servers replaced by 11,000 intel based servers running Linux

Burlington Coat…etc

Economics are too compelling

1. Form a team to focus on the Data Center Linux

Need the focus to enter this new market

Reaches all parts of the organization (sales, marketing, PBUs, consulting)

Define the data center product that sits above the distribution

Alliances (Application Certification - ISV’s, IHV’s)

Certify SUSE Linux on all major line of business and vertical applications

Provide testing environment for application certification

Expand development resources with key partners (Oracle, DB2 and others) for on-site consultation and development

Marketing/Consulting

Publish detailed white papers, migration practices, and benchmarks

Coordinate the delivery and implementation of open source management tools

Unix to Linux, and Windows to Linux documentation

Positioning Novell in the Data Center as the market leader

Determine correct licensing/pricing structure

Neutralize RedHat market momentum

Determine correct support structure

4. Sales

Different sales pitch and structure than in work group space

Training within partner sales organizations is critical

5. Security

Publish policies and best practices

Default to Off

Need to Add Ron's Edit's here

This is a slide with a lot of clicks that you should probably spend some time on.

System Management Functions:

- Server/App/OS Provisioning

- Resource Monitoring

- Service Level Enforcement

Load Balancing

Server/storage config. Mgmt.

Role Summary:

Unified systems management system providers:

What should happen

When it should happen

PolyServe:

How it happens

Higher Availability of Applications:

Nodes can be taken off line without affecting the system

In many cases there is <1 minute failover so business users aren’t impacted but we can’t guarantee this because configurations vary. Certainly we can manually failover/redeploy instances in less than a minute usually.

Avoidance of costly downtime

Support for rolling upgrades

Simplified Systems Management:

Single system image to users, applications, and network, while providing a single point of control for administrators

Easily move Oracle instances

Add capacity on demand

Fast centralized back up and restore

Configure clusters and failover with ease and confidence

Fast software updates

Manage more resources with fewer people

Cost

Reduced hardware costs through fewer servers, elimination of idle “passive” servers, sharing of fewer spares

Since we support heterogeneous servers, you can right size servers to the majority of the needs of an application and move additional resources to it only when periodic peak demands necessitate it, not full time as is the case quite often today. This results in higher asset utilization from the now heterogeneous server farm.

Reduced software costs through fewer server software licenses for active/passive pairs and only needing a single copy of many software packages including the backup software.

No need to over-provision to avoid “fork-lift” upgrades. Grow cluster and storage as business dictates without having to provision hardware forecasted peak load. Ride the curves that cause each increment of compute capacity and each megabyte of storage to get less expensive every six months.

Adding a SAN permits access to any disk from any server

Adding PolyServe Oracle HA solution means that the clients are not tied to their database on a particular server. We virtualize the Oracle database so that in the event that the primary server dies, and the Oracle instance restarts on a second server, the clients will automatically find it and continue!

One spare can be used for many servers, eliminating the need for idle passive servers

This also eliminates the replication of data and Oracle binaries, and enables fewer more highly utilized disk subsystems

Software costs are reduced by eliminating un-needed software licenses and by having fewer servers

Easily add more servers and storage to live clusters – on demand

Eliminate costs of over provisioning to deal with peak demand

Consolidates management of Oracle HA platforms

Concurrent access to a central set of files means:

No need to replicate logs or files

No need to waste storage space

A single view of storage for software updates and configuration

A single view of storage for management and backup/restore

How long does it take to failover – this depends on the size of the database, how many transactions in flight, size of redo log. The database startup can take a few minutes as it plays redo log (roll back).

A Share Oracle Home allows the installation of the 105K binaries in a single location that is shared by all nodes without making per node copies. The single Oracle home also allows all cluster members to use the same configuration files. This dramatically simplifies the deployment of even large Oracle9i RAC clusters. Additionally Context Dependent Symbolic Links provide support for files/directories that are node specific.

Installing and patching Oracle consumes a significant amount of a DBA’s time without a shared Oracle home, as each node needs to have access to the 105K binary files. Share Oracle Home automates allows the binaries to stored in one location with access from all nodes.

PolyServe’s shared data model allows for a global view of the data by any of the servers. Thus, when a failure occurs, we simply migrate the virtual database instance from the failed node to the designated node automatically. No data transition or data hand off takes place since all servers can access all data.

Failover can be fast but it depends on if the database is cleanly shutdown. Therefore a manual deployment of an instance can be very fast, quite probably less than a minute. However an unexpected server failure will certainly take longer than 1 minute since the database will have to startup on the target node, and there will have to be a consistency check. The time is more likely to be about 15 minutes – but check with a technical person before making commitments.

We cannot say there will be no effect on clients. The connection is broken and any requests will fail during the failover process. If the client software is smart enough to retry, it may not be noticed. But if they have custom apps, the app has to manage the broken connection. The issue revolves around how we deal with the client with our virtualized database connection.

Traditional HA solutions utilize SCSI-2 reservation (protocol that handles reserve and release of disk). A typical problem with this model is that sometimes a failover that should occur is not initiated because of the unsuccessful release of disk from the failed node to the target node. This provides extra complexity (and an extra step) that PolyServe avoids. We also don’t do any LUN Masking.

There is no need to have a single point of failure in a PolyServe Cluster. All paths and elements can be made redundant: the servers, the connections to the SAN, the FC switches, the connection to the storage, and the storage itself. The PolyServe software solution is fully distributed and symmetrical. Ie there is no one server or element that can be a central bottleneck or single point of failure. On Linux PolyServe has our own implementation of MPIO. But we also work with PowerPath and SecurePath. Many arrays work differently when it comes to MPIO so ask first before committing to support on a new array.

A typical problem with cluster software from other companies is that sometimes a failover that should occur is not initiated because of the unsuccessful release of disk from the failed node to the target node. Since we do not depend on this mechanism (because we have CFS) and since we are continuously testing the target node to ensure it is good, failovers with Matrix Server are successful

Oracle alert log is the background dump destination. Without Matrix Server If a node crashes you can’t see Oracle specific info until that node comes back alive. In systems without PolyServe, every node has its own instance logging files and in the event that a node fails those files become inaccessible until that node is recovered. It may be important to know exactly what was running on the node when it crashed. Also, each node is archiving its own redo logs. These would also be inaccessible without PolyServe, locked up with the failed node. By adding CFS all of this data remains accessible even when a node goes down.

Archived redo logs must be available to all nodes in the cluster in the event a roll-forward is required.

All file types can be shared by all servers in the cluster including:

Oracle Home

Database tables

Oracle executables

Oracle transaction logs

Configuration files

Patches

Flat files

Database table spaces can be stored in standard files enabling simpler backup

Media files, and other databases

Matrix Server enables database table spaces to be stored in standard file system files, and supports access by standard backup tools and utilities. This permits standard third-party backup tools to be used for Oracle database tables.

OCFS only supports data files.

All file types can be shared by all servers in the cluster including:

Oracle Home

Database tables

Oracle executables

Oracle transaction logs

Configuration files

Patches

Flat files

Database table spaces can be stored in standard files enabling simpler backup

Media files, and other databases

The soon to be released product, VSF 4.0, from Veritas is untested in the Linux world and based on an architecture that has a fundamental bottleneck. There is no question we are more experienced than Veritas on Linux, we have large production sites running today, we can out perform them, we don’t have the bottlenecks they face, and we are more reliable.

MxS is fully symmetric and has no single resource that could be a source of contention. Red Hat bought Sistina and integrated it’s CFS product as GFS. Combined with their Cluster Suite (CS) they have High Availability capability. They also support SANs and MPIO. They have a remarkable lack of customer testimonials…It is very doubtful that the GFS and the CS product are in any way integrated and so using this in a commercial environment is highly inadvisable.

Clusters High Availability software has been available for many years dating back to mainframes in the 70s. The current products from Microsoft, Steel eye, Legato and Veritas on the Windows operating system all have similar features found in Clusters software 30 years ago. The chief benefit of Cluster HA software is that it can significantly increase availability of an application. It does little to make the system more manageable however.

The most significant advancement to clusters was the addition of Cluster File System support which allowed all nodes in a cluster to access the same files on the same disks concurrently and even update those files. The addition of CFS to the HA software not only increased the availability of the cluster substantially but it eased manageability considerably by presenting the administrator with a single view of storage, files, and of all the servers.

Red Hat and Veritas VSF seem more focused on RAC but we may start to see them in this solution space.

5 Dell servers with EMC cx600 storage

Single Shared Oracle Home

Reduces management cost of maintaining and configuring a highly available cluster of many small Oracle10g RAC servers. Helps make many small servers manageable just as cost effectively as a large UNIX server. It decreases configuration and patching time and errors

Enables the use of Oracle Managed Files (OMF) with Oracle10g RAC

Enables full use of the OMF capability with an Oracle10g RAC cluster. OMF delivers significant manageability benefits to Oracle administrators. OMF cannot be used with a shared raw device.

Better Manageability than Raw Devices

Appears as a standard file system to third-party utilities and tools. Administrators can use familiar tools and utilities to provision, backup, and restore information. No new tools are needed.

Improves the performance of Oracle10g Extract

Allows all nodes in an Oracle10g RAC cluster to participate in parallel. Transform, and Load (ETL) functionality. ETL processing. A cluster of Oracle10g RAC nodes can load the data much faster than a single node processing the entire workload by itself. External Tables combined with the Parallel Query Option can reduce data movement by 50% during a typical ETL procedure.

Independent Scale Out of Computing and Storage Assets.

Enables dynamic provisioning of capacity to keep up with changing load requirements of Oracle10g RAC. Computing and storage systems can be added independently

I/O Optimizations Improved Over Raw Devices

Uses Oracle Disk Manager (ODM) to deliver superior performance over raw devices. Eliminates the need for write locks and metadata updates and provides async, direct I/O.

Easy migration from Oracle9i RAC

Enables clusters running Oracle9i RAC to be upgraded in place to Oracle10g. Allows a single Matrix Server cluster to host both Oracle9i and Oracle10g databases simultaneously, providing cluster-wide I/O monitoring across both versions

Easy migration to Automatic Storage Management (ASM)

Allows tables to be migrated into ASM gradually, while permitting others, such as transportable tables, to remain directly accessible to operating system tools without the use of RMAN.

Complements Oracle Cluster Ready Services (CRS)

Provides complete high-availability, and reliability through improved I/O fencing.

Seamless Integration with the Oracle Disk Manager (ODM)

Tightly couples Matrix Server and Oracle10g. Ensures improved performance, full data integrity, and forward compatibility with Oracle10g RAC.

Centralized Management of Storage Assets

Centralized file system management simplifies SAN administration. In fact, SAN based architectures are less costly to manage than DAS.

Multi-Path I/O

Supports the ability to use multiple data paths to shared storage. Supports multiple HBAs per server with connection to different switches. If a data path is lost, Matrix Server seamlessly fails over all data traffic from one HBA to another, while maintaining data coherency and integrity. The administrator can create a fully redundant, highly available switched SAN fabric with no single point of failure.

Guaranteed Data Integrity

Data is protected from corruption or error because the file system enforces a distributed locking and cache coherency mechanism that guarantees data integrity and validity for file access. Matrix Server provides direct I/O for block level access and guaranteed data integrity.

No Single Point of Failure

Matrix Server can endure N-1 server failures and continue to process requests.

Matrix Server supports cluster-wide patching, with a single location to patch, greatly easing on going maintenance. Patches and updates can now be loaded once, eliminating “fat fingered” operator errors.

Matrix Server enables Oracle’s External Table feature to allow all cluster nodes to access data stored in flat files, Virtual Device Layer provides persistent administrator-assignable, cluster-wide names.

ODM supports cluster wide summary view of the I/O statistics across the entire cluster. Without this, an administrator has to manually roll up the information from each node to gain a cluster wide perspective.

Matrix Server is a general purpose file system that allows not just database files, but other external tables, logs, and flat files to reside and be accessible from all nodes within a cluster. This can be incredibly important when you consider that without Matrix Server each node has sole access to its own instance logs, and redo logs, and if that node crashes and is slow to reboot, that data is inaccessible. Matrix Server allows the database admins to access all of these files ALL the time, regardless of the state of the nodes (save one…).

Simplified Systems Management

Single Oracle home simplifies management and updates

Easily move Oracle RAC instances

Cluster wide I/O statistics

Use standard backup and restore utilities from a central location

Configure clusters with ease and confidence

Fast software updates, patches all go in just once

Single system image to users, applications, and network, while providing a single point of control for administrators

Performance improvement and capacity growth flexibility

With Cluster ready applications scale up to 16 servers

Speed up data warehouse load times up to 16x

Add servers and storage to a live cluster with ease

Anticipate times of peak demand, adding resources as needed

ODM dramatically improves I/O performance

Higher Availability of your database

Ability to add servers and storage on demand

Continuous access to all Oracle files including instance logging and redo logs

Multipath I/O for no single point of failure

Sub-minute redeployment of RAC instances

Fewer configuration errors leading to more stable clusters

Control your costs

Get the cost savings you expected with Linux and industry standard servers without giving up on centralized management or centralized storage. Moving to a pool of servers and SAN storage with PolyServe Oracle RAC on Linux solution allows admins to retain the single view of the storage, the Oracle instance files, and the servers that they had with the single large Unix system.

The OCFS product is still relatively immature and unstable. The soon to be released product from Veritas is untested in the Linux world and based on an architecture that has a fundamental bottleneck. There is no question we are more experienced than Veritas on Linux, we have large production sites running today, we can out perform them, we don’t have the bottlenecks they face, and we are more reliable.

MxS is fully symmetric and has no single resource that could be a source of contention. Red Hat bought Sistina and integrated it’s CFS product as GFS. Combined with their Cluster Suite (CS) they have High Availability capability. They also support SANs and MPIO. They have a remarkable lack of customer testimonials…

8 licenses, MxS 2.6.2, MxDB 2.1.2, ODM 2.1.2 (12 licenses total)

IBM servers, “stand alone 4-way boxes” (xSeries 255 ?)

EMC DMX5000, Emulex LP10000, McData Spherion 4500

Novell SuSE Linux

Egenera w/ OCFS was a competitor—I believe this was a 2nd surgery as Egenera was not meeting their needs

Doug & Yoffee convinced Cy and team of the benefits of PSFS over OCFS

ApplTop Shared home for Application server, plus shared home for Oracle 9i RAC drives significant management improvement in upgrade and patch processes

Problems with SANs without PolyServe:

No shared data

No shared binaries

High availability is hard because of SCSI reservation issues

Shifting workloads is cumbersome, causing poor server utilization

PolyServe realizes the potential of SANs and industry standard servers

Fundamentally changes the relationship between servers and storage and creates an “any to any” relationship

Allows servers as storage to be viewed as interchangeable but unified resources

Matrix Server maintains data integrity in the face of high speed, concurrent data reads and writes with little overhead

Matrix Server is built on top of a high availability infrastructure that absorbs server, storage, network, and application failures and eliminates, or minimizes, user impact

Since all servers can own all data with no fear of data corruption, access to data transitions quickly and certainly to designated back up servers

Matrix Server allows clusters to be comprised of multiple server types: Vendors, processor counts, clock speeds, memory compositions and Operating systems, providing maximum utilization of server capacity

Unlike some high availability solutions

Provides flexibility in moving applications between servers during peak and no-peak load situations

Because of the high availability infrastructure, and the data sharing and data integrity measures, server and storage capacity can be added to applications without affecting other servers, storage, or applications

Even the application to which it is being added if it is a scale-out application like file serving

With less than 90 seconds of disruption if it is not a scale-out application, like SQL Server

Matrix Server provides a Matrix-wide console showing the status of all servers in respect to applications running on servers, and the status of the servers

Applications can be shifted between servers with one click

Capacity can be added to applications with one click

Problems with SANs without PolyServe:

No shared data

No shared binaries

High availability is hard because of SCSI reservation issues

Shifting workloads is cumbersome, causing poor server utilization

PolyServe realizes the potential of SANs and industry standard servers

Fundamentally changes the relationship between servers and storage and creates an “any to any” relationship

Allows servers as storage to be viewed as interchangeable but unified resources

Matrix Server maintains data integrity in the face of high speed, concurrent data reads and writes with little overhead

Matrix Server is built on top of a high availability infrastructure that absorbs server, storage, network, and application failures and eliminates, or minimizes, user impact

Since all servers can own all data with no fear of data corruption, access to data transitions quickly and certainly to designated back up servers

Matrix Server allows clusters to be comprised of multiple server types: Vendors, processor counts, clock speeds, memory compositions and Operating systems, providing maximum utilization of server capacity

Unlike some high availability solutions

Provides flexibility in moving applications between servers during peak and no-peak load situations

Because of the high availability infrastructure, and the data sharing and data integrity measures, server and storage capacity can be added to applications without affecting other servers, storage, or applications

Even the application to which it is being added if it is a scale-out application like file serving

With less than 90 seconds of disruption if it is not a scale-out application, like SQL Server

Matrix Server provides a Matrix-wide console showing the status of all servers in respect to applications running on servers, and the status of the servers

Applications can be shifted between servers with one click

Capacity can be added to applications with one click

Data Center Linux

2005-03-09T16:15:33