Open Source Development
Platform Walk-Through

Bart Vergauwen

Novell

Bavo De Ridder

Novell

Frank De Bolle

Novell

Agenda

Introduction
History

Open Source

JBoss AS, Tomcat

PFC and Velocity

Hibernate

Eclipse, Ant

MySQL

Introduction

Internal Novell NTS MAT EMEA : Team Scheduling Spreadsheets

Goal : To schedule the team members for PSE/DSE support for NTS MAT EMEA customers

Day to day planning

Indicate Backup's

Gather statistics

Trace work days, vacation, training,...

Introduction

Setting the scene : How it was before

Taking the decision : Which development platform ?

The Open Source Way ?

Yes ==> The Open Source Development Platform

Why not Novell exteNd ?

How it was before...

How it was before...

Microsoft Excel spreadsheets

One per team across Novell EMEA

Only visible to that team

Only exchange of data between teams via copy/paste...

No central repository of data

Different flavors of the Scheduling spreadsheet

Each team having their own macros, formulas, etc..

Each team having their own way of working

Each team having their own requirements

Taking the decision...

Which development platform?

Criteria

Some specific requirements

The decision

Novell Open Source commitment

JBoss support

Existing framework

Different Tools

The Big Picture

Application Classes

Portal Foundation Classes

Hibernate

MySQL

JBoss J2EE AS

Velocity

Tomcat

The way it is now

JBoss J2EE AS

The JBoss JMX Micro-Kernel

JTS/JTA

EJB Container

Remote Management

JMS

Security

Databases

Data Sources

JAVA Server Pages

JMX Implementation

Jboss Application
Architecture Overview

JTS/JTA

EJB Container

Remote Management

JMS

Security

Databases

Data Sources

JAVA Server Pages

JMX Implementation

Jboss Application
Architecture Overview

Jboss Application
Architecture Overview

The JBoss Application Server architecture can be divided

into four primary layers:

Microkernel Layer

At its core is a microkernel-based server that is extremely small in footprint

By utilizing Java Management Extensions (JMX), the microkernel delivers a lightweight component model that offers hot deployment and advanced class-loading features and full lifecycle management.

Services Layer

On top of the microkernel resides a Services Oriented Architecture (SOA).

It consists of a series of services, each of which are neatly packaged and fully hot-deployable.

Services deployed in JBoss can range anywhere from transaction and messaging services to mail services to security services to a connection pool.

You can easily add new services or remove unnecessary services from your deployments to reduce the overall footprint and improve performance.

You can also build your own services and deploy them as SARs within JBoss Application Server.

Each service is packaged as a Service Archive or SAR and each SAR is individually hot-deployable, making it extremely easy to extend JBoss.

1

2

Jboss Application
Architecture Overview

The Aspect Layer

The aspect layer is based on the Aspect-Oriented Programming (AOP) model.

A long running tradition of the JBoss Application Server is the concept of interceptors.

Interceptors enable the system to transparently add the behavior provided by the services into any objects.

The JBoss EJB container, for example, is built as a pre-packaged set of interceptors.

You can easily add or remove interceptors based upon your specific needs.

The latest 4.0 release of the JBoss Application Server simplifies programming by allowing you to easily weave in these aspects with tag-driven development, a bit like C#.

Application Layer

This is where your applications live.

Your applications can greatly leverage the vast infrastructure capabilities of JBoss regardless of whether you utilize the container services directly (programmatically) or by using the AOP layer and tag-driven aspects to add behaviors to your objects.

JBoss offers a number of very useful features such as No Compilation that have made JBoss Application Server the choice for serious Java developers.

3

4

MySQL

MySQL Architecture

MySQL

Server

TH1

TH2

Application

Application

MySQL

Server

TH1

TH2

Application

Application

Table handlers

(MyISAM, InnoDB,etc)

MySQL Architecture

MyISAM

InnoDB

Memory

NDB

MySQL

Parse

Optimize

Retrieve

Store

MySQL Database

Management Level

Storage Engine

Level

MySQL Architecture

Four storage engines, each with different characteristics:

MyISAM is a disk based storage engine. Aiming for very low overhead, it does not support transactions.

InnoDB is also disk based, but offers versioned, fully ACID transactional capabilities. InnoDB requires more disk space than MyISAM to store its data, and this increased overhead is compensated by more aggressive use of memory caching, in order to attain high speeds.

Memory (formerly called "HEAP") is a storage engine that utilizes only RAM. Special algorithms are used that make optimal use of this environment. It is very fast.

NDB, the MySQL Cluster Storage engine, connects to a cluster of nodes, offering high availability through redundancy, high performance through fragmentation (partitioning) of data across multiple node groups, and excellent scalability through the combination of these two. NDB uses main-memory only, with logging to disk.

MySQL

MySQL version 4.1

Major Features

Enhanced binary protocol

Client-integrated help system

Better I18N support (including Unicode support)

Prepared statements

Spatial data support

Subqueries

Warning system

MySQL version 5.0

Major Features

Cursors

Stored Procedures

(Updatable) Views

MySQL

Tomcat

Tomcat Architecture

Hibernate

Hibernate Architecture

A (very) high-level view of the Hibernate architecture

Shows Hibernate using the database and configuration data to provide persistence services (and persistent objects) to the application

Hibernate Architecture

A more detailed view of the runtime architecture

Hibernate is flexible and supports several approaches

The "lite" architecture has the application provide its own JDBC connections and manage its own transactions

This approach uses a minimal subset of Hibernate's APIs

Hibernate Architecture

A more detailed view of the runtime architecture

Hibernate is flexible and supports several approaches

The "full cream" architecture abstracts the application away from the underlying JDBC/JTA APIs and lets Hibernate take care of the details.

Hibernate Architecture
Some definitions

SessionFactory (net.sf.hibernate.SessionFactory)

A threadsafe (immutable) cache of compiled mappings for a single database. A factory for Session and a client of ConnectionProvider. Might hold an optional (second-level) cache of data that is reusable between transactions, at a process- or cluster-level.

Session (net.sf.hibernate.Session)

A single-threaded, short-lived object representing a conversation between the application and the persistent store. Wraps a JDBC connection. Factory for Transaction. Holds a mandatory (first-level) cache of persistent objects, used when navigating the object graph or looking up objects by identifier.

Persistent Objects and Collections

Short-lived, single threaded objects containing persistent state and business function. These might be ordinary JavaBeans/POJOs, the only special thing about them is that they are currently associated with (exactly one) Session. As soon as the Session is closed, they will be detached and free to use in any application layer (e.g. directly as data transfer objects to and from presentation).

Hibernate Architecture
Some definitions

Transient Objects and Collections

Instances of persistent classes that are not currently associated with a Session. They may have been instantiated by the application and not (yet) persisted or they may have been instantiated by a closed Session.

Transaction (net.sf.hibernate.Transaction)

(Optional) A single-threaded, short-lived object used by the application to specify atomic units of work. Abstracts application from underlying JDBC, JTA or CORBA transaction. A Session might span several Transactions in some cases.

ConnectionProvider (net.sf.hibernate.connection.ConnectionProvider)

(Optional) A factory for (and pool of) JDBC connections. Abstracts application from underlying Datasource or DriverManager. Not exposed to application, but can be extended/implemented by the developer.

TransactionFactory (net.sf.hibernate.TransactionFactory)

(Optional) A factory for Transaction instances. Not exposed to the application, but can be extended/implemented by the developer.

Hibernate Architecture
Some definitions

Given a "lite" architecture, the application bypasses the Transaction/TransactionFactory and/or ConnectionProvider APIs to talk to JTA or JDBC directly.

JMX Integration

JMX is the J2EE standard for management of Java components. Hibernate may be managed via a JMX standard MBean but because most application servers do not yet support JMX, Hibernate also affords some non-standard configuration mechanisms.

Hibernate to run as a JMX component inside Jboss.

JCA Support

Hibernate may also be configured as a JCA connector.

Hibernate Integration

Hibernate is a very popular object persistence framework developed by Jboss

It maps Java objects to tables in relational databases and vice versa.

The object-relational mapping rules and data sources are specified in special Hibernate configuration files.

In JBoss AS 4.0, Hibernate integration support is provided by the jboss-hibernate.deployer service, which is available in the default, standard and all configurations.

The jboss-hibernate.deployer service provides Hibernate framework libraries to all application on the server.

Hibernate Integration

For Hibernate applications, JBoss defines a new .har service archive type.

You can package your Hibernate mapped Java objects and mapping configuration files in the .har archive.

You can also specify a data source name and an JNDI name for this particular Hibernate configuration in the hibernate-service.xml file in the .har archive.

The benefit is that, in your applications, you only need to do a JNDI lookup to retrieve the correctly configured Hibernate SessionFactory object.

There is no need to load the mapping and data source configuration files manually in the application via API calls. In addition, the configuration settings in the hibernate-service.xml file is manageable via the JBoss JMX management console.

The .har file can be bundled inside a .ear file or deployed standalone.

Internal Tool Persistence Layer : Hibernate

O/R mapping tool

Open source (LGPL)

Mature software, driven by user requests

Popular (15.000 downloads per month)

using version 2.1.6

www.hibernate.org

Why O/R Mapping Tool ?

Writing a persistence layer is non-trivial, why re-invent the wheel ?

Productivity

API for CRUD operations

Query API

Performance

efficient fetching strategies

caching

database specific optimizations (e.g. prepared statement caching)

Architectural Overview

Steps

Steps

Starting point : business model

Persistent classes

Define O/R mapping

Configure JDBC connections

Use Hibernate Session API

(Performance) Tuning

Steps

Steps

Starting point : business model

Persistent classes

Define O/R mapping

Configure JDBC connections

Use Hibernate Session API

(Performance) Tuning

Business Model Snipplet

Team

Engineer

id

lastName

team

customers

skills

Customer

Skill

Steps

Steps

Starting point : business model

Persistent classes

Define O/R mapping

Configure JDBC connections

Use Hibernate Session API

(Performance) Tuning

Persistent Classes

no-arg constructor (may be private)

define getter/setters for persistent properties (using JavaBeans naming convention) :

Type getPropertyName() { ... }

void setPropertyName(Type value) { ... }

collection property is an interface

java.util.set, java.util.list, java.util.map

(optional) identifier property

mapped to primary key of database

Example:
Basic Property

// property “lastName”

private String _lastName;

public String getLastName()

{

return _lastName;

}

public void setLastName(String lastName)

{

_lastName = lastName;

}

Example:
Identifier Property

// property “id”

private String _id;

public String getId()

{

return _id;

}

private void setId(String id)

{

_id = id;

}

Example : Component Property

// property “address” (Engineer)

private Address _address;

public Address getAddress()

{

return _address;

}

public void setAddress(Address address)

{

_address = address;

}

Address Class

public class Address

{

public String getStreet() { ... }

public void setStreet(String street) { ... }

public String getCity() { ... }

public void setCity(String city) { ... }

// business method

public void sendLetter(String text) { ... }

}

Example:
Persistent Object Ref Property

// property “team”

private Team _team;

public Team getTeam()

{

return _team;

}

public void setTeam(Team team)

{

_team = team;

}

Example:
Collection Property (one-to-many)

// property “customers”

private Set _customers; // set of Customers

public Set getCustomers()

{

return _customers;

}

public void setCustomers(Set customers)

{

_customers = customers;

}

Example:
Collection Property (many-to-many)

// property “skills”

private Set _skills; // set of Skills

public Set getSkills()

{

return _skills;

}

public void setSkills(Set skills)

{

_skills = skills;

}

Persistent Classes:
Conclusion

Transparent Persistence

persistence has no impact on business object code

existing business objects can be persisted without change

no interfaces have to be implemented

no superclass to be extended

no restrictions on behaviour (business methods)

Steps

Starting point : business model

Persistent classes

Define O/R mapping

Configure JDBC connections

Use Hibernate Session API

(Performance) Tuning

O/R Mapping

defined in XML metadata

mapping is java-centric :

persistent classes are mapped to tables

properties are mapped to columns

Hibernate “mapping types”

object associations and collections involve

foreign keys

association/link tables (many-to-many)

class inheritance : 3 mapping strategies

single table per class hierarchy

single table per concrete subclass

subclass tables

Example

<class name=“Engineer” table=“ENGINEER”>

<property name=“lastName” column=“LAST_NAME”/>

<id name=“id” column=“ENGINEER_ID”>

<generator class=“native”/>

</id>

<many-to-one name=“team” column=“TEAM_ID”/>

Example (cont.)

<set name=“customers”>

<key column=“ENGINEER_ID”/>

<one-to-many class=“Customer”/>

</set>

<set name=“skills” table=”ENGINEER_SKILL”>

<key column=“ENGINEER_ID”/>

<many-to-many class=“Skill” column=”SKILL_ID”/>

</set>

Example (cont.)

<component name=“address”>

<property name=“street” column=“STREET”/>

<property name=“city” column=“CITY”/>

</component>

</class>

Components

Components may contain :

basic properties (<property> tag)

many-to-one's (<one-to-many> tag)

sets (one-to-many, many-to-many) (<set> tag)

nested component

A set may contain components as its elements

==> fine-grained object models (“more classes than tables”)

Mapping Types

<property name=“lastName” column=“LAST_NAME”/>

upon saving an engineer object :

java.lang.String ==> VARCHAR

upon loading an engineer object :

VARCHAR ==> java.lang.String

<property name=“name” column=“NAME” type=”string”/>

Hibernate “mapping types”specify how to map Java types to/from SQL types

built in types : string, integer, date, time, ...

create custom mapping types, e.g. for type-safe enumeration types

Bidirectional Navigation (cont.)

// property “engineers” (Team)

public Set getEngineers() { ... }

public void setEngineers(Set engineers) { ... }

<set name=“engineers” inverse=”true”>

<key column=“TEAM_ID”/>

<one-to-many class=“Engineer”/>

</set>

Steps

Steps

Starting point : business model

Persistent classes

Define O/R mapping

Configure JDBC connections

Use Hibernate Session API

(Performance) Tuning

JDBC Connectivity

Managed environment (J2EE server) : JNDI datasource

hibernate.connection.datasource = java:comp/env/jdbc/NessiDS

Non-managed environment (e.g. Swing client) : Hibernate can create its own connection pool

hibernate.connection.driver_class = com.mysql.jdbc.Driver

hibernate.connection.url = jdbc:mysql://localhost/Nessi

hibernate.connection.username = root

hibernate.connection.password = ...

hibernate.connection.pool_size = ...

built in support for C3PO, DBCP, and Proxool

can plug in custom pool provider

Hibernate Configuration File (hibernate.cfg.xml)

<!DOCTYPE hibernate-configuration PUBLIC "-//Hibernate/Hibernate Configuration DTD//EN" "http://hibernate.sourceforge.net/hibernate-configuration-2.0.dtd">

<hibernate-configuration>

<session-factory>

<property name="connection.datasource">java:MySqlDS</property>

<property name="dialect">net.sf.hibernate.dialect.MySQLDialect</property>

...

<mapping resource="nessi/data/Engineer.hbm.xml"/>

<mapping resource="nessi/data/Team.hbm.xml"/>

<mapping resource="nessi/data/Customer.hbm.xml"/>

<mapping resource="nessi/data/Skill.hbm.xml"/>

...

</session-factory>

</hibernate-configuration>

Steps

Starting point : business model

Persistent classes

Define O/R mapping

Configure JDBC connections

Use Hibernate Session API

(Performance) Tuning

Hibernate Session

encapsulates a conversation between the application and the persistent store (RDBMS)

provides

basic persistence services (CRUD)

querying API

serves as a (1st level) cache of persistent objects

holds a JDBC connection

transactional (local or 2PC)

guarantees that database identity equals java instance identity (i.e. a persistent object exists at most within scope of a given session)

non recoverable

Obtaining a Session Handle

// create configuration object

Configuration conf = new Configuration();

conf.configure(“hibernate.cfg.xml”);

// create sessionfactory (heavyweight)

SessionFactory sf = conf.buildSessionFactory();

// create session (lightweight)

Session session = sf.openSession();

... use the session ...

session.close();

Example Using a Session

// adding an Engineer to a Team

Session session = ...

Transaction trx = session.beginTransaction();

Team team = (Team) session.get(Team.class, “teamId”);

Engineer engineer = (Engineer) session.get(Engineer, “engId”);

team.getEngineers().add( engineer );

team.update();

trx.commit();

Automatic Dirty Object Checking

// adding an Engineer to a Team

Session session = ...

Transaction trx = session.beginTransaction();

Team team = (Team) session.get(Team.class, “teamId”);

Engineer engineer = (Engineer) session.get(Engineer, “engId”);

team.getEngineers().add( engineer );

// team.update();

trx.commit();

Note : implemented using inspection (not interception)

Detached Object Support

Session session1 = sf.openSession();

Transaction trx1 = session1.beginTransaction();

Engineer engineer = (Engineer) session1.get(Engineerd.class, “engineerId”);

trx1.commit();

session1.close();

engineer.setLastName(“De Brolle”);

Session session2 = sf.openSession();

Transaction trx2 = session2.beginTransaction();

session2.update( engineer );

trx2.commit();

session2.close();

Querying

HQL - Hibernate Query Language

object oriented version of ANSI SQL

uses classes/properties instead of table/columns

understands association, inheritance, polymorphism

Criteria queries (QBC)

queries are constructed as set of objects

supports Query By Example (QBE)

Native SQL queries

direct passthrough with automatic mapping

Note :

both HQL and QBC support named query parameters and result paging (“limit” queries)

queries may be named/externalized in metadata

Example HQL

select engineer

from Engineer engineer

join engineer.customers as customer

where customer.name like :customerName

Query query = session.createQuery(“.....”);

query.setString(“customerName”, “...”);

List results = query.list();

Iterator result = query.iterate();

Steps

Starting point : business model

Persistent classes

Define O/R mapping

Configure JDBC connections

Use Hibernate Session API

(Performance) Tuning

Debugging strategies

Data fetching strategies

Concurrent object access (isolation)

Caching

Debugging Strategies

Turn on SQL logging (hibernate_show_sql = true)

Use P6Spy : an open source passthrough JDBC driver that delegates to a real JDBC driver and logs SQL statements (and values bound to prepared statements)

Data Fetching Strategies

Engineer eng = (Engineer) sess.get(Engineer.class, “id”);

... eng.getTeam() ....

... eng.getCustomers() ...

... eng.getSkills() ...

“Lazy” fetching : fetch data piecemeal as needed

only data that is actually used is fetched

lots of database roundtrips (n+1 select problem)

session must still be open (LazyInitializationException)

“Eager” fetching : prefetch using single (outer-join) SQL

fewer roundtrips (but big SQL)

may fetch too much data (Carthesian product problem)

Data Fetching Strategies (cont.)

You tell Hibernate which fetching strategy to use (depending on the use case) :

fetching strategies can be specified at the individual association level, e.g.

use eager loading for engineer.getTeam()

use lazy fetching for engineer.getCustomers()

fetching strategies can be specified

statically (defaults) : xml mapping metadata

dynamically (runtime) : at query level

(Batch fetching)

Example : Data Fetching Strategies

<many-to-one name=“team” column=“TEAM_ID”

outer-join=”true” />

<set name=“customers” lazy=”true”>

<key column=“CUSTOMER_ID”/>

<one-to-many class=“Customer”/>

</set>

select engineer

from Engineer as engineer

join fetch engineer.customers

Concurrent Object Access (Isolation)

Database transaction isolation level

read uncommitted

read committed

repeatable read

serializable

Optimistic concurrency checks

check <version> property (or <timestamp> property)

check all properties

check dirty properties only

Pessimistic locking (database locking, never in memory !)

session.get(Engineer.class, id, lockmode)

session.lock(obj, lockmode)

Caching Architecture

Dual Layer Caching Architecture

Session_1

1st level cache

Session_n

1st level cache

2nd level cache

JVM or Cluster

Persistent Store

(RDBMS)

Session_2

1st level cache

2nd Level Caching

can have an enormous impact on performance

not a “fix” for bad design

can be configured on a per class/collection basis

optional

pluggable

assumes that all updates go via the cache (i.e. cache needs to know about updates) ! If not, cached data may be stale (and cache expiry policy is needed)

ideal caching candidate : reference data

PFC – Portal Foundation Classes

Topics

Web Frameworks Explained

Different types

Issues with web frameworks

Portal Foundation Classes

Overview

Technical Topics

Components

Model (Active Properties)

Validation

Rendering & Velocity

Services

Web Frameworks

Why a Web Application Framework?

higher level API that shields low level Servlet API

extension API for common and reusable parts

out-of-the-box experience

Web Framework Types

Page Centric Framework

Application Centric Framework

Page Centric Framework

set of pages (templates)

a well known flow (navigation) between the pages

a limited set of interaction

Examples:

Struts, Java Server Faces, PHP, ASP ...

Application Centric Framework

server side contains an object model of the GUI

unknown set of interactions

no “page” notion

suffers from Back Button syndrome

Examples:

Echo, WDialog, Portal Foundation Classes

Issues with Web Frameworks

Packaging

Distribution

Java Server Pages

Images, ...

Java Libraries (jar files)

Configuration Files (XML)

Upgrades?

Dead material!

Blind Swing Model / API Adoption

Not fit for request/response paradigm

heavy Java Script or high round-trip frequency

Introduction to PFC

What is it?

A J2EE 1.3 library for developing highly interactive web applications

Application Centric Framework

Project Facts

Initial coding started as early as 2000

SourceForge project since 2003

Used in production applications since 2003

http://www.sourceforge.net/projects/pfc

Architecture

Runtime Engine

Portal Foundation
Classes - API

Basic
Components

J2EE Application Server

Servlets, API implementation

Core UI

Components

Container

Rendering

Layout

Security

Security Service

Login Module

Services

Applications

Extensions

UI Extensions

Components

Wizard, Table

Services

Hibernate

Binding

Frameworks

Management Console

Snapins, Plugins

Object Model

Hierarchical object model for GUI objects

Top level base class Component

Common components available

Button, TextInput, TabbedPane, Wizard ...

Container GUI classes

Recursive object models

Component Container API

Abstracts away HTTP Request/Response

Components

Components

Components

Internal Engine Model

Portal Process
Servlet

Render Portal
Servlet

GraphicsDevice

SystemDesktop

Login
Desktop

Application
Desktop

Error
Desktop

Locked
Desktop

Your
Application

Desktop Management

Core Graphics Engine

Life of a POST/GET

UI
Object Model

GraphicsDevice

update fields

perform action

render

Portal Process
Servlet

Render Portal
Servlet

process

4

1

2

5

7

6

3

Rendering

Each GUI component can render itself independently

either by implementing the draw() method

or by setting a LayoutManager to do the drawing

Common Layout Managers

VelocityLayout

JspLayout

ContainerLayout

FlowLayout

TableLayout

Sample UI

Render Stack

Country
Input

FirstName
Input

LastName
Input

PhoneNumber
Panel

Simple
Render Context

DemoFormPanel

Area
Input

Local
Input

Velocity

draw()

draw()

draw()

draw()

draw()

Layout

Velocity
Render Context

Simple
Render Context

Render Stack

FlowLayout Example

VelocityLayout Example

Model (Active Properties)

Java Bean Property

private field

get<PropertyName>()

set<PropertyName>(...)

Can only be passed by value

Unless reflection: by name

Runtime errors (name is a String type)

Pass by reference

Active Properties

Model (Active Properties)

Active Properties

Represent the property as an object

Validation

A reusable and extensible validation framework

Validation rules

Recursiveness

Properties to validate passed by reference

Active Properties

Validation Feedback in GUI

Validation

Services

Support for Services

Constant available functionality

Public API to write own services

Framework managed life-cycle and dependencies

Out of the Box Services

Registry Service

Resources Service

Hibernate Service

Monitor Service

Binding Service (in development)

Packaging

JAR Package Contains

Java classes

Resources

Velocity templates

Images

Multi-language support

Configuration Files (XML)

Everything!

Advantages

Single entity, easily distributable

Just drop in class path

/WEB-INF/lib, EAR ...

Q&A

2003-12-08T12:43:39

Frank De Bolle