Ensuring that Your Application will run on any Linux Distribution

Darren R. Davis

Senior Software Engineer

Novell, Inc.

Paul MacKay

Senior Software Engineer

Novell, Inc.

Session Overview

This Developer Hands-On Session accompanies the following lectures

DL 100 – Linux Porting and Migration Strategies

DL 103 – Developing LSB/FHS Compliant Applications

DL 104 – Porting Red Hat Applications to Novell SUSE Linux

DL 105 – Creating RPMs for Novell SUSE Linux

We will be doing “Hands On” exercises from these lectures. We will perform them in the session and you can follow along at the same time.

Open Questions and Answers!

Presenter Overview

Senior Software Engineer working in Novell Strategic Partner Engineering group helping ISVs to port their applications to SUSE Linux.

Masters degree in Computer Science from Utah State University.

I have worked in the UNIX and Linux industry since 1987 in roles of development, management, technology strategy and technology evangelism for several companies.

I have been involved in the Open Source Community since 1993. Started by helping move open source software to Novell UnixWare. Using since 1987.

Darren R. Davis (mailto://ddavis@novell.com)

Presenter Overview

Senior Software Engineer working in Novell Developer Services Division. Helps define Novell strategy in working with third-party ISVs (Independent Software Vendors) and IHVs (Independent Hardware Vendors).

Paul MacKay (mailto://pmackay@novell.com)

Session Outline

Problem Definition and Demonstration

C Applications

API and ABI definition

Libraries

libc.so – C run-time library

libdl.so – Dynamic library

libm.so – math library

libs etc...

C++ Applications and ABI Issues

What is the issue?

Demo: C++ ABI Problem

Session Outline (cont.)

Potential Solutions to Portability Issues

LSB – Linux Standard Base

POSIX – API

LSB – ABI

DEMO: Using LSB tools

FHS – Filesystem Hierarchy Standard

DEMO: Show the filesystem layout

RPMs – Application Packaging

DEMO: Making an RPM

(What about tar archive/compress?)

(What about custom installers?)

Session Outline (cont.)

Potential Solutions to Portability Issues

Build Environments

DEMO: Build.RPM

Build Buddy

Autoconf - http://gnu.org/software/autoconf

Autopackage - http://autopackage.org/

C++ ABI DEMO

C++ Standard Library ABI Issue

From: http://gcc.gnu.org/onlinedocs/libstdc++/abi.html

Library API + Compiler ABI = Library ABI

GCC has the option to set ABI with the flag

-fabi-version

ABI checkers

Intel

http://developer.intel.com/software/products/opensource/

ABICheck

http://abicheck.sourceforge.net/

C++ Standard Library ABI Issue

#include <iostream>

int main()

{ std::cout << "Hello C++ World!" << std::endl; return 0; }

> g++ hello.cc -o hello.out

> ldd hello.out

libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x00764000)

libm.so.6 => /lib/tls/libm.so.6 (0x004a8000)

libgcc_s.so.1 => /mnt/hd/bld/gcc/gcc/libgcc_s.so.1 (0x40016000)

libc.so.6 => /lib/tls/libc.so.6 (0x0036d000)

/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x00355000)

> nm hello.out

nm man page

NM(1) GNU Development Tools NM(1)

NAME

nm - list symbols from object files

SYNOPSIS

nm [-a│--debug-syms] [-g│--extern-only] [-B] [-C│--demangle[=style]] [-D│--dynamic]

[-S│--print-size] [-s│--print-armap] [-A│-o│--print-file-name]

[-n│-v│--numeric-sort] [-p│--no-sort] [-r│--reverse-sort] [--size-sort] [-u│--undefined-only]

[-t radix│--radix=radix] [-P│--portability] [--target=bfdname] [-fformat│--format=format]

[--defined-only] [-l│--line-numbers] [--no-demangle] [-V│--version] [-X 32_64] [--help] [objfile...]

DESCRIPTION

GNU nm lists the symbols from object files objfile.... If no object

files are listed as arguments, nm assumes the file a.out.

ldd man page

LDD(1) LDD(1)

NAME

ldd - print shared library dependencies

SYNOPSIS

ldd [OPTION]... FILE...

DESCRIPTION

ldd prints the shared libraries required by each program or shared

library specified on the command line.

Linux Standard Base

Linux Standard Base

“We strive to develop and promote a set of standards that will increase compatibility among Linux distributions and enable software applications to run on any compliant system. In addition, the LSB will help coordinate efforts to recruit software vendors to port and write products for Linux.”

Also described as “Shrink-Wrapped Applications”

HTTP://www.linuxbase.org/

Linux Standard Base

IBM Press:

Building Applications with the Linux Standard Base

(ISBN 0-13-145695-4)

http://lsbbook.gforge.freestandards.org/lsb-book.html

Free on the Internet

Linux Standard Base (APIs and Libs)

libc – Standard C Library

libdl – Dynamic Loader Library

libm – Math Library

libutil – Utility Functions Library

libcrypt – Cryptology Library

libz – Compression Library

libpthread – POSIX (Linux) Threads Library

(Use the new NPTL – Native POSIX Thread Library)

LSB

Linux Standard Base (APIs and Libs)

libncurses – Text/Terminal User Interface Library

libX11 – X Windows System Interface Library

libXext – X Windows Extensions Library

libXt – X Toolkit Intrinsics Library

libICE – X Inter-Client Exchange Library

libSM – X Session Management Library

libGL – OpenGL Graphics Library

LSB

Linux Standard Base (Using)

Commands

/opt/lsb/bin/lsbappchk

/opt/lsb/bin/lsblibchk

/opt/lsb/bin/libchk

/opt/lsb/bin/lsbcc

/opt/lsb/bin/lsbc++

/opt/lsb/bin/dummy_link

Libraries and Headers

/opt/lsb

Linux Standard Base (Using)

#include <stdio.h>

#include <unistd.h>

main()

{

printf(“Hello world: %d\n”, getpid());

}

Example “Hello World”

Linux Standard Base (Using)

$ lsbcc -o hw_good helloworld.c

$ lsbappchk hw_good

lsbappchk for LSB Specification 2.0

Checking binary hw_good

$

Checking for LSB Compliance

Linux Standard Base (Using)

#include <stdio.h>

#include <unistd.h>

main()

{

printf(“Hello world: %d\n”, _getpid());

}

Example Bad “Hello World”

Linux Standard Base (Using)

$ lsbcc -o hw_bad helloworld.c

/tmp/cc5CITzio.o: In function 'main':

/tmp/cc5CITzio.o(.text_0xd): undefined ref..

collect2: ld returned 1 exit status

$

Command lsbcc fails

Linux Standard Base (Packaging)

LSB Defines a standard package format

RPM

HTTP://www.rpm.org

RPM Checking

$ lsbpkgchk lsb-mypkg

LSB Defines a standard location for application files

File installation location standard

Filesystem Hierarchy Standard (FHS)

HTTP://www.pathname.com/fhs/

LANANA – Linux Assigned Names and Numbers Authority

HTTP://www.lanana.org/

FHS DEMO

/ (root) FHS Hierarchy

/bin	- Essential command binaries

/boot	- Static files of the boot loader

/dev	- Device files

/etc	- Host-specific system configuration

/home	- User home directories (optional)

/lib	- Essential shared libraries and kernel modules

/lib32	- 32-bit architecture dependent libraries (optional)

/lib64	- 64-bit architecture dependent libraries (optional)

/media	- Mount point for removable media

/mnt	- Mount point for mounting a filesystem temporarily

/opt	- Add-on application software packages

/proc	- Linux filesystem for handling process and system info

/root	- Home directory for the root user (optional)

/sbin	- Essential system binaries

/srv	- Data for services provided by this system

/tmp	- Temporary files

/usr	- Secondary hierarchy

/var	- Variable data

root

/user FHS Hierarchy

/usr/bin	- Most user commands

/usr/include	- Header files included by C programs

/usr/lib	- Libraries

/usr/local	- Local hierarchy (empty after main installation)

/usr/sbin	- Non-vital system binaries

/usr/share	- Architecture-independent data

/usr/X11R6	- Xwindow System, version 11 release 6 (optional)

/usr/games	- Games and educational binaries (optional)

/usr/lib<qual>	- Alternate Format Libraries (optional)

/usr/src	- Source code (optional)

/var FHS Hierarchy

/var/cache	- Application cache data

/var/lib	- Variable state information

/var/local	- Variable data for /usr/local

/var/lock	- Lock files

/var/log	- Log files and directories

/var/opt	- Variable data for /opt

/var/run	- Data relevant to running processes

/var/spool	- Application spool data

/var/tmp	- Temporary files preserved between system reboots

RPM DEMO

Where to start?

The best place to start is to analyze existing source RPM files

hello-<latest-version>-src.rpm

A simple RPM to start with is the GNU Hello world package

What is in a Source RPM?

A source RPM contains a spec file, a tarball containing the source files and perhaps some patch files

To look at the content of a source RPM use rpm2cpio and cpio

For example:

rpm2cpio hello-2.1.1-309.2.src.i386.rpm > hello.cpio

cpio -id < hello.cpio

Where are Source RPMs Installed?

A source RPM it is installed under “/usr/src/packages”.

If you install hello<latest-version>.rpm the following files are installed:

/usr/src/packages/SOURCES

/usr/src/packages/SOURCES/hello-2.1.1.tar.gz

/usr/src/packages/SPECS/hello.spec

RPM Package Building Directories

/usr/src/packages/BUILD

Area where the source is unpacked and built

/usr/src/packages/SOURCES

Contains the original source and patch files

/usr/src/packages/SPECS

Contains the package's spec file

/usr/src/packages/RPMS

Contains the created binary RPM

/usr/src/packages/SRPMS

Contains the created source RPM

Build the Existing Source RPM

To build an RPM in /usr/src/packages you need to login as root

Change directory to the /usr/src/packages/SPECS directory and type:

rpmbuild -ba hello.spec

rpmbuild

It is used to build both binary and source packages

rpmbuild major options:

-bp

Execute %prep section

-bc

Execute %build section

-bi

Execute %install section

-bs

Build source package

-bb

Build binary package

-ba

Build both source and binary packages

Build the Existing Source RPM?

If you are lucky it may succeed. However, most of the time it does not

What are some of the issues in creating RPMs using only the rpmbuild Tool?

Issues in Creating RPMs

Using rpmbuild to create RPMs poses several issues for the developer/packager:

Build dependencies must be managed by hand

The installation of required packages to build the package is often determined by trail and error (run configure and see what happens)

Not building in an isolated environment

The current system may have tools and libraries that are not compatible or reflect the target installation environment

Easy to pollute the current environment

Testing the installation of binary RPMs may affect the current system because of RPM dependencies

Build.RPM DEMO

SUSE's Build Tool

build is a tool used to create RPMs in an isolated environment

build will install a minimal SUSE Linux environment into a given directory, chroot to this created Linux environment, and compile the package

SUSE's Build Tool (cont.)

build searches the spec file for an extra line “BuildRequires: ”

If such a line is found, all specified RPMs after the “BuildRequires: ” tag are installed into the temporary Linux environment*. Otherwise a default set of packages will be installed

*Note: build does not automatically resolve missing dependencies. The specified rpms on the “BuildRequires:” line must be sufficient for the build to succeed

Adding the BuildRequires Section

The “ buildRequires: ” section is a list RPM package names separated by a space all on a single line

BuildRequires: aaa_base acl attr bash bind-utils bison bzip2 coreutils cpio cpp cracklib cvs cyrus-sasl db devs diffutils e2fsprogs file filesystem fillup findutils flex gawk gdbm-devel glibc glibc-devel glibc-locale gpm grep groff gzip info insserv kbd less libacl libattr libgcc libselinux libstdc++ libxcrypt m4 make man mktemp module-init-tools ncurses ncurses-devel net-tools netcfg openldap2-client openssl pam pam-modules patch permissions popt procinfo procps

For example:

Advantages in Using build

Building packages with build:

Ensures a consistent process for creating RPMs

Isolates a build environment from the current system (chroot)

Specifies a convention for what is required to build your package

Installs required RPMs in the chroot'ed environment automatically

Checklist

Make sure your code is “lint-free”

Cleanup any non-portable code

Use standard tools (make, autotools, etc.)

Use “ldd” to analyze what libraries and versions are required for your application

Ensure SUSE versions of the required libraries are installed on your SUSE system

Programmatically check for check library versions

Statically link or bundle unique libraries

Compile on SUSE using official versions of gcc (3.2, 3.3 etc.)

Checklist (cont.)

If possible make your existing RPM relocatable

Use “rpm –-prefix” or/and “rpm –-relocate” to install the package in a temporary area

Make the appropriate changes to your application's installation to be LSB/FHS compliant

Applications with init scripts that are installed/removed RPM spec files should use:

“/usr/lib/lsb/install_initd” in “%post” section

“/usr/lib/lsb/remove_initd” in “%preun” section

Checklist (cont.)

Use only bash functions specified in the LSB in init scripts to ensure they exist on SUSE

Be aware of which threading model your application uses (NPTL or not) and understand what level of support is available (NPTL is not in SUSE version 8 but it in SUSE 9)

Create and maintain good test scripts/programs that can validate application behavior on both systems

If possible, compile your application on a 64-bit architecture (if it does not already run on it)

Get your application Novell YES Certified

Question and Answer

Novell wants to help!

Novell Developers

HTTP://developer.novell.com/

Novell Linux Migration

HTTP://developer.novell.com/linux/migration.html

The root filesystem must contain all files and directories to boot, restore, recover, and/or repair the system. The goal is to keep the root hierarchy as minimal as possible, containing only the essentials to meet the afore mentioned purposes.

Applications must never create or require special files or subdirectories in the root directory.

The /usr hierarchy conains shareable and read-only data. To be FHS compliant systems and applications must not write date into this hierarchy.

Large applications must not use a direct subdirectory under /usr hierarchy.

The /var hierarchy contains variable data files. It only contains transient and temporary files. Some portions of /var are not shareable between different systems (/var/mail, /var/cache/man, /var/lock, /var/log etc.)

/var makes it possible for /usr to be read-only.

Applications must not generally add subdirectories to /var

You could do this all in one line

rpm2cpio hello-2.1.1-alt1.src.rpm | cpio -id

As one can see, to build packages in this area one needs to be root. There are was to create rpm packages by not being root. However, some packages require in building and installing to be in system directory structures.

You can see what the default packages that will be installed by looking at

/usr/lib/build/init_buildsystem

DHO 200 - Ensuring that your application will run on any Linux distribution.

2003-12-08T12:43:39