Categories

  • No categories

Archives

Google

Google
Web www.barmala.com

Yahoo

RPM recipes

» Linux » Software

 

 

The following article is not a complete tutorial on RPMs. It rather gives you an overview about relevant topics, pointers to more in depth material and “cooking recipes”, scripts and templates for some steps of the building process.

What is an RPM and why should I use or even build it?

Most Open Source software comes in the form of a “tarball”. To install it, you usually have to

tar -tzf progname-version.tar.gz # extract the archive
cd progname-version
sh configure # run the configuration utility
make # run the make utility to compile all sources
make install # install the product

Purists prefer that way, because

    1. That's the original, genuine product
    2. It is generic, i.e. not bound to a specific Linux distribution (e.g. SuSE, RedHat, Debian, ...)
    3. It's the most up-to-date version
    4. Building the product from source allows you to inspect the code to make sure, it doesn't contain any malicious components
    5. If necessary you can add your own modifications to the code

However there are some drawbacks:

    1. A development environment has to be installed. This is usually the case on the Linux enthusiast's machine, but not on Joe Ordinary's desktop
      1. The configure utility might need some parameters
      2. The code might need modifications
      3. The package might need some auxiliary files like a start/stop script for /etc/init.d
    3. The previous steps are necessary in order to
      1. run at all under this specific Linux distribution
      2. match with other products already installed or to be installed later
      3. match the distribution's standards

You might claim that beginners should learn how to run a compiler even if they don't know the programming language, but then the advantage 1.4 and 1.5 are void for them. You could also claim that 2.2.1 and 2.3.1 are just a matter of RTFM, and that's what beginners should learn most of all, but the other issues aren't that easy. Furthermore if everyone ends up with his answer to those issues, i.e. with his flavor of Linux, this may give you a feeling of programmer's freedom, but it makes support harder; for others, whom you ask for help as well as for yourself if you have to care for more than a single machine.

“RPM” is known as the “RedHat Package Manager”, but it's an open standard, which is used by other distributions (e.g. SuSE, Mandrake, PLD) too. The rpm.org home page therefore follows the GNU tradition, and declares it a recursive acronym for “RPM Package Manager”.

    1. RPMs in contrast to tarballs are distribution specific
    2. If the packager did a good job, RPMs fulfill all properties of 2.2
    3. RPMs are binaries, which don't need compilation
    4. RPMs can be easily installed by a newbie
    5. RPMs are cataloged in a database so you can easily determine what is installed on which system and dependencies between packages can automatically be checked

The best source for an RPM is the CD of your Linux distribution. You have good reason to hope that this RPM will satisfy best 2.3.3. If you use automatic update mechanisms (RedHat's up2date or SuSE's you) you'll always get the latest released version of all packages. The latest released RPM usually lags behind the latest released original (see 1.3), but this may be intentional in order to match with other released components of the system or to verify that the latest release doesn't create more problems than it solves.

Other sources for RPMs are

If you can't find an RPM for a specific product, your can either fall back to installing a tarball or build your own RPM. As you can imagine, 3.5 works best, if you always (de-)install via RPM and never do manual additions or deletions, like installing a tarball. This can be one of the motivations to write your own RPM.

Note: Debian uses a different package mechanism (“dpgk”) instead of RPM, but the intention is similar.

Using and naming RPMs

If you aren't yet familiar with using RPMs look up the manpage for the most important commands:

  • rpm --install somefile.rpm
  • rpm --erase package
  • rpm -qa

RPM-files follow a well defined naming convention. They might be called for instance:

  • postfix-2.1.1-1mdk.i586.rpm or
  • clamav-0.70-2.i586.rpm i.e.:
  • ProductName-ProductVersion-PackageVersion(System).Archive.rpm

“ProductName” is quite obvious. The “ProductVersion” is the version of the tarball on which the RPM is based. The “PackageVersion” numbers the RPM version independent of the underlying product version. Sometimes the PackageVersion contains a hint to the “System”, i.e. the Linux distribution for which it is tuned. “Archive” refers to the processor architecture, usually i386 or i586. Two special archive types exist:

  1. noarch describes code, which is independent of the underlying hardware, e.g. shell scripts.
  2. src refers to a SRPM which is described in more detail in the next section.

Building RPMs from SRPMs

“SRPM” stands for Source-RPM. An SRPM contains

  • A description of the package, which will go into and is checked against the central database
  • The original tarball
  • scripts to control the make and install process and to feed the configuration utility with the appropriate parameters
  • patch files to apply necessary modifications to the original sources
  • auxiliary files like a start/stop script for /etc/init.d

(Does this remind you to 2.2?) From these components you can easily rebuild the binary RPM. The build process is normally done in a directory structure under /usr/src (/usr/src/packages for SuSE or /usr/src/redhat). Properly crafted SRPMS however can be built as a normal, i.e. non-root user in the user's home directory. This is preferable to avoid messing up the system during tests.

Recreate the directory structure in your home directory and point to it:

#! /bin/sh
# Copyright (c) 2004 Christian Barmala http://www.barmala.de/
# License GNU Public License http://opensource.org/licenses/gpl-license.php
# @(#) $Id: rpmsetup.sh,v 1.0 2008/04/19 08:00:00 cvs Exp $
# create RPM directory structure under ${HOME} and point to it
# for RedHat replace "packages" with "redhat"

cd $HOME
cat >> .rpmmacros << EOF
%_topdir ${HOME}/packages
%_signature gpg
%_gpg_path ${HOME}/.gnupg/
%_gpg_bin `which gpg`
%_gpg_name $(grep `logname` /etc/passwd | cut -d : -f 5)
EOF

mkdir packages
cd packages
mkdir `ls /usr/src/packages`
cd RPMS
mkdir `ls /usr/src/packages/RPMS`
cd $HOME

(The rpmmacros about gpg are optional. Only _topdir is required.)

Now copy the SRPM to the ${HOME}/packages/SRPMS subdirectory and do a rpmbuild --rebuild --target=i586 ${HOME}/packages/SRPMS/program.src.rpm. When the rebuild finishes, you should find ${HOME}/packages/RPMS/i586/program.i586.rpm.

Building SRPMs from sources

As a starting point for further experiments, we'll simply dissect and reassemble an existing SRPM. The following scripts only contain minimal error checking code in order not to distract from the important lines starting with “rpm2cpio” and “rpmbuild”:

#! /bin/sh
# Copyright (c) 2004 Christian Barmala http://www.barmala.de/
# License GNU Public License http://opensource.org/licenses/gpl-license.php
# @(#) $Id: rpmdis.sh,v 1.0 2008/04/19 08:00:00 cvs Exp $
# dissect an SRPM
# for RedHat replace "packages" with "redhat"

if [ ! -f ${HOME}/packages/SRPMS/$1 ]; then
  missing ${HOME}/packages/SRPMS/$1
  echo usage $0 SRPMname without SRPMdirectory
  exit 1
fi
cd ${HOME}/packages/SOURCES
rpm2cpio ${HOME}/packages/SRPMS/$1 | cpio --extract # extract src.rpm file
if [ $? -ne 0 ]; then
  echo extract failed
  cd $HOME
  exit 2
fi
mv ${HOME}/packages/SRPMS/$1 ${HOME}/packages # move original SRPM to a safe place
cd $HOME
exit 0

You should now have some files in the ${HOME}/packages/RPMS/SOURCES directory. In some cases this is just the tarball and a .spec file. The spec file is an ASCII Text file, which controls the whole RPM building process.

#! /bin/sh
# Copyright (c) 2004 Christian Barmala http://www.barmala.de/
# License GNU Public License http://opensource.org/licenses/gpl-license.php
# @(#) $Id: rpmass.sh,v 1.0 2008/04/19 08:00:00 cvs Exp $
# reassemble an SRPM
# for RedHat replace "packages" with "redhat"

if [ ! -f ${HOME}/packages/SOURCES/$1 ]; then
  echo usage $0 SPECname without SOURCESdirectory
  exit 1
fi
mv ${HOME}/packages/SOURCES/$1 ${HOME}/packages/SPECS # move spec file to the SPECS dir
cd ${HOME}/packages/
rpmbuild -ba ${HOME}/packages/SPECS/$1 # rebuild the SRPM and the RPM from the spec file
cd $HOME
exit 0

You should find now ${HOME}/packages/RPMS/SRPMS/yourfile.src.rpm and ${HOME}/packages/RPMS/RPMS/ix86/yourfile.ix86.rpm beeing rebuilt.

Some authors of tarballs are so kind to include a spec file in the tarball. This way, the purists can go for the traditional way, while the rpm-lovers can easily build an rpm. In this case the process is even easier then the script rmpass.sh: Simply copy the tarball to the SOURCES directory and do a rpmbuild -ta ${HOME}/packages/SOURCES/tarball.tar.gz.

You may guess already what you have to do if you want to update an RPM to the latest version of the tarball:

  • Dissect the old rpm with rmpdis.sh
  • Replace the old tarball with the new one
  • Edit the spec file. Usually “version” is a macro which you only have to set once
  • Look at the other parts of the spec file and the patch file. Sometimes there is nothing else to do or the required changes are obvious. If they aren't continue reading and try to understand the whole process

For a discussion of the elements and the syntax of a spec file, refer to the tutorials under Links.

Warning

Many of the following steps have to be done as root and affect your system. You have to repeat these steps and some of them may turn out to be erroneous. Creating a backup of your system is the least thing you should do in order to be safe if you messed up your system. However backup and restore will take too long if you have to do multiple iterations. What you need is the ability to mirror your whole disk. “Split mirror backups” or the more economic alternative “Snapshots” can be done with a volume manager, a storage subsystem or a virtual machine like VMware or UML.

Before you continue to read and package a tarball into your RPM you should be able to compile and install the tarball properly. Depending on the product, this may just take the simple steps mentioned at the beginning of this article or it may take some investigation about the proper options, modifications and additions. As you can see, building an RPM instead of using the tarball doesn't save you any work, in contrary: It creates additional work. However once you've done this, your results are easy to reproduce by you and others. Building an RPM can also act like a checklist to make you aware of some configuration issues, which you might have overlooked if you just build and install a tarball. This won't make your life easier, but safer!

CheckInstall: The quick&dirty way

CheckInstall supervises the installation process of a program and creates a binary RPM from this trace. As an intermediate result, a spec file is generated and by setting an option, you can exempt this file from the cleanup process. This spec lacks some features of a full featured RPM, but some sections, especially the “%files” section, can be used as a starting point for a properly handcrafted version.

For a complete reference of the checkinstall command refer to its README. Checkinstall will ask you for input interactively when invoked without parameters, but it's preferable to use a script, because you may have to run it several times in a trial & error manner. Below you find a template for a script, which you have to adjust to your needs. You see that the last few lines are commented out. Some hints about this:

  • Until the invocation of checkinstall, you don't need to be root and you hardly have to worry about messing up your system. Run this script as regular user, then do an su or sudo and cut&paste the checkinstall line from the script to the command line.
  • When you arrive at the patch section for the first time, the script creates a copy of the current directory tree and stops. Edit all files, which require modifications and then create the patch file with the diff command. Then modify the script to execute the patch command on that file on all subsequent invocations.
  • Among the parameters of the configure command are those that control the location of certain files. The spec file template in the next section shows how to set them automatically to the correct values in most cases. Changing these parameters however means that the list of files generated by CheckInstall will change too. So you may have to modify this list when you cut and paste it from CheckInstall's output to the template.
#! /bin/sh
# Copyright (c) 2004 Christian Barmala http://www.barmala.de/
# License GNU Public License http://opensource.org/licenses/gpl-license.php
# @(#) $Id: rpmchk.sh,v 1.0 2008/04/19 08:00:00 cvs Exp $
# prepare a tarball for CheckInstall

progname=name of the tarball to convert to an RPM
version=version of the tarball

cd $HOME; mkdir tarball; cd tarball
wget URL, e.g. http://SourceForgeMirror/sourceforge/${progname}/${progname}-${version}.tar.gz
tar -tzf ${progname}-${version}.tar.gz
cd ${progname}-${version}
mkdir doc-pak # CheckInstall will look for this directory
cp [A-Z]* docs/* *.pdf doc-pak # these are good candidates for %doc in the %files section
cat > description-pak <<EOT1 # this goes to Summary and %description
The $progname Program 

Some verbose description
EOT1
cat > preinstall-pak <<EOT2
# the %pre section (not to mix up with %prep)
if [ make some checks ]; then
  echo Error
  exit 1
fi
do some other stuff
EOT2
cat > postinstall-pak <<EOT3
# the %post section
# e.g. install an init script, which doesn't come with the tarball
install -m755 ../initscript /etc/init.d/${progname}
EOT3
# optionally create preremove-pak for the %preun section
# and postremove-pak for the %postun section

# !! create the patch file !!
cd ..
cp -R ${progname}-${version} ${progname}-${version}p
# vi ... # edit files as necessary
diff -uNr ${progname}-${version} ${progname}-${version}p > ${progname}0.patch
exit 1
# !! end create the patch file !!

patch -p0 < ../${progname}0.patch
sh configure --with-option1=value1 --with-option2...make
su -c 'checkinstall --delspec=no make install'
exit 0

CheckInstall will present you a menu, which allows you to interactively configure the showHeader of the spec file and then generates an RPM file, which you can install, uninstall or pass to other users like any other RPM.

--delspec=no parameter tells checkinstall not to delete the spec file after the RPM is built. With this spec file you could build an SRPM from the tarball as described above, but such a file would have some drawbacks, one of which is, that it could only be built as root and that the build process would install the package on your build system, which usually is not desired. Better use this spec file, especially the %files section to fill in the template of the next section.

A template for a spec file

See notes below and documents under Links for the meaning of the sections.

# spec file for package progname

Name:         progname
Summary:      1 line summary
Version:      version of the tarball
Release:      version of the RPM
License:      GPL
Group:        see /usr/doc/rpm*/GROUPS
Requires:     prerequisite packages
Source0:      %{name}-%{version}.tar.gz
Source1:      additional sources, e.g. an init file
Patch0:       %name0.patch
BuildRoot:    %{_tmppath}/%{name}-%{version}-build

%description
multi-line verbose description

%prep
%setup -q
%patch0

%build
./configure \
	--prefix=%_prefix \
	--libdir=%_libdir \
	--mandir=%_mandir \
	--sysconfdir=%_sysconfdir \
	--with-someoption=somevalue \
	--disable-otheroption
make

%install
rm -rf %buildroot

%makeinstall
# add files, which are not part or the tarball
mkdir -p %buildroot/etc/init.d
install -m755 %{SOURCE1} %buildroot/etc/init.d/%{name}
ln -s /etc/init.d/%{name} %buildroot/usr/sbin/rc%{name}

%clean
rm -rf %buildroot

%files
%defattr(644, root, root, 755)
%config(noreplace) %_sysconfdir/*.conf
%config /etc/init.d/clamd
%doc [A-Z]* docs/html docs/*.pdf
%doc %_mandir/*/*
%_bindir/*
%_includedir/*
%_libdir/*
%_sbindir/*

%pre
# do some checks

%post
# sample entry if the package is a server
chkconfig --add %{name}
if [ -f /var/lock/subsys/%{name} ]; then
  /etc/init.d/%{name} restart >&2
else
  echo Run /etc/init.d/%{name} start to start %{name}
fi

%preun
# sample entry if the package is a server
if [ -f /var/lock/subsys/%{name} ]; then
  /etc/init.d/%{name} stop
fi
chkconfig --del %{name}

%postun
# usually nothing to do

%changelog
* Tue Feb 24 2004 - me@mydomain.de
- New version: 2.0
* Sat Jan 01 2000 - he@hisdomain.net
- Version 1.1 Fixed the Y2k problem
* Thu Aug 24 1995 - someone@evilempire.com
- Version 1.0

Notes:

  • Some programmers put a URL instead of a file name into the Source or Patch line of the showHeader. While rpmbuild is only interested in a file name, a human reader might want to know, where to download this file. If the URL reads something like http://server/downloads/filename?version=4.2&format=txt rpmbuild will take “txt” as the filename, which is wrong. The following trick however will help: http://server/downloads/filename?version=4.2&format=txt#/filename as you want rpmbuild to see it.
  • %_libdir and other directories are predefined macros. Hopefully the configure command will allow to set these options. Usually a lot of intelligence is put into the configure script to make it flexible and it's a pity that we feed it with fixed values. However in a fixed environment, we need the right values rather than flexible ones.
  • The %attr or %defattr directive tells rpm about the ownership and permissions of the files. Without this directive, the files would end up belonging to the user who built the rpm.
  • The %config handles this file in order to survive a de/reinstallation.

KRPMBuilder: The GUI way

KRPMBuilder is a GUI to create a SPEC file and initiate the build process. Think of it as if a spec template like the one above and checkinstall was built into the tool and the input fields allow you to fill in the gaps.

Check

When you've finally built your binary RPM, use CheckInstall again to make sure, the %files section is correct. checkinstall --delspec=no rpm --install progname.ix86.rpm creates a new spec file. Compare the %files section of this new file with your handcrafted file. They should be almost the same. “Almost” because the new file will reflect that the rpm command modified the RPM database. These modifications however are handled by rpm on its own and should not go to your spec file.

YaST

Whether or not you should use YaST is even more debated than if you should use RPMs. Below you find some hints and comments.

YaST maintains its own XML database of package versions and installations on top of RPM. To make YaST aware of the package you installed, use yast -i MyPackage.rpm or add the directory, where your rpm resides to the YaST installation sorces with yast inst_source and then add or delete RPMs with yast sw_single.

Since yast -i doesn't check dependencies, you may want to use rpm --test --install package.rpm && yast -i package.rpm to stay on the safe side.

On Fri, 29 Apr 2005 10:05:46 GMT Nico Kadel-Garcia wrote in novell.support.suse.linux.professional

Christian Barmala wrote:

If the tarball contains a “.spec” file then you can do an rpmbuild -ta tarball.tar.gz.

You missed the vital word “valid” to describe the .spec file. A lot of tarballs contain a slightly out-of-date .spec file that needs to be updated to work correctly. Others contain a .spec. that the autoconf tools will set set up and allow a “make rpm” command to do the right thinngs: it varies with the support of the authors for RPM's.

Unfortunately, most such RPM's bear only a passing resemblance to the SuSE RPM's, since SuSE does a lot of strangeness to add YaST support, ignoring or in fact over-riding the author's own configuration tools. This means for example, that if you build up a new kernel RPM from the tarball using the “make rpm” command, you won't be able to recompile the new kernel SRPM successfully, at least under SuSE 9.1 and 9.2.

While tarballs are (hopefully) generic and can be adjusted to your needs, RPMs contain already all adjustments needed for a specific distribution. That's the main advantage and disadvantage of RPMs

True, especially for SuSE and it's modifications for YaST. In general, I recommend taking the tarball and integrating it into a new SRPM with a modification number, so that your package management is consistent and dependencies are reported correctly.

Unfortunately, because YOU (“YaST Online Update”) is too brain-dead to deal with merging multiple source repositories, you'll generally need to keep your packages up-to-date by hand, to avoid YOU updates replacing them, or configure YOU not to touch your custom packages.

Saturday, April 30, 2005 12:51 PM Nico Kadel-Garcia wrote:

Also, if you're building RPM's for packages provided by SuSE and using SRPM's or tarball .spec files from the original authors instead of working from those SuSE SRPM's, beware that SuSE integrates a lot of SuSE specific changes into their packages. These are for the YaST management system, especially SuSEconfig and the /etc/sysconfig configuration files. These layers of middleware package management are nominally controllable by YaST itself, but eliminating them by using a non-SuSE package can destabilize your software deployment. YaST isn't smart enough to know that you've thrown out the SuSE management tools and are using the author provided tools, such as SWAT for Samba, and YaST will try to edit both the typical software configurations, and its expected /etc/sysconfig files, and break systems in extremely unpredictable fashion. Examples of this behavior include grub, Apache, DHCP, and BIND.

Frankly, SuSE should put a bullet through YaST and spend their time on webmin. It would be cheaper in engineering time for SuSE users and SuSE package maintainers.

For another example of me being quoted with complaints about user interfaces, check The Luxury of Ignorance: An Open-Source Horror Story. YaST violates every single one of the rules I suggested for writing good open face interfaces, and almost every rule Eric Raymond suggested.

Links