OpenPKG Advent Calendar 2006

Saturday 2006-12-02: Backstage
Version Tracking in OpenPKG-CURRENT

Once an upstream vendor releases a new version of its software, usually just within 12 hours on average OpenPKG-CURRENT provides a corresponding updated OpenPKG package. How is this possible all the time and since so many years? Especially, as OpenPKG doesn't have hundreds of engineers like FreeBSD or even thousands of engineers like GNU/Debian?

Tracking Snippets

Every OpenPKG package specification contains a vcheck(1) based %track section which specifies how we can automatically determine whether a new upstream vendor version is released. For instance the package specification uuid.spec of the OpenPKG uuid package contains:

      [...]
      ##  uuid.spec — OpenPKG RPM Package Specification
      [...]
      Version:      1.5.1
      [...]
      %track
      prog uuid = { 
          version   = %{version}
          url       = ftp://ftp.ossp.org/pkg/lib/uuid/
          regex     = uuid-(\d+\.\d+\.\d+)\.tar\.gz
      }
      [...]

Bi-Daily Tracking

Twice per day all such %track sections are evaluated. For this the contained macros are expanded by OpenPKG RPM. Here the %{version} construct is replaced with the value 1.5.1 as implicitly defined by the Version header. Then all such tracking configuration snippets of all OpenPKG CURRENT packages are concatenated together and passed through vcheck(1).

The vcheck(1) program is a nifty Perl utility which fetches the content of all configured URLs (the ftp://ftp.ossp.org/pkg/lib/uuid/ in the above example) and parses the content against the configured regular expressions (the uuid-(\d+\.\d+\.\d+)\.tar\.gz in the above example).

As the regular expression usually matches multiple times, for every match the content which matched against the sub-expression in parenthesis (here \d+\.\d+\.\d+) is remembered. Now the remembered matches are sorted according to a special version comparison strategy and the newest one becomes the latest determined vendor version. This now in turn is compared against the configured version (here 1.5.1). If it is the same, the package is still up-to-date. If it is not the same, the package is considered outdated.

Web User Interface

All the results determined by vcheck(1) are stored into a database. Twice per day OpenPKG GmbH employees and OpenPKG Foundation e.V. members are going to the Web user interface of this database at http://www.openpkg.org/development/tracker/ and view a compact summary information about the results of the latest (or any previous) vcheck(1) runs. For every out-dated package, the OpenPKG engineer updates the OpenPKG-CURRENT package to the new upstream vendor version. For every tracking error the OpenPKG engineer tries to adjust the %track section — usually by configured new URLs (if the vendor changed it), adjusting the regular expressions (in case the distribution files changed), etc.

This way, once the upstream vendor releases a new version if its software, OpenPKG-CURRENT easily can be upgraded within a very short time.