This is an old revision of the document!


OAR's Debian repositories setup

For a list of the existing management tools for Debian packages, see:

Til August 2015, reprepro was used to manage OAR's Debian packages (thx plb). But:

  • it was difficult to use
  • it was difficult to maintain
  • it did not support multiple versions of a package in a distribution
  • snapshot support was limited.

We switched to Aplty.

Installing aptly

Repositories setup

OAR's Debian repos are managed with the following script:

https://github.com/oar-team/oar-aptly/blob/master/oar-aptly.sh

Which performs the following tasks:

  • manage-debian-mirrors: create/update mirrors of the official Debian repositories + snaptshots
  • create-testing-repos: create the testing local repositories for the target official Debian distributions
  • handle-incoming: Handle incoming packages (dput → inoticoming)

The handle-incomming task is run by inoticoming, setup as a service started by crontab:

ftpmaster@lig-oar:~$ crontab -l
@reboot inoticoming --logfile $HOME/logs/inoticoming.log $HOME/incoming/ --chdir $HOME/incoming/ --stdout-to-log --stderr-to-log --suffix .changes $HOME/bin/oar-aptly.sh handle-incoming {} \;
f

This allows us to place packages in a local repo for testing/local distrubtions before considering uploading them to the official Debian repositories:

  • sid → sid_beta
  • jessie-backports → jessie-backports_beta
  • wheezy-backports → whezzy-backports_beta

Also we eventually keep mirrors or snapshots repos of the Debian official repositories, with OAR's packages inside only thanks to filters.

Issues to follow

Some other notes about the use of aptly

These are examples of aptly commands, if manual actions are required.

# Create a mirror for Debian sid
$ aptly mirror create -architectures=amd64 -with-sources=true -dep-follow-all-variants=true -filter='Name (oar) | $Source (oar)' -filter-with-deps=false sid http://ftp.fr.debian.org/debian sid
$ aptly mirror update sid
 
# Create a snaptshot for the current version of oar in sid
$ aptly aptly snapshot create sid_oar-2.5.4-2 from mirror sid
$ aptly publish snapshot -distribution="sid_oar-2.5.4-2" sid_oar-2.5.4-2
 
# Create a local repository for testing oar packages
$ aptly repo create -architectures=amd64 sid_testing
$ aptly repo import sid sid_testing '$Source (oar)'
$ aptly repo include -repo=sid_testing $PWD/build-area/oar_2.5.5~rc1-1_amd64.changes
$ aptly publish repo -distribution="sid_testing" sid_testing

Here we mirror Vincent's repo for the bpo8 packages, waiting for them to appear in Debian's official backports (https://ftp-master.debian.org/backports-new.html).

ftpmaster@lig-oar:~$ aptly mirror create -architectures=amd64 -with-sources=true  -dep-follow-source=true -dep-follow-all-variants=true -filter-with-deps=false -filter='$Source (oar)| Name (oar)' vdanjean-jessie https://people.debian.org/~vdanjean/debian jessie main
Downloading https://people.debian.org/~vdanjean/debian/dists/jessie/InRelease...
gpgv: Signature made Thu 25 Feb 2016 11:22:17 AM CET using RSA key ID 7EC7B751
[...]
Mirror [vdanjean-jessie]: https://people.debian.org/~vdanjean/debian/ jessie [src] successfully added.
You can run 'aptly mirror update vdanjean-jessie' to download repository contents.
 
 
ftpmaster@lig-oar:~$ aptly mirror update vdanjean-jessie
Downloading https://people.debian.org/~vdanjean/debian/dists/jessie/InRelease...
gpgv: Signature made Thu 25 Feb 2016 11:22:17 AM CET using RSA key ID 7EC7B751
[...]
Downloading & parsing package files...
Downloading https://people.debian.org/~vdanjean/debian/dists/jessie/main/binary-amd64/Packages.bz2...
Downloading https://people.debian.org/~vdanjean/debian/dists/jessie/main/binary-amd64/Packages.gz...
Downloading https://people.debian.org/~vdanjean/debian/dists/jessie/main/source/Sources.bz2...
Downloading https://people.debian.org/~vdanjean/debian/dists/jessie/main/source/Sources.gz...
Applying filter...
Packages filtered: 98 -> 13.
Building download queue...
Download queue: 15 items (7.73 MiB)
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-server-mysql_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-common_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-doc_2.5.6-2~bpo8+1_all.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-user-mysql_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-web-status_2.5.6-2~bpo8+1_all.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-server_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-node_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-user_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-server-pgsql_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/liboar-perl_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-user-pgsql_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar-restful-api_2.5.6-2~bpo8+1_amd64.deb...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar_2.5.6-2~bpo8+1.debian.tar.xz...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar_2.5.6-2~bpo8+1.dsc...
Downloading https://people.debian.org/~vdanjean/debian/pool/main/o/oar/oar_2.5.6.orig.tar.gz...
 
Mirror `vdanjean-jessie` has been successfully updated.
 
 
ftpmaster@lig-oar:~$ aptly snapshot create jessie-backports_waiting from mirror vdanjean-jessie
 
Snapshot jessie-backports_waiting successfully created.
You can run 'aptly publish snapshot jessie-backports_waiting' to publish snapshot as Debian repository.
 
 
ftpmaster@lig-oar:~$ aptly publish -batch=false -gpg-key=D90D0568 -distribution=jessie-backports_waiting snapshot jessie-backports_waiting
Loading packages...
Generating metadata files and linking package files...
Finalizing metadata files...
Signing file 'Release' with gpg, please enter your passphrase when prompted:
Clearsigning file 'Release' with gpg, please enter your passphrase when prompted:
 
Snapshot jessie-backports_waiting has been successfully published.
Please setup your webserver to serve directory '/home/ftpmaster/.aptly/public' with autoindexing.
Now you can add following line to apt sources:
  deb http://your-server/ jessie-backports_waiting main
  deb-src http://your-server/ jessie-backports_waiting main
Don't forget to add your GPG key to apt with apt-key.
 
You can also use `aptly serve` to publish your repositories over HTTP quickly.

Removing a version of the OAR packages:

ftpmaster@lig-oar:~$ aptly repo remove stretch-backports_beta "Version (=2.5.8~rc4-1~bpo9+1)"
Loading packages...
[-] oar-user_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-doc_2.5.8~rc4-1~bpo9+1_all removed
[-] oar-server-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-restful-api_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-web-status_2.5.8~rc4-1~bpo9+1_all removed
[-] oar-server-pgsql_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-node_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] liboar-perl_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-common_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-user-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-user-pgsql_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-user-mysql_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-common-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-restful-api-dbgsym_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-server_2.5.8~rc4-1~bpo9+1_amd64 removed
[-] oar-server-mysql_2.5.8~rc4-1~bpo9+1_amd64 removed

And then publish the modified repo:

$ aptly publish -force-overwrite=true update jessie-backports_beta
WARNING: force overwrite mode enabled, aptly might corrupt other published repositories sharing the same package pool.
 
Loading packages...
Generating metadata files and linking package files...
Finalizing metadata files...
Signing file 'Release' with gpg, please enter your passphrase when prompted:
Clearsigning file 'Release' with gpg, please enter your passphrase when prompted:
Cleaning up prefix "." components main...
 
Publish for local repo ./jessie-backports_beta [amd64] publishes {main: [jessie-backports_beta]} has been successfully updated.
wiki/aptly.1499201310.txt.gz · Last modified: 2017/07/04 22:48 by neyron
Recent changes RSS feed GNU Free Documentation License 1.3 Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki