Happy New Year
Happy new year everyone! I hope you could spend a nice time with your family and friends.
I'm very sorry, that the third part took me so long. The end of the year is always a tough time, but this year was especially busy. We had a lot work over at dajool and I needed to take some time off during the holidays to relax a bit.
In part 2 we looked into the details of the package generation. Now I'd like to give you an idea of how to deploy the packages and even how to automate the deployment.
The Easy Way With scp… For Testing
After building the deb package you will want to deploy the package. For a simple test, you can just scp the deb package to the server and directly install the package with
dpkg -i my_package.deb. However dpkg might complain about missing dependencies. Note that this won't happen, if you are able to install the package with
apt-get - like when you are installing from your own repository. Just type
apt-get install -f to install the missing dependencies.
Full Blown Deployment With reprepro
Manually copying deb file to multiple servers is not very convenient and no time saver at all. This is where reprepro comes into play. reprepro is a Debian package repository producer. This means reprepro just builds the folder and file structure needed for a repository, which might look like this:
│ ├── distributions
│ └── options
│ ├── checksums.db
│ ├── contents.cache.db
│ ├── packages.db
│ ├── references.db
│ ├── release.caches.db
│ └── version
│ ├── precise
│ ├── quantal
│ └── raring
│ └── public1.key
An installation and configuration howto can be found in this wiki article from debian.org. It basically tells you how the configs should look like and how the GPG-key can be generated. Generating a GPG-key is highly recommended if you are really going to use this setup. Otherwise apt-get won't be able to verify the origin and authenticity of the deb file and will complain. If you prefer Nginx over Apache you can use this very simple, uncomplete but sufficient sample config.
Your distribution file might look like this:
Label: neverland - precise
Architectures: i386 amd64 powerpc
Components: main non-free contrib
Description: Ubuntu repository for precise packages from neverland.
The 'Codename' should always be conform to the codename used in the
sources.list - otherwise this will be a nice source of errors for anyone else but you. If you'd like to add more distributions, just copy this section and make the distribution specific modifications.
After reprepro knows which distributions and architectures you'll want to host, you can add new deb packages to your repository like this:
reprepro -b <path_to_repo> includedeb <codename> <path_to_deb>
The package is immediately available and can be installed with
apt-get update && apt-get install my_django_app – at least if you added the repository to your
/etc/apt/sources.list. To add this repository to your
sources.list you'll just have to append the line
deb http://my_neverland_domain.com precise main.
That's it. Now everyone on the planet can
apt-get install packages from your server.
apt-get also supports https and basic authentication - just in case you want to host more private deb files. You'll just have to adjust your Apache or Nginx config.
RPM users can use
createrepo explained in this blog post.
Automated Deployment With SaltStack
If you are not familiar with SaltStack; It's an infrastructure management tool to orchestrate your servers (aka minions) from your master. Salt can do everything: You can execute commands on the minions, do things periodically, distribute config files or install packages. You should especially take a look at the grains. With the information SaltStack collects from the system - the grains - you are able to target minions from a certain group/role or location, with a specific OS, but also minions with e.g. four CPU or just the minions with a specific GPU if you are doing some CUDA number crunching.
A very nice installation instruction can be found in the excellent documentation.
Now you can do multiple things. Let's assume that you are using Jenkins as a build server. You trigger builds either from an incoming hook in your central mercurial repository or just let Jenkins check the mercurial repository periodically for changes. Git uses a very similar system. The deb files (artifacts called in Jenkins) then get pushed into reprepro by the salt master, which updates the package on your test server afterwards for QA.
It would also be possible to not use Jenkins at all and manage the builds from the salt master. But then you would have to write some more glue code. You could also use a specific branch in your mercurial repository or tags for production builds. The possibilities are endless.
Nice, isn't it? Since this topic ist so manifold and interesting I would like to start an experiment here. I've created a mailing list on librelist.com. What are your thoughts on deployment with packages? Do you have questions, annotations or do you just want to discuss a part of the setup? Just send an e-mail to firstname.lastname@example.org and you'll be subscribed to the list. Any e-mail send afterwards will be posted to the list.