Git-Backed Icinga

How to Add comments

I want to make git commits to configure my icinga2 monitor: commit to a “devel” tree for config validation, commit to a “prod” tree to validate-and-activate a config.

[wp_graphviz simple=”yes”] devel -> { devel prod } -> check -> { prod devel “live\nicinga” } [/wp_graphviz]


Felipe Contreras has slowly converted me to using git.  I fight with git, but it seems the way forward based on critical mass of user-base, and only takes 7 more steps to do each thing, so my fingers get used to the additional commands.


I’m looking at a few tasks to get this done:

  1. init the repository
  2. load the example config shipped with icinga2 to pre-seed the “master” branch
  3. create “prod” (from “master”) and “devel” branches
  4. pre-commit hooks (both branches): run a “/etc/init.d/icinga2 configcheck” on the content
  5. post-commit hook on “prod”: export to running directory and “/etc/init.d/icinga2 configcheck”

Let’s see how this goes…

I used gitolite to get this done simply to accelerate the act of setting up a git repository with a restricted command set. I could have taken the longer way, but this works, and it’s not a huge waste of disk space, or a massive dependency on bogus chewy scripting languages that were not dependencies before (i.e. why does a linux system NEED perl or python? It’s a kernel and a run control, let’s keep the focus on what it is, but I digress). Let’s get to it.

Initialize the Repository

(on icinga server)

yum install gitolite
usermod -s /bin/bash icinga
install -o icinga -g icinga -m 0660 ~/.ssh/ ~icinga/
su - icinga -c '/usr/bin/gl-setup -q'
rm ~icinga/

Create the Icinga Repository

(on icinga server) (first check to ensure ssh key is available)

ssh-add -l
git clone ssh://icinga@localhost/gitolite-admin
cd gitolite-admin
echo >> conf/gitolite.conf 
echo 'repo	icinga' >> conf/gitolite.conf 
echo '	RW+ = allan' >> conf/gitolite.conf
git commit -a -m 'add icinga project' && git push

Seed “prod” Branch with Example Config; Merge to Devel Branch

(on icinga server) (first check to ensure ssh key is available)

ssh-add -l
git clone -b master ssh://icinga@localhost/icinga
cp -r /etc/icinga2/* icinga/
cd icinga
git add -A
git commit -a -m 'initial config' && git push origin master
git branch -m master prod && git push origin prod
git branch devel && git push origin devel

Pre-commit ConfigCheck Hooks

edit ~icinga/repositories/icinga.git/hooks/pre-receive thusly:
(thanks to Jani Hartikainen )

declare -i ACCUM

uniq | while read oldrev newrev refname
    TEMPREVDIR=$(mktemp -d -t ${newrev}.XXXXX)

    git archive ${newrev} | tar -x -C ${TEMPREVDIR}
    /usr/sbin/icinga2 -c ${TEMPREVDIR}/icinga2.conf -C | sed -e "s@${TEMPREVDIR}/@@g"

    rm -fr ${TEMPREVDIR}

    # if it is a production commit, activate it on successful check
    # running as the icing user, we can signup our own process
    if test ${refname} = "refs/heads/prod" -a ${RES} = 0
        git archive ${newrev} | tar -x -C /etc/icinga2/ && killall -HUP icinga2
        echo Revision ${newrev} now active configuration

exit ${ACCUM}

Leave a Reply

WP Theme & Icons by N.Design Studio
Entries RSS Comments RSS Log in