I decided to update to Debian 8. All of the instructions are laid out here. So let’s get to it.

Backups

From the above page:

The main things you’ll want to back up are the contents of /etc, /var/lib/dpkg, /var/lib/apt/extended_states and the output of dpkg –get-selections “*” (the quotes are important).

elatov@kerch:~$ dpkg --get-selections "*" > /tmp/dpkg-jessie.txt
elatov@kerch:~$ sudo tar cpvjf deb-jess.tar.bz2 /etc /var/lib/dpkg /var/lib/apt/extended_states /tmp/dpkg-jessie.txt
elatov@kerch:~$ls -lh deb-jess.tar.bz2 
-rw-r--r-- 1 root root 5.1M Jun 19 09:00 deb-jess.tar.bz2

Check the status of Packages

I was doing my upgrade remotely so I went ahead and started a screen session at this point. Then checking to make sure all the packages are okay:

elatov@kerch:~$sudo  dpkg --audit
elatov@kerch:~$

I also checked I didn’t have any packages on hold:

elatov@kerch:~$aptitude search "~ahold" 
elatov@kerch:~$dpkg --get-selections | grep 'hold$'

Modify /etc/apt/source.lists

Prior to the update here is how my file looked like:

elatov@kerch:~$cat /etc/apt/sources.list
#

# deb cdrom:[Debian GNU/Linux 7.2.0 _Wheezy_ - Official amd64 NETINST Binary-1 20131012-14:04]/ wheezy main

#deb cdrom:[Debian GNU/Linux 7.2.0 _Wheezy_ - Official amd64 NETINST Binary-1 20131012-14:04]/ wheezy main

deb http://ftp.us.debian.org/debian/ wheezy main
deb-src http://ftp.us.debian.org/debian/ wheezy main

deb http://security.debian.org/ wheezy/updates main
deb-src http://security.debian.org/ wheezy/updates main

# wheezy-updates, previously known as 'volatile'
deb http://ftp.us.debian.org/debian/ wheezy-updates main
deb-src http://ftp.us.debian.org/debian/ wheezy-updates main

# wheezy backports
deb http://ftp.debian.org/debian wheezy-backports main contrib non-free

# non-free ports
deb http://ftp.fr.debian.org/debian/ wheezy main contrib non-free

I just ended up using sed to replace wheezy with jessie:

elatov@kerch:~$sudo sed -i "s/wheezy/jessie/" /etc/apt/sources.list

I also had alienvault and puppetlabs sources:

elatov@kerch:~$ls -l /etc/apt/sources.list.d
total 12
-rw-r--r-- 1 root root 100 Nov  1  2014 alienvault.list
-rw-r--r-- 1 root root 385 Aug 25  2014 puppetlabs.list

They just had a couple of lines:

elatov@kerch:~$cat /etc/apt/sources.list.d/*.list
#file generated by puppet
# alienvault
deb http://ossec.alienvault.com/repos/apt/debian wheezy main
# Puppetlabs products
deb http://apt.puppetlabs.com precise main
deb-src http://apt.puppetlabs.com precise main

# Puppetlabs dependencies
deb http://apt.puppetlabs.com precise dependencies
deb-src http://apt.puppetlabs.com precise dependencies

# Puppetlabs devel (uncomment to activate)
# deb http://apt.puppetlabs.com precise devel
# deb-src http://apt.puppetlabs.com precise devel

So I ran the following to update them to jessie:

elatov@kerch:~$sudo sed -i "s/wheezy/jessie/" /etc/apt/sources.list.d/alienvault.list
elatov@kerch:~$sudo sed -i "s/precise/jessie/" /etc/apt/sources.list.d/puppetlabs.list

I also have zabbix installed on that system, but reading over this zabbix forum, it looks like there is no zabbix jessie repo yet. Hopefully it will come out soon.

I disabled the zabbix repo for now and the jessie repos actually have a 2.2 version:

elatov@kerch:~$apt-cache showpkg zabbix-server-mysql
Package: zabbix-server-mysql
Versions:
1:2.2.9-1+wheezy (/var/lib/dpkg/status)
 Description Language:
                 File: /var/lib/apt/lists/ftp.us.debian.org_debian_dists_jessie_main_binary-amd64_Packages
                  MD5: 47873543449b5d6688721f179370dff6
 Description Language: en
                 File: /var/lib/apt/lists/ftp.us.debian.org_debian_dists_jessie_main_i18n_Translation-en
                  MD5: 47873543449b5d6688721f179370dff6

1:2.2.7+dfsg-2 (/var/lib/apt/lists/ftp.us.debian.org_debian_dists_jessie_main_binary-amd64_Packages) (/var/lib/
apt/lists/ftp.fr.debian.org_debian_dists_jessie_main_binary-amd64_Packages)
 Description Language:
                 File: /var/lib/apt/lists/ftp.us.debian.org_debian_dists_jessie_main_binary-amd64_Packages
                  MD5: 47873543449b5d6688721f179370dff6
 Description Language: en
                 File: /var/lib/apt/lists/ftp.us.debian.org_debian_dists_jessie_main_i18n_Translation-en
                  MD5: 47873543449b5d6688721f179370dff6


Reverse Depends:
  zabbix-server-pgsql,zabbix-server-mysql
  zabbix-proxy-sqlite3,zabbix-server-mysql
  zabbix-proxy-pgsql,zabbix-server-mysql
  zabbix-proxy-mysql,zabbix-server-mysql

And I was on 2.2 so hopefully that will be okay :)

Upgrading the packages

To be extra safe we can also use script to record the session so we can review what we did later on. To do that just run the following:

elatov@kerch:~$script -t 2>~/upgrade-jessie_1.time -a ~/upgrade-jessie_1.script
Script started, file is /home/elatov/upgrade-jessie_1.script

Now let’s update the package list:

elatov@kerch:~$sudo apt-get update

And now let’s upgrade the packages:

elatov@kerch:~$sudo apt-get upgrade
Reading package lists... Done  
Building dependency tree
Reading state information... Done
The following packages have been kept back:
  acpid anacron apache2 apache2-mpm-prefork apache2-utils apache2.2-bin apache2.2-common apt apt-utils
...
...
365 upgraded, 0 newly installed, 0 to remove and 263 not upgraded.
Need to get 116 MB of archives.
After this operation, 47.7 MB of additional disk space will be used.
Do you want to continue [Y/n]?

Upgrade the System

Now we are ready to perform the important step. First let’s make sure we have enough space, we can run the following to confirm:

elatov@kerch:~$df -Ph
Filesystem              Size  Used Avail Use% Mounted on
/dev/mapper/kerch-root   15G  8.3G  5.8G  59% /
udev                     10M     0   10M   0% /dev
tmpfs                   202M  216K  202M   1% /run
tmpfs                   5.0M     0  5.0M   0% /run/lock
tmpfs                   403M     0  403M   0% /run/shm
/dev/sda1               228M   26M  191M  12% /boot

elatov@kerch:~$sudo apt-get -o APT::Get::Trivial-Only=true dist-upgrade
..
..
257 upgraded, 289 newly installed, 14 to remove and 5 not upgraded.
Need to get 352 MB of archives.
After this operation, 580 MB of additional disk space will be used.
E: Trivial Only specified but this is not a trivial operation.

The upgrade will remove the zabbix-server package:

elatov@kerch:~$sudo apt-get -o APT::Get::Trivial-Only=true dist-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be REMOVED:
  apache2-prefork-dev cpp-4.7 g++-4.7 gcc-4.7 gcc-4.7-base libaudit0 libgnutls-dev libgvc5 libkadm5srv-mit8
  libperl5.14 librtmp-dev libsnmp15 libstdc++6-4.7-dev zabbix-server-mysql

That’s okay, that won’t remove the db and the settings. We can just install the one from the debian jessie repo on top of the db and it should be okay. If the space is good let’s do the system upgrade:

elatov@kerch:~$sudo apt-get dist-upgrade

After that’s done clean up old packages

elatov@kerch:~$sudo apt-get autoremove

After that we can just reboot the system:

elatov@kerch:~$reboot

Config clean up

Before rebooting we can clean delete packages that have left over configs. To list those run the following:

dpkg -l | awk '/^rc/ { print $2 }'

Mine listed the zabbix-server, so I just saved the output to a file:

elatov@kerch:~$dpkg -l | awk '/^rc/ { print $2 }' > remove

Then I modified the file and removed any packages I wanted to keep and then ran the following to clean up packages I was okay with removing:

elatov@kerch:~$sudo apt-get purge $(cat remove)

Post Upgrade

I then reinstalled the zabbix-server-mysql package

elatov@kerch:~$sudo apt-get install zabbix-server-mysql

And I was able to load the web interface with all the graphs. I also used the iptables-persistent package to load iptables rules, but it looks like that has changed. So now I can just use the netfilter-persistent package and just keep my files under /etc/iptables/rules.v4. So I just enabled that via systemd, but upon starting the service it failed with the following:

Jun 19 13:57:56 kerch systemd-modules-load[19081]: Failed to insert 'ipmi_si': No such device
Jun 19 13:57:56 kerch kernel: ipmi_si: Unable to find any System Interface(s)
Jun 19 13:57:56 kerch puppet-agent[18602]: Could not start Service[netfilter-persistent]: Execution of '/bin/systemctl start
Jun 19 13:57:56 kerch puppet-agent[18602]: (/Stage[main]/Iptables::Service/Service[netfilter-persistent]/ensure) change from
Jun 19 13:57:56 kerch systemd[1]: systemd-modules-load.service: main process exited, code=exited, status=1/FAILURE
Jun 19 13:57:56 kerch systemd[1]: Failed to start Load Kernel Modules.
Jun 19 13:57:56 kerch systemd[1]: Dependency failed for netfilter persistent configuration.

I noticed that ipmitool was loading some old modules, so after removing the ipmitools package:

elatov@kerch:~$sudo apt-get purge ipmitools

I was able to start the netfilter-persistent service without issues:

elatov@kerch:~$sudo systemctl status netfilter-persistent.service 
- netfilter-persistent.service - netfilter persistent configuration
   Loaded: loaded (/lib/systemd/system/netfilter-persistent.service; enabled)
   Active: active (exited) since Fri 2015-06-19 14:09:09 MDT; 3s ago
  Process: 20461 ExecStart=/usr/sbin/netfilter-persistent start (code=exited, status=0/SUCCESS)
 Main PID: 20461 (code=exited, status=0/SUCCESS)

Jun 19 14:09:09 kerch netfilter-persistent[20461]: run-parts: executing /usr/share/netfilter-persistent/plugins.d/15...start
Jun 19 14:09:09 kerch netfilter-persistent[20461]: run-parts: executing /usr/share/netfilter-persistent/plugins.d/25...start
Jun 19 14:09:09 kerch netfilter-persistent[20461]: Warning: skipping IPv6 (no rules to load)
Hint: Some lines were ellipsized, use -l to show in full.

Also instead of compiling passenger I installed and enabled it for apache from the jessie repos:

elatov@kerch:~$sudo apt-get install libapache2-mod-passenger
elatov@kerch:~$sudo a2enmod passenger

After that my snorby rails app kept failing to load, and that’s because now the default ruby for debian 8 is 2.1 while snorby only runs on 1.9, luckily I had both versions installed. So I ended up with the following apache-passenger config:

Alias /snorby /usr/local/snorby/public
<Location /snorby>
    PassengerBaseURI /snorby
    PassengerAppRoot /usr/local/snorby
    PassengerRuby /usr/bin/ruby1.9.1
</Location>

<Directory /usr/local/snorby/public>
  #Options Indexes FollowSymLinks MultiViews
  Options -MultiViews
  AllowOverride all
  #Require all granted
  Order allow,deny
  allow from all
</Directory>

ErrorLog /var/log/apache2/snorby_error.log
CustomLog /var/log/apache2/snorby_access.log combined

After that I was able to load the snorby rails app but I was unable to start the delayed_job:

elatov@kerch:$cd /usr/local/snorby
elatov@kerch:/usr/local/snorby$rails c production
Loading production environment (Rails 3.1.12)
irb(main):001:0> Snorby::Worker.start
/var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/spec_set.rb:92:in `block in materialize': Could not find rake-0.9.2 in any of the sources (Bundler::GemNotFound)
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/spec_set.rb:85:in `map!'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/spec_set.rb:85:in `materialize'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/definition.rb:140:in `specs'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/definition.rb:185:in `specs_for'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/definition.rb:174:in `requested_specs'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/environment.rb:18:in `requested_specs'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/runtime.rb:13:in `setup'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler.rb:127:in `setup'
	from /var/lib/gems/1.9.1/gems/bundler-1.10.4/lib/bundler/setup.rb:18:in `<top (required)>'
	from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
=> ""
irb(main):002:0>

And that’s because the default ruby link is pointing to 2.1:

elatov@kerch:~$ls -l /usr/bin/ruby*
lrwxrwxrwx 1 root root    9 Jun 20 00:36 /usr/bin/ruby -> ruby2.1
-rwxr-xr-x 1 root root 6336 Apr 27 15:59 /usr/bin/ruby1.9.1
-rwxr-xr-x 1 root root 6168 Apr 27 18:41 /usr/bin/ruby2.1

After modifying the link to point to 1.9:

elatov@kerch:~$ls -l /usr/bin/ruby*
lrwxrwxrwx 1 root root    9 Jun 20 00:36 /usr/bin/ruby -> ruby1.9.1
-rwxr-xr-x 1 root root 6336 Apr 27 15:59 /usr/bin/ruby1.9.1
-rwxr-xr-x 1 root root 6168 Apr 27 18:41 /usr/bin/ruby2.1

The job started:

elatov@kerch:/usr/local/snorby$rails c production
Loading production environment (Rails 3.1.12)
irb(main):001:0> Snorby::Worker.start
=> ""
irb(main):002:0> Snorby::Worker.running?
=> true
irb(main):003:0>