Yum Remove duplicate packages

During mayor upgrades on several systems I came along the bad situation with duplicate packages..

This helped me solve the issue of having duplicate packages installed. I only needed to add another flag to have:

rpm -e --justdb --nodeps <package-old_version>

Below you will find the steps to semi automate this, but beware, Your millage may vary, make sure you check the output that are put into the files.

Bottom of output from:

yum --assumeno upgrade
Warning: RPMDB altered outside of yum.
** Found 247 pre-existing rpmdb problem(s), 'yum check' output follows:
#(truncated)
chkconfig-1.7.4-1.el7.x86_64 is a duplicate with chkconfig-1.7.2-1.el7.x86_64
1:gmp-6.0.0-15.el7.x86_64 is a duplicate with 1:gmp-6.0.0-12.el7_1.x86_64
gobject-introspection-1.50.0-1.el7.x86_64 is a duplicate with gobject-introspection-1.42.0-1.el7.x86_64
grep-2.20-3.el7.x86_64 is a duplicate with grep-2.20-2.el7.x86_64
gzip-1.5-9.el7.x86_64 is a duplicate with gzip-1.5-8.el7.x86_64
pcre-8.32-17.el7.x86_64 is a duplicate with pcre-8.32-15.el7_2.1.x86_64
4:perl-5.16.3-292.el7.x86_64 is a duplicate with 4:perl-5.16.3-291.el7.x86_64
4:perl-libs-5.16.3-292.el7.x86_64 is a duplicate with 4:perl-libs-5.16.3-291.el7.x86_64
4:perl-macros-5.16.3-292.el7.x86_64 is a duplicate with 4:perl-macros-5.16.3-291.el7.x86_64
shared-mime-info-1.8-3.el7.x86_64 is a duplicate with shared-mime-info-1.1-9.el7.x86_64
2:tar-1.26-32.el7.x86_64 is a duplicate with 2:tar-1.26-31.el7.x86_64

Put above package names (starting from packages listed) into:

vi yum-fix.txt

Suggested to run from yum, but doesn’t seem to help. maybe it does a bit..

yum-complete-transaction --cleanup-only
cat yum-fix.txt|grep "is a duplicate with" > yum-fix_dupe.txt
cat yum-fix.txt|grep -v "is a duplicate with" > yum-fix_manually.txt

Parse output of some packages that start with a number and colon:

cat yum-fix_dupe.txt |awk '{print $6}'|awk -F":" '{if ((length($1) != 1) && (length($1) != 2)) print $1}' > yum-fix_dupe2.txt
cat yum-fix_dupe.txt |awk '{print $6}'|awk -F":" '{if ((length($1) == 1) || (length($1) == 2)) print $2}' >> yum-fix_dupe2.txt

Remove old package version from rpm db:

for i in $(cat yum-fix_dupe2.txt);do rpm -e --justdb --nodeps $i;done

Remove last 2 columns from output, and put on same line:

cat yum-fix_dupe2.txt | awk -F"-[0-9]" '{print $1}' |tr '\n' ' ' > yum-fix_dupe2-reinstall.txt

Reinstall packages:

yum reinstall $(cat yum-fix_dupe2-reinstall.txt)

Don’t forget to go back and manually fix the non-duplicates above could have fixed them.. try yum upgrade to see if any more errors):

cat yum-fix_manually.txt
yum --assumeno upgrade

Remove all stopped containers

Remove all stopped containers

docker rm -v $(docker ps -a | grep "Exited" | awk "{print $1}")

Remove all stopped containers v2

docker rm $(docker ps -a -q)

How to fix SSH “Host key verification failed” error

If you’ve ever rebuilt a server that you have connected to in the past, chances are you’ve received an error when trying to ssh back into it for the first time since the rebuild. If you’re getting a screen that says “WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!”:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is b7:f5:48:4c:d0:1d:76:6a:50:4a:88:12:c7:80:f1:e5.
Please contact your system administrator. Add correct host key in /home/user/.ssh/known_hosts to get rid of this message.
Offending key in /home/user/.ssh/known_hosts:2 RSA host key for myhost has changed and you have requested strict checking.
Host key verification failed.

Since we just rebuilt our server, this error is expected as the it is sending a new RSA key different than the one stored in /home/user/.ssh/known_hosts and now your computer’s ssh system is saying, “Hold on, this server you’re connecting to via this IP is not giving me the same ID. It could be that you just rebuilt the server, or it could be a man-in-the-middle attack!”

The workaround is actually quite simple, we have to remove the stored RSA key so we can add safely the new one. To do so, take a look at the last integer of the line that says “Offending ECDSA key,” because this integer is actually the line number inside your known_hosts file that’s throwing the error.

/home/user/.ssh/known_hosts:2

Then type in the following command, changing the number 2 to whatever matches your own error.

sed -i '2d' ~/.ssh/known_hosts

The offending RSA key can be also be removed with:

ssh-keygen -f "/home/user/.ssh/known_hosts" -R <server ip>

Unattended security updates in Ubuntu

In order to have automatic and unattended security updates in Ubuntu, one needs to install the according package:

sudo aptitude install unattended-upgrades The file /etc/apt/apt.conf.d/10periodic needs to be created with the following content:

APT::Periodic::Enable "1";
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "1";
APT::Periodic::AutocleanInterval "5";
APT::Periodic::Unattended-Upgrade "1";
APT::Periodic::RandomSleep "1800";

Also, change the first few lines of /etc/apt/apt.conf.d/50unattended-upgrades as follows so that only security updates are considered:

Unattended-Upgrade::Allowed-Origins {
        "Ubuntu lucid-security";
        "Ubuntu lucid-updates";
};

Unattended-Upgrade::Package-Blacklist {
};

Unattended-Upgrade::Mail "root@localhost";

Unattended-Upgrade::Remove-Unused-Dependencies "false";
Unattended-Upgrade::Automatic-Reboot "false";

It is vital to redo these setting after a global upgrade to a new distro release.

If configured correctly the following command should produce this output:

$ apt-config shell UnattendedUpgradeInterval APT::Periodic::Unattended-Upgrade
UnattendedUpgradeInterval='1' 

Postfix IPv6 capable + RBL + BIND9 as DNSBL

Here we go again…

For using ipv6 dnsbl, we need postfix version => 2.6 as the author of postfix state in postfix-users list.

How ipv6 dnsbl keep AAAA record in their zone? this is how it done. for example we got ipv6: 2001:1af8:4400:a00d:1::1 (my webserver)

RBL query lookup would be like this:

$ dig aaaa 1.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.d.0.0.a.0.0.4.4.8.f.a.1.1.0.0.2.dnsbl.example.tld.
$ dig txt 1.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.d.0.0.a.0.0.4.4.8.f.a.1.1.0.0.2.dnsbl.example.tld.

So, we need configure our private BIND9 RBL like this: first create dnsbl.example.tld zone in /etc/named.conf

zone "dnsbl.example.tld" {
      type master;
      file "dnsbl.example.tld";
};

second, we have to create dnsbl.example.tld zone file.

$TTL 86400
@       IN      SOA     ns1.dnsbl.example.tld.     hostmaster.dnsbl.example.tld. (
                        2011082200      ; serial number YYMMDDNN
                        28800           ; Refresh
                        7200            ; Retry
                        864000          ; Expire
                        86400           ; Min TTL
                        )

                NS      ns1.dnsbl.example.tld.
                NS      ns2.dnsbl.example.tld.

$ORIGIN dnsbl.example.tld.
blackhole       IN      A       127.0.0.2
                IN      AAAA    ::2
                IN      TXT     "Blocked by dnsbl.example.tld for SPAM Sources"

2.0.0.0.0.0.f.7.f.f.f.f.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0         IN      CNAME   blackhole
2.0.0.127                                                               IN      CNAME   blackhole
118.62.140.11                                                           IN      CNAME   blackhole
238.128.73.115                                                          IN      CNAME   blackhole
1.0.0.0.0.0.0.0.0.0.0.0.1.0.0.0.d.0.0.a.0.0.4.4.8.f.a.1.1.0.0.2         IN      CNAME   blackhole

why do i using CNAME instead of direct AAAA record? it’s just for efficiency, to avoid repetitions when adding ipv6 address on the blacklist. beside, postfix resolver can follow CNAME until found AAAA and TXT record.

IN postfix configuration, main.cf add this line:

smtpd_recipient_restrictions =
    ...
        reject_unauth_destination,
        reject_rbl_client dnsbl.example.tld,
    ...

now test all the things we’ve configured. (with my own ipv6 in the temporary list)

$ telnet mx.example.tld 25
220 mx.example.tld ESMTP Postfix (Ubuntu)
ehlo s1.example.tld
250-mx.example.tld
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from:<pietje@example.tld>
250 2.1.0 Ok
rcpt to:<jantje@example.tld>
554 5.7.1 Service unavailable; Client host [2001:1af8:4400:a00d:1::1] blocked using dnsbl.example.tld
Blocked by dnsbl.example.tld for SPAM Sources
quit
221 2.0.0 Bye

Connection to host lost.

In Postfix log, we will see rejection like this:

Aug 22 16:21:14 mx postfix/smtpd[25932]: NOQUEUE: reject: RCPT from s1[2001:1af8:4400:a00d:1::1]: 554 5.7.1 Service unavailable; Client host [2001:1af8:4400:a00d:1::1] blocked using dnsbl.example.tld; from=<pietje@example.tld> to=<jantje@example.tld> proto=ESMTP helo=<s1.example.tld>

that’s all

Migrate IMAP accounts from server A to server B

There is a easy to use tool that facilitates the quick’n dirty move of the content of one IMAP account to a new server. Its name is imapsync. In Ubuntu, you may install it as this:

aptitude install imapsync

Now, set two files for the old and the new account in your home, containing the respective passwords of the two accounts. They are called passfile_1 and passfile_2 from now on.

Now run imapsync with the following options:

imapsync --host1 {old_imap_host} --user1 {old_imap_user} --authmech1 LOGIN \ --passfile1 passfile_1 --port1 993 --ssl1 \
--host2 {new_imap_host} --user2 {new_imap_user} --authmech2 LOGIN \ --passfile2 passfile_2 --port2 993 --ssl2

You just need to adjust the ports (143 vs. ssl) and the authentication mechanisms to your needs and you’re set.

To show how this works, here’s an example from a personal migration:

Turned ON syncinternaldates, will set the internal dates (arrival dates) on host2 same as host1.
TimeZone:[europe/amsterdam]
Will try to use LOGIN authentication on host1
Will try to use LOGIN authentication on host2
From imap server [servera] port [993] user [usera]
To   imap server [serverb] port [993] user [userb]
Banner: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE AUTH=PLAIN ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2008 Double Precision, Inc.  See COPYING for distribution information.
Host servera says it has NO CAPABILITY for AUTHENTICATE LOGIN
Success login on [servera] with user [usera] auth [LOGIN]
Banner: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN] Dovecot ready.
Host serverb says it has NO CAPABILITY for AUTHENTICATE LOGIN
Success login on [serverb] with user [userb] auth [LOGIN]
host1: state Authenticated
host2: state Authenticated
From separator and prefix: [.][INBOX.]
To   separator and prefix: [.][INBOX.]
++++ Calculating sizes ++++
From Folder [INBOX]                             Size:   4122535 Messages:   252
From Folder [INBOX.Belangrijk]                  Size:  13395371 Messages:   127
flags from: [\Seen]["16-Jul-2011 09:45:30 +0200"]
Copied msg id [2053] to folder INBOX.Mailinglijsten.Debian.Bugs msg id [2053]
+ NO msg #2054 [9+vsr6Mzv58wM/L4Jt4Bvg:7095] in INBOX.Mailinglijsten.Debian.Bugs
+ Copying msg #2054:7095 to folder INBOX.Mailinglijsten.Debian.Bugs

Nagios commands via web-interface on Debian/Ubuntu

oops, I did it again…

When I reinstalled my monitoring VPS with Ubuntu 10.04 LTS and Nagios3, I got this annoying error message again:

Error: Could not stat() command file ‘/var/lib/nagios3/rw/nagios.cmd’!

In “/etc/nagios3/nagios.cfg” the “check_external_commands=1″ was already set. So there was something more required to make it run on Debian/Ubuntu…

Deep in my memory I know that there was a debian way to solve this user right related problem. This time I’ll write it down here, just as a reminder to myself, should I ever have to install it again.

/etc/init.d/nagios3 stop
dpkg-statoverride --update --add nagios www-data 2710 /var/lib/nagios3/rw
dpkg-statoverride --update --add nagios nagios 751 /var/lib/nagios3
/etc/init.d/nagios3 start

Converting kvm guests from lvm to qcow2, base images and snapshots

lvm based kvm guests are fast but you lose some flexibility, playing with kvm on my desktop I prefer to use file based images. Converting from lvm images to qcow2 isn’t hard but the documentation is sparse.

  1. use qemu-img to convert from an lvm to qcow2 format:
qemu-img convert -O qcow2 /dev/vg_name/lv_name/ /var/lib/libvirt/images/image_name.qcow2

If you want the image compressed add ‘-c’ right after the word convert.

  1. edit the xml for the image
virsh edit image_name

modify the disk stanza, adding a type to the driver line; on the source line change ‘dev’ to ‘file’ and modify the path:

driver name='qemu' type='qcow2'
source file='/var/lib/libvirt/images/image_name.qcow2'

Creating images from with a base image allows quick rollouts of many boxes based on an single install – for example I have a ‘golden image’ of Ubuntu, I can stop that VM and create 2 servers using the original VM disk as a base file and writing changes to different files.

qemu-img create -b original_image.qcow2 -f qcow2 clone_image01.qcow2
qemu-img create -b original_image.qcow2 -f qcow2 clone_image02.qcow2

Taking this further I can then snapshot both images so once I start making changes, rolling back to a point in time prior to the changes is very easy:

qemu-img snapshot -c snapshot_name vm_image_name.qcow2