SysAdmin'ish Blog

OpenSSL self-signed certificates & Thunderbird

Researching how to use OpenSSL can be quite a PITA. Thus, I am especially delighted when I find instructions that are easier and shorter than what I had previously known. One such delight was provided by diegows accompanied by the wonderful remark "you can do this in one command". So, here are the shortest instructions I could come up with to create and install self-signed certificates in Thunderbird to be used for S/MIME:

Say, we want a certificate for our address someone@example.net:


Generate the self-signed certificate and the encrypted key file valid for 3 years (1095 days):

openssl req -x509 -newkey rsa:2048 -keyout $EMAIL.key -out $EMAIL.crt -days 1095

You get 2 files, your secret private key $EMAIL.key and your public certificate in $EMAIL.crt. For most applications that's it, however, Thunderbird requires the key to be in PKCS #12:

openssl pkcs12 -export -in $EMAIL.crt -inkey $EMAIL.key -out $EMAIL.p12

To be able to import the $EMAIL.p12 in Thunderbird, we first need to import the $EMAIL.crt file as a "Certificate Authority". Doing so we declare that we trust our self-signed certificate.

In order to do so, open the Thunderbird settings/preferences menu and navigate to Advanced → Certificates → [View Certificates]. Eventually change to the "Authorities" tab, click [Import] and import $EMAIL.crt. Thunderbird will ask you to specify the intended purpose. Select "Trust this CA to identify email users."

Then change to the "Your Certificates" tab and import $EMAIL.p12. In case you ever think like double checking whether your entry is still listed among the "Authorities" … it isn't! ;-) Once your certificate is installed and fully trusted Thunderbird removes the entry.

When you start writing the first signed message, Thunderbird will prompt you to finish the setup. I.e., it will open the account settings where you can associate your accounts with your certificates (in the "Security" menu).

Use a master password

On the mozillaZine page Installing an SMIME certificate it says "you must first set a master password". "Must" here means "if you don't do it chances are you are making a big mistake". All your passwords and certificates are utterly unprotected if you do not set a master password. So, while encryption does work without a master password, please make sure your Thunderbird is safe.

Trusting other people's self-signed certificate

While we are at it, here is how to import and trust other people's self-signed certificates

  • Have them send you a signed message.
  • Click the security envelope or go to View → View Security Info.
  • Click [View Signature Certificate].
  • Ideally, double check the "SHA1 Fingerprint" e.g. via phone.
  • Change to the [Details] tab, click [Export…], save the certificate as e.g. friend.pem.
  • Import friend.pem file as "Certificate Authority" in settings/preferences → Advanced → Certificates → [View Certificates] (see above)


FAM-ily's Windows XP - R.I.P.

FAM-ily's Windows XP boot logo

At my department, we used Microsoft Windows XP for about 10 years. We put a lot of effort into setting it up, speeding it up, making it more secure etc.. After all was done, I wanted to replace the MS boot logo1). That turned out to be quite troublesome. I remember that within a reasonable time frame, anyway stretched by the usual factor, I only managed to convert 2 images that were shown correctly. The resolution was extremely low (even back then) and only 16 specific colors were supported (IIRC). Anyway, I thought it was worth it and the result was a converted/artistified/mutilated photo of one of my old Amaryllis photos.

In April 2014, Windows XP should be more or less history. Thus, we are already taking preps to replace the remaining old PCs. The photo above shows one of them at what will be one of the last times this boot screen will be shown.

BTW, user feedback was quite mixed. Most, I guess, didn't care. Some tolerated it. Some liked it. And some didn't like it. One person actually asked me to replace it with the default boot screen. She said, she didn't like the colors.

Missing user in access.log with PHP-FPM

Since we switched to PHP5-FPM and FastCGI all protected PHP scripts showed an empty "Remote user" (Apache logged a dash) even though Apache did a basic authentication.

Installations of Apache often come with default LogFormat settings such as

LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined

where %u stands for "Remote user" according to the Custom Log Formats.

In order to fix the logging of usernames replace %u with %<u. The < is a modifier:

%<u can be used to record the original authenticated user on a request that is internally redirected to an unauthenticated resource

— quoted from mod_log_config - Apache HTTP Server (2013-10-18)

Short pasSWords and john

Surprisingly short passwords that John The Ripper doesn't find quickly

John the Ripper is a "password cracking software". It should not be mistaken for a safety tool, or software to evaluate password strength. While it definitely will find some weak passwords quickly it might fail to guess some (at least when using the default configuration as shown below). Incidentally, this is not john's fault. The strength of passwords depends on a lot of circumstances. We can't consider them all.

exAMple session

Occasionally, I use john to look for weak passwords. Fortunately, my users are well trained ;-) but passwords could get accidentally reset to the empty string or the login name or similar. To my mind, john is especially useful here as it can find these quickly.

After the last check, I was surprised that john didn't find a password that I happened to know: 7 characters long, no numbers, no special characters, 2 caps. I thought it was rather weak by modern standards.

Here is the log of a "debugging" session that IMHO illustrates the problem quite well. I am running john 1.7.8 from Debian Wheezy on a 64-bit Debian Squeeze on an old Intel Core i7.

First I am creating an example file passwords with login credentials:

$ echo "user0:$(openssl passwd -1 "user0"):1:1:New Example:" >passwords
$ echo "user1:$(openssl passwd -1 "newexample"):1:1:New Example:" >>passwords
$ echo "user2:$(openssl passwd -1 "newexamplE"):1:1:New Example:" >>passwords
$ echo "user3:$(openssl passwd -1 "new example"):1:1:New Example:" >>passwords
$ echo "user4:$(openssl passwd -1 "newexAMple"):1:1:New Example:" >>passwords
$ echo "user5:$(openssl passwd -1 "newexAmple"):1:1:New Example:" >>passwords
$ echo "user6:$(openssl passwd -1 ""):1:1:New Example:" >>passwords
$ echo "user7:$(openssl passwd -1 " "):1:1:New Example:" >>passwords
$ echo "user8:$(openssl passwd -1 "asdf"):1:1:New Example:" >>passwords
$ echo "user9:$(openssl passwd -1 "kdnwir"):1:1:New Example:" >>passwords

john's "single crack" mode uses what we have in the passwords file itself:

$ /usr/sbin/john --single passwords
Created directory: /home/schamane/.john
Loaded 10 password hashes with 10 different salts (FreeBSD MD5 [32/64 X2])
user0            (user0)
newexample       (user1)
newexamplE       (user2)
new example      (user3)
newexAmple       (user5)
guesses: 5  time: 0:00:00:04 100%  c/s: 15087  trying: examplenew1900 - enew1900
Use the "--show" option to display all of the cracked passwords reliably

So, the apparent ones are found. user4 with a password with 2 consecutive caps is not found. Neither are the empty password, the single space, nor the trivially short ones. That's quite a bummer. At least for me, because I thought that "single mode" at least also tries the empty password.

Now, let's try again with a very good dictionary ;-)

$ rm -r ~/.john
$ echo new >dict ; echo example >>dict
$ /usr/sbin/john --wordlist=dict --rules passwords
Created directory: /home/schamane/.john
Loaded 10 password hashes with 10 different salts (FreeBSD MD5 [32/64 X2])
guesses: 0  time: 0:00:00:00 100%  c/s: 1980  trying: Exampling

Nothing! So, let's make it easier!

$ rm -r ~/.john
$ echo "new example" >dict
$ /usr/sbin/john --wordlist=dict --rules passwords
Created directory: /home/schamane/.john
Loaded 10 password hashes with 10 different salts (FreeBSD MD5 [32/64 X2])
new example      (user3)
newexample       (user1)
guesses: 2  time: 0:00:00:00 100%  c/s: 126  trying: newexample
Use the "--show" option to display all of the cracked passwords reliably

Better, but still almost nothing, especially if we consider the fact that dictionaries generally do not feature word pairs.

The reason is a lack of rules. Many interesting rules in the Debian configuration file are set only for "single mode", not for "wordlist mode". They are not applied with --wordlist=... --rules. BTW, one can get a list of processed words with

$ /usr/sbin/john --wordlist=dict --rules --stdout | /usr/sbin/unique mangled.lst

Have a look at mangled.lst and check what your rules produce.

Using the defaults

Running john with defaults, i.e. all 3 modes: single, wordlist, incremental (brute-force).

$ rm -r ~/.john
$ timeout 600 /usr/sbin/john passwords
Created directory: /home/schamane/.john
Loaded 10 password hashes with 10 different salts (FreeBSD MD5 [32/64 X2])
user0            (user0)
newexample       (user1)
newexamplE       (user2)
new example      (user3)
newexAmple       (user5)
asdf             (user8)
guesses: 8  time: 0:00:10:00 (3)  c/s: 15932  trying: 48802209 - 48802263
Use the "--show" option to display all of the cracked passwords reliably
Session aborted

These 8 guesses are found within very short time, however most of them are just found because our test users used their name as passwords). And, 10 minutes are not enough to find a 6 letter all lower char random password.

That's understandable. However, I find it puzzling that simple word pairs like "catmouse" are not found. Neither are relatively simple passwords with a few caps.

So, I added some more example passwords:

$ echo "userA:$(openssl passwd -1 "catmouse"):1:1:New Example:" >>passwords
$ echo "userB:$(openssl passwd -1 "ExAmplE"):1:1:New Example:" >>passwords
$ echo "userC:$(openssl passwd -1 "exAMple"):1:1:New Example:" >>passwords
$ echo "userD:$(openssl passwd -1 "examplE"):1:1:New Example:" >>passwords
$ rm -r ~/.john
$ /usr/sbin/john passwords
Created directory: /home/schamane/.john
Loaded 14 password hashes with 14 different salts (FreeBSD MD5 [32/64 X2])
examplE          (userD)
new example      (user3)
newexAmple       (user5)
asdf             (user8)
guesses: 8  time: 0:00:00:54 85% (2)  c/s: 15872  trying: 6stephi - 6stevens
guesses: 9  time: 0:00:01:16 (3)  c/s: 15840  trying: shadone - shadon1
guesses: 9  time: 0:00:11:42 (3)  c/s: 15974  trying: mutly61 - mutly63

We see that after about 1 minute john was in incremental mode. Neither the simple word pair ("catmouse") nor the dictionary words with funny caps have been found so far. The rest would be a matter of time, CPU power and burned fossils. But we are not going to waste them, are we?

User feedback drain

How (not) to deal with user feedback

I save bookmarks of bug reports I make. Have others also observed that some companies tend to simply erase complete bug trackers, feedback sites, forums and such? What a clever approach! – Since I just stumbled over such a case again, here is my good advice for companies who launch new products:

  1. Make a great announcement and tell users how committed you are.
  2. Create a new site just for user feedback, critiques, suggestions, bug reports etc..
  3. I recommend to provide this site with its own subdomain and install a nice forum or issue tracker like Jira (there you are in great company). Or you could buy a business plan from getsatisfaction.com which allows domain aliasing. It looks great to have this for instance on support.mycompany.com!
  4. Employ a few people who answer the user feedback. Though, seriously, one person should be enough. And it makes users comfortable if they know who is in charge.
  5. Replies should be appreciative and engaging, of course. Tell people what is planned. Be critical, too. Ask for details. You know the deal.
  6. robots.txt: Very important! Make sure you disallow all "User-agent"s that do archiving such as archive.org. Search engines are OK, they help the marketing. But do not allow any archiving.
  7. Finally, after a few months redirect the whole site to your main product page, or, to an FAQ page.
  8. Wait a few weeks, and *bang*, all criticism is gone! No records of unresolved bugs, no records of your stupid mistakes, painful omissions and what not. Simple and effective.

Thanks to AVOS Systems, Inc., owner of delicious.com. From them I first learned about this clever approach when they redesigned delicious.com. They had put up help.delicious.com which was run by getsatisfaction.com, now it redirects to delicious.com/help which is the FAQ page. Apparently, getsatisfaction.com also works in their favor. Even my account there doesn't show my contributions anymore.

Thanks also to Microsoft and the Skype team. They provided another confirming case. The Skype Issue Reporting Guide (accessed 2013-06-05 at http://dev.skype.com/issue-reporting-guide, gone for good as of October 2013) still explains^Hed how to "make a good bug report" on https://jira.skype.com/ – and many people sure did! But jira.skype.com now redirects to www.skype.com. (Update 2014-06-03: Hostname jira.skype.com seems to be gone since end of May 2014.)


Munin 2.0: "Date::Manip::Backend" error

Munin as in Debian, currently version 2.0.6, features a noisy error, at least in my servers' Apache error.log:

munin-cgi-graph: Name "Date::Manip::Backend" used only once:
possible typo at /usr/lib/munin/cgi/munin-cgi-graph line 31.

This is due to a bug. It should have been dealt with already (DBTS #561970) but either not sufficiently or something else is going on.

A reasonable workaround is to add no warnings; before line 31 in /usr/lib/munin/cgi/munin-cgi-graph:

    no warnings;
    $Date::Manip::Backend = 'DM5';

Yet Another or fix DokuWiki blogging?

What requires less time? Writing a new blog generator or fixing your old? — I can tell you my answers were changing rather often within the last 12 hours :-/ Eventually, I ended up fixing my blogs (including this one here) running on DokuWiki and the blog plugin.

Since many years, I am happily using DokuWiki for documentation and to publish scripts and such. In 2007, I started to also use the blog plugin for blogging about technical stuff. I love the simplicity, and the plain text flatfile backend. However, the experience has always been rather troublesome. Plugin updates often broke the layout, had vulnerabilities, needed fixing, or just didn't work.

Naturally, I was constantly looking for alternatives (why abuse a wiki for blogging anyhow?). I was hoping to find a simple, static blog generator that I could run from the command line. Of course, there are plenty. Strangely enough, though, when looking closer only very few withstood my criteria. As time passed by I started to sketch out some ideas for writing my own blog generator: Yet Another Shell Blog Engine (YASHBE) sounded like a cute name :-)

Eventually, I created a comparison table. At first, doing so seemed to be a very good idea because my first thoughts were "Oh, look, the DokuWiki thing is not that bad after all" and "Well, my own code could do a lot if I wrote it." – To cut a long story short, reviewing alternatives, writing the comparison table, researching and fixing the problems with DokuWiki took more than 12 hours. If I had started to code YASHBE right away I guess I would have finished it by now, too.

Problems with DokuWiki

  • The blog index page was partly broken and looked horrible which is why I updated it manually for some time. Fortunately, thanks to work by Michael Hamann that was fixed in the "blog" plugin. {{blog>blog?7&nouser&permalink&noeditbutton}} provides reasonable results for me.
  • RSS feeds could only be called broken. Apparently, older versions of plugin:feed could be used for blog posts but I could'nt get this work with the current version.2) The documentation recommends to use DokuWiki's native RSS syndication, however, results were unacceptable.
  • Archive pages looked horrible, workarounds using plugin:pagelist didn't fit my bill.

Fixing RSS feeds

After reading the source code of feed.php I found that actually there is a way to exclude "media" (file uploads). It just wasn't documented. So,

[[this>feed.php?ns=blog&linkto=current&content=html&mode=recent&view=pages&num=7|Blog posts]]

was pretty close to what I expect from a blog feed: Pages only (no uploads), linking to the current version. Due to a bug num=7 is ignored, but the feed delivers at least 5 entries which should be sufficient.

Update of 2013-04-21: feed.php delivered less than 7 entries because the number of matching entries in meta/_dokuwiki.changes was lower than 7. See my comment on DW bug #2765.

Unfortunately, the titles of the feed items include the "edit summaries". So, if I fix a typo in a post entitled "Feed woes", and if I document this in the edit summary, the feed item will be e.g. "Feed woes - typo fixed". This is hard-coded in feed.php, line 227 in the "Adora Belle" version. I've added && ($_GET['ns']!='blog') at

if($conf['rss_show_summary'] && ($_GET['ns']!='blog') && !empty($ditem['sum'])) {
    $item->title .= ' - '.strip_tags($ditem['sum']);

This or a similar switch should do.

Update of 2013-04-21: Today, I found that the "blog" component of plugin:feed works quite OK and I doubt that it is going to be removed (as it says on its web page). So, one can use e.g. {{blogfeed>blog?7|Blog posts}} to get a feed of 7 items without the need to hack feed.php. However, the {{blogfeed...}} shows only abstracts, not full text entries.

Fixing plugin:feed

Update of 2013-04-21: The feeds produced by plugin:feed have a hard-coded favicon.ico pointing at lib/images/favicon.ico. Consider to add the file, hack plugins/feed/feed.php or add e.g.

RewriteRule ^lib/images/favicon.ico   /favicon.ico [L]

to .htaccess.

Fixing the look of archives

For SysAdmin'ish Blog Archive I use {{archive>blog?2013-*&nouser}} etc.. This generates an HTML table for each year. Since these are separate tables the column widths vary. It's just cosmetics but IMHO it really hurts. I could use {{archive>blog?*&nouser}} to generate 1 large table for all entries but I wanted the year headlines. The following CSS improves the look (on my template):

/* make tables full width */
table.pagelist { width: 100%; text-align: left; }
/* reserve some fixed width for the page titles */
table.pagelist td.page { width: 47%; }
/* make date and comment columns non-wrapping and non-expanding */
table.pagelist td.date, table.pagelist td.comments { white-space: nowrap; width: 1px; }


Added 2013-04-21

I wanted to add this as a reminder (to myself and whoever cares): I am normally much in favor of recycling and fixing the old instead of getting new stuff or re-inventing things. However, in this case it seems like if I had started to write my own program right away, without checking out alternatives, without making a list of what I actually disliked about the old solution, without researching bugs and whether they might be easy to fix, by now, I would have a working solution, too, plus a lot of hacking fun and the joy of learning new stuff (my plan was to use Markdown and Bootstrap).

RSS feeds: Blog posts (abstracts), Blog posts (full text), Comments

See also: SysAdmin'ish Blog Archive & Tag cloud

Back then the name "Microsoft" really wasn't what I wanted to read every morning.
Update of 2013-04-21: The "blog" component of the "feed" plugin still works, and it is indeed to be preferred.
blog/index.txt · Last modified: 2013-04-21 18:51 by andreas