April 1, 2011

kojid dead but subsys locked

I have been toying with Koji ( A build server by Fedora) and encountered an issue while starting the kojid daemon on one of the builder host.

[[email protected] ~]# /etc/init.d/kojid status
kojid dead but subsys locked

while the log kept showing

[INFO] koji.build: Starting up

Time for some debugging, I issue

[[email protected] ~]# /usr/sbin/kojid --verbose --fg

and see

koji.AuthError: Traceback (most recent call last):` koji.AuthError: Traceback (most recent call last):
File "/usr/share/koji-hub/kojixmlrpc.py", line 191, in _marshaled_dispatch
response = self._dispatch(method, params)
File "/usr/share/koji-hub/kojixmlrpc.py", line 253, in _dispatch
ret = func(*params,**opts)
File "/usr/share/koji-hub/kojihub.py", line 9005, in getID
host = Host()
File "/usr/share/koji-hub/kojihub.py", line 8811, in __init__
raise koji.AuthError, "No host specified"
AuthError: No host specified

Aha! “AuthError: No host specified” - kojid is complaining that no host was specified! But wait a minute, the “user” directive is correctly specified in the the kojid.conf file

Let’s check

[[email protected] ~]# awk /builder1$/ /etc/kojid/kojid.conf

Hmm… time to check the host registered in koji

[[email protected] ~]# su - kojiadmin
[[email protected] ~]$ koji list-hosts
Hostname                     Enb Rdy Load/Cap Arches           Last Update
builder1                   Y   N    0.0/2.0 x86_64      -

All good. Let’s check our host certificate

[[email protected] koji]# openssl x509 -in certs/builder1.crt -noout -text
Version: 3 (0x2)
Serial Number: 9 (0x9)
Signature Algorithm: md5WithRSAEncryption
Issuer: C=XX, ST=XX, L=XX, O=XX, CN=builder1.domain.local

Only obvious different is that I have use a FQDN as a CN in the certificate (good!), could it be that kojid isn’t expanding the hostname user to a FQDN?

Let’s remove the current builder and re-add it as a FQDN as I am not able to add “builder1.domain.local” since it already exists as a user in the users table of the koji database and this is creating a key violation constraint as follow:

[[email protected] ~]$ koji add-host builder1.domain.local x86_64
Fault: <Fault 1: 'pg.DatabaseError: error \'ERROR:  duplicate key violates unique constraint "users_name_key"\n\' in \'INSERT INTO users (id, name, usertype, status, krb_principal)\n        VALUES (9, \'builder1.domain.local\', 1, 0, NULL)\''>

Aha, time for some db cleaning. After cleaning the koji postgres database, I re-issue the “koji add-host” command

[[email protected] ~]$ koji add-host builder1.domain.local x86_64
builder1.domain.local added: id 1

[[email protected] koji]# /etc/init.d/kojid start
Starting kojid:

[[email protected] koji]# ps aux |grep kojid
root     17274  1.3  0.0 227868 12380 ?        S    23:40   0:00 /usr/bin/python /usr/sbin/kojid --force-lock --verbose

Yeah :-)! Problem fixed.