After the challenges with the iCalDAV server in OSX, I gave up on getting the Addressbook server up-and-running. Somehow, the clients couldn't wouldn't connect. No matter what configuration parameters I tried.
This week I tried to get it up-and-running again. Mainly because I can't concentrate for longer than two hours for my Cisco exams next week. Sometimes you need to clear your head.
For some reason I found the solution within the hour. No idea what the original problem was exactly.
This is what happened;
Clients couldn't connect. Mostly because they were unable to find the server or they just stopped working. Even with SSL turned off, nothing happened. Putting the logging level on debugging, I found that the SSL certificate was not correct. There were mentions that the name in the certificate was wrong. But the strange thing is that I should have been able to get it working with SSL turned off.....
The error regarding the certificate was that the common name was set to the correct server name, but that a Subject Alternative Name (SAN) was called something different. For some reason, the server/client communication bogged at that. Normally you get a warning that there might be something wrong with the certificate which you can confirm (or deny).
So I created a new certificate with xCA, and added the proper SAN's to the certificate and everything worked.