Automating Certificate Signing Requests (CSR) generation for Dell iDRAC

I've been trying to get Puppet to automate the issuing of certificates to the iDRAC (Dell Remote Access Controller) for PowerEdge servers. One of the problems with Dell iDRACs is that on a certain batch of servers, the default key length was too short (1024 bit), rather than the minimum key length required by most Issuing Certificate Authorities (2048 bit).

Bumping the Certificate Signing Request (CSR) key length to 2048 bits requires the use of the racadm.exe utility: there is no way to change the CSR key length from the iDRAC UI, at least not in version 7.

Here are the steps you'll need to automate the generation of CSRs for all new servers that identify themselves as Dell.

Changing the CSR cryptographic key length size

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrKeySize 2048

Changing the CSR Common Name

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrCommonName "dellServer.myCloud.local"

Changing the CSR Organisation Name

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrOrganizationName "BURGER BURGER BURGER Pty Ltd"

Changing the CSR Organization Unit

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrOrganizationUnit "Security Operations"

Changing the CSR Locality

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrLocalityName "Sydney"

Changing the CSR State

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrStateName "NSW"

Changing the CSR Country

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrCountryCode "AU"

Changing the CSR e-mail address

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" config -g cfgRacSecurity -o cfgRacSecCsrEmailAddr "pki@burgerburgerburger.com"

Resetting the iDRAC unit

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" racreset soft

Generating the CSR

racadm.exe -r dellServer.myCloud.local -u "DRAC_USERNAME" -p "DRAC_PASSWORD" sslcsrgen -g -f "C:\temp\dellServer.mycloud.local.csr"

Once you've approved the CSR, you'll get a nice minted certificate you can use to eliminate those pesky iDRAC errors. It should look something like this (if you're using Chrome on OSX)

I've blacked out the Issuing CA details, but all the details in the certificate Subject Name
match with the script above.

Some other areas that you may want to automate in your environment include

• Configuration of SNMP (for hardware alerting)
• Uploading the certificate
• Renaming the default iDRAC user account and setting a strong password
• Disabling features that are not required
• Changing the default IPMI key

Remember, once you've automated it for one server, the next 1000 servers are easy!

One caveat: I think iDRAC is unstable or has a memory leak: generating a Certificate Signing Request (CSR) only works reliably if you reset the iDRAC beforehand. Once I added this step in, the CSR generation process became more reliable.

Certificate Templates not appearing in Windows Server 2012 R2-based Microsoft Certificate Authority (CertUtil error 0x80070057)

You may have created some certificate templates in your Microsoft Certificate Authority (CA), such as a template for your VMware hosts. Derek Seaman has a good blog post on the exact settings and extensions required.

After creating a certificate template, I had a problem enabling it in the CA. While the certificate template appeared in the Certificate Templates console, it couldn't be enabled. The certificate template just wasn't appearing in the Certification Authority MMC snapin.

It appears in Certificate Templates..

...but you can't enable it. Because it doesn't appear.
IT JUST DOESN'T APPEAR. WHY??!?!?!?!?!?!?!
WHERE ARE THE ERROR MESSAGES MICROSOFT?!?!?!?

I tried using the certutil.exe command to enable the certificate template manually

certutil.exe -SetCATemplates VMware-SSL

Unfortunately, same problem: certificate template wasn't enabled, but this time I got a deceptive and nonsensical error message complaining that the "parameter" was "incorrect".

CertUtil: -SetCATemplates command FAILED: 0x80070057 (WIN32: 87 ERROR_INVALID_PARAMETER).
CertUtil: The parameter is incorrect.

When you create a certificate template, it needs time to replicate to all domain controllers. A certificate template is just another object in Active Directory, just like a user or computer account. So if the certificate template doesn't appear immediately, just wait the same amount of time you'd wait for a user to replicate across your DCs.

Back to our problem: why isn't the certificate template appearing? Well, it turns out that every online certificate enrolment service has to have contacted Active Directory and downloaded the certificate templates before it can be enabled. If you've previously configured an issuing CA and then destroyed it without cleaning up its entries, you'll never be able to enable the certificate template.

Performing a cleanup of issuing CAs in Active Directory Certificate Services

It's ADSI Edit Time!

Open ADSI Edit and connect to the Configuration context.

Select a well known Naming Context like Configuration, or Paul, or Jimmy.

If you see the names of OUs, you connected to the wrong context.

Navigate to CN=Certification Authorities,CN=Public Key Services,CN=Services,CN=Configuration in your domain.

CN=Certification Authorities contains your root CAs and CN=Enrollment Services contains your issuing CAs. If there are any extra CAs listed that no longer exist, you'll need to delete them.

In my case, I had an additional issuing CA in CN=Enrollment Services that no longer existed. When I deleted the CA, I could enable the template.

Yes!

But I want to do everything from the command line because I want to use Server Core in the future.

Now you understand why the original error message "The Parameter is incorrect" is deceptive.
This is the same command that was run last time.

Unfortunately there are no event log error messages for this error. Microsoft just expect you to figure it out.

Errors installing VMware ESXi dump collector: it's probably your complex password!

The VMware ESXi dump collector installer has some vague error messages.

Error 1: Login failed due to a bad user name or password.

"Login probably failed due to a bad user name or password" would
be a more accurate error message.

Causes

• The username is incorrect.I'm assuming you've verified the username and password are correct. If you haven't done this, try logging into Windows with the credentials and see if they are valid.
• The user account does not permission in vCenter.Ensure the user account has permissions in vCenter. For the duration of troubleshooting, you may wish to give the user account administrative access. If you look at the netdump-reg-debug.txt file, you can see the following error.
  
  ERROR:ndreg-app:error: cannot connect to VC -- (vim.fault.NoPermission) {
     dynamicType = <unset>,
     dynamicProperty = (vmodl.DynamicProperty) [],
     msg = 'Permission to perform this operation was denied.',
     faultCause = <unset>,
     faultMessage = (vmodl.LocalizableMessage) [],
     object = 'vim.Folder:group-d1',
     privilegeId = 'System.View'
  }
• Your password contains the special character "VMware haven't escaped parameters correctly. Remove the " from your password and try again.

Error 2: Error 29457. A specified parameter was not correct.

Of the 30,000 error messages, I received error 29457.

Causes

• Your password contains the character ;Fool me once, shame on you. If you look at the vminst.txt log file, you'll see something like
  
  esxiInstUtil: 01/12/15 13:06:12 ExecuteCmd::Cmd:  --register --address "vc.cloudlab.local" --user "svcvmwaredump@cloudlab.local" --password "*****" -s "vUq<~[" --thumbprint "C:\ProgramData\VMware\VMware ESXi Dump Collector\vmconfig-netdump.xml"
  
  The passwords I use are generated by a password management tool, which makes long non-sensical passwords with lots of special characters like !@#$%^&*();. Unfortunately, VMware haven't properly escaped the password field so installation will fail if the password contains the character ;
  
  In this case, my password was JI@$QH$7*@eie$Hhg8;vUq<~[. The installer thinks my password is JI@$QH$7*@eie$Hhg8, ignored the ;  and has left the vUq<~[ dangling.