Posted by: David | July 31, 2012

Blackberry Administration Service (BAS) not working on Standby Server

I recently came across this issue when trying to install a Standby server in a HA pair for Blackberry Enterprise Server (as part of a migration). After following KB22499 I got to step 9 to find that I could not log in to the Administration service.

It should be noted that at Step 7, where I was supposed to enter a new name for the Administration Service Pool, this option was greyed out, and the installation completed with the existing name. Unfortunaly the existing name was the FQDN of the Primary BES server, this would need to be changed.

After changing this in the Blackberry Server Confirugation Tool (Administration Service – High Availabiliy Tab) and restarting the BAS-AS and BAS-NCC services on the primary server the pool had a new name, (BESHA.domain.local). I then created a DNS record for this in my local DNS server and pointed this for the time being at the Primary Server IP.

Checking the BAS-AS logs on the new server showed me a java.net.BindException: Address already in use: JVM_Bind error.

After some troubleshooting and with the command netstat -ano | findstr ":443" this showed that a process with id of 4 was using the port. This is the Windows System process and through a considerable amount of googling I was unable to identify which roles was causing System to listen on port 443. IIS was not installed on the server.

I changed the port of the BAS to 663 using the Blackberry Server Confirugation Tool (Administration Service – High Availabiliy Tab) and restarted the BAS service on the Primary server. Still no Administration Service on the Standby server.

Checking the BAS-AS logs showed the following errors:

java.io.IOException: Error initializing server socket factory SSL context: null
[org.apache.coyote.http11.Http11Protocol] [ERROR] Error initializing endpoint
[org.apache.catalina.startup.Catalina] [ERROR] Catalina.start LifecycleException: Protocol handler initialization failed:

Checking the BAS-NCC logs showed the following errors:

[org.jboss.system.ServiceController] [WARN] Problem starting service jboss.security:service=JaasSecurityDomain,domain=SSLAdvanced
[org.jboss.system.server.Server] [ERROR] Root deployment has missing dependencies; continuing

After a lot of digging I found that these errors were related to the SSL certificate that the Administration Service is trying to use an incorrect certificate. Checking the Administration Guide this shows how to import a new certificate for the site, looking at Step 10, parts, 1-3 I noticed that if you change the certificate it needs to be copied to other servers in the BAS pool.

As the BAS was working on the primary server I did the following steps to fix the issue.

  1. On Standby server log in a BES Administrator account (Same account used by Blackberry Services)
  2. On Standby server stop the BAS-AS and BES-backup the <program files>Research In MotionBlackBerry Enterprise ServerBASbinweb.keystore file
  3. On standby server backup the following regkey: HKCUSoftwareResearch In MotionBlackberry Enterprise ServerAdministration ServicesKey Store
  4. On Primary Server log in as BES Administration account and copy the <program files>Research In MotionBlackBerry Enterprise ServerBASbinweb.keystore over the same file on the Standby server
  5. On Primary Server export: HKCUSoftwareResearch In MotionBlackberry Enterprise ServerAdministration ServicesKey Store, import in to Registry on the Standby server
  6. Start the BAS-NCC service and then BAS-AS service on the Standby server
  7. In DNS update the A record for your BAS pool address (BESHA.domain.local) to point to the IP address of your Standby server.
  8. On the standby server flush the dns (ipconfig /flushdns) and check to see if the Admin service is now working. Check netstat -ano | findstr ":663" look for process ID, then use task manager (processes tab) add PID column and check that the ID number for the process resolves to java.
  9. If you want BAS work in round robin on DNS, create a second A record pointing to the standby server instead of changing the address.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: