Enable SSL on remoting-jmx connections for JBoss EAP6 and Zabbix

After setting up Zabbix to monitor JBoss EAP 6 hosts, my next goal is to secure the connections between the Zabbix server and the JBoss EAP 6 hosts using SSL.

If you’re reading this to just enable SSL for the management interface just ignore the port numbers I use. I need use port 7777 because of my Zabbix environment. The default management port would be 9999.

1. Create a keystore to hold the certificate

Java comes with a tool called “keytool” which can be used to generated an SSL certificate. Let’s run it to create the keystore, I just used the defaults by pressing Enter at every question, but feel free to modify it to your needs. Just make sure:

  • Remember the alias name
  • Use the same password  for the keystore and the key!
# keytool -genkey -alias kanbier.lan -keyalg RSA -keystore /opt/jboss-eap-6.0/standalone/configuration/kanbier-lan.keystore
Enter keystore password:changeit
Re-enter new password: changeit
What is your first and last name?
 [Unknown]:
What is the name of your organizational unit?
 [Unknown]:
What is the name of your organization?
 [Unknown]:
What is the name of your City or Locality?
 [Unknown]:
What is the name of your State or Province?
 [Unknown]:
What is the two-letter country code for this unit?
 [Unknown]:
Is CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown correct?
 [no]: yes

Enter key password for <kanbier.lan>(RETURN if same as keystore password): changeit

Verify the keystore has been generated:

# ll /opt/jboss-eap-6.0/standalone/configuration/kanbier-lan.keystore
-rw-r--r--. 1 root root 1369 Feb 19 06:21 /opt/jboss-eap-6.0/standalone/configuration/kanbier-lan.keystore

2. Tell JBoss to use the keystore we created

Currently I don’t believe it’s possible to configure the use of SSL via the JBoss management console, so we’ll be using the jboss-cli.sh tool.

Make sure your JBoss instance is running and connect to the CLI:

 # /opt/jboss-eap-6.0/bin/jboss-cli.sh --connect --controller=10.37.129.5:7777
[standalone@10.37.129.5:7777 /] /core-service=management/security-realm=ManagementRealm/server-identity=ssl:add(keystore-password="changeit", keystore-path="kanbier-lan.keystore", keystore-relative-to="jboss.server.config.dir", alias="kanbier.lan")
{
 "outcome" => "success",
 "response-headers" => {
 "operation-requires-reload" => true,
 "process-state" => "reload-required"
 }
}
[standalone@10.37.129.5:7777 /] /core-service=management/management-interface=native-interface/:write-attribute(name=socket-binding,value=management-https)
{
 "outcome" => "success",
 "response-headers" => {
 "operation-requires-reload" => true,
 "process-state" => "reload-required"
 }
}

Restart your JBoss instance now to load the new configuration, let’s test it out. After restarting, try to connect to the CLI again:

# /opt/jboss-eap-6.0/bin/jboss-cli.sh --connect --controller=10.37.129.5:7777
org.jboss.as.cli.CliInitializationException: Failed to connect to the controller
 at org.jboss.as.cli.impl.CliLauncher.initCommandContext(CliLauncher.java:264)
 at org.jboss.as.cli.impl.CliLauncher.main(CliLauncher.java:242)
 at org.jboss.as.cli.CommandLineMain.main(CommandLineMain.java:34)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.jboss.modules.Module.run(Module.java:270)
 at org.jboss.modules.Main.main(Main.java:294)
Caused by: org.jboss.as.cli.CommandLineException: The controller is not available at 10.37.129.5:7777
 at org.jboss.as.cli.impl.CommandContextImpl.tryConnection(CommandContextImpl.java:888)

The management is not available anymore on the non-SSL port. Let’s try the default SSL enabled https port:

# /opt/jboss-eap-6.0/bin/jboss-cli.sh --connect --controller=10.37.129.5:9443
Unable to connect due to unrecognised server certificate
Subject - CN=Unknown,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown
Issuer - CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Valid From - Wed Feb 19 06:21:20 CET 2014
Valid To - Tue May 20 07:21:20 CEST 2014
MD5 : 63:c2:78:d0:68:da:01:25:07:2a:c0:7b:70:f9:10:b0
SHA1 : 6d:42:01:26:fb:9c:60:91:99:e5:f0:6f:c5:d4:a4:aa:6e:83:cd:fd

Accept certificate? [N]o, [T]emporarily, [P]ermenantly :

We can now connect using SSL.

3. Configure the SSL enabled port to be 7777 for Zabbix

We enabled SSL on the JBoss end, the Zabbix Java Gateway now has two issues we need to deal with:

  • Zabbix connects to port 7777 for JMX checks
  • Zabbix doesn’t know about the certificate

I don’t want to modify the port number which Zabbix uses to connect to the JMX interface because port 7777 is the only port available for remoting-jmx connections. So I need to modify the JBoss configuration file so that port 7777 is the https management port.

I run the standalone-full configuration, so open up /opt/jboss-eap-6.0/standalone/configuration/standalone-full.xml and put the native management port back to the default of 9999 and the https port to 7777:

<socket-binding name="management-native" interface="management" port="${jboss.management.native.port:9999}"/>
<socket-binding name="management-https" interface="management" port="${jboss.management.https.port:7777}"/>

After changing this, restart your JBoss instance. Now we can use SSL enable connections on port 7777.

4. Create a truststore and enable SSL on the Zabbix Java Gateway

We need to create a thruststore for the Zabbix Java Gateway so it knows we trust the self-signed certificate we made in the previous steps. Again we’ll use the keytool to do this.

Extract the certificate from our keystore:

$ keytool -export -alias kanbier.lan -keystore /opt/jboss-eap-6.0/standalone/configuration/kanbier-lan.keystore -rfc -file kanbier-lan.crt
Enter keystore password:
Certificate stored in file <kanbier-lan.crt>

Create a truststore and add the certificate to it:

$ keytool -import -alias kanbier.lan -file kanbier-lan.crt -keystore kanbier-lan.truststore
Enter keystore password:changeit
Re-enter new password:changeit
Owner: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Issuer: CN=Unknown, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown
Serial number: 53043f50
Valid from: Wed Feb 19 06:21:20 CET 2014 until: Tue May 20 07:21:20 CEST 2014
Certificate fingerprints:
 MD5: 63:C2:78:D0:68:DA:01:25:07:2A:C0:7B:70:F9:10:B0
 SHA1: 6D:42:01:26:FB:9C:60:91:99:E5:F0:6F:C5:D4:A4:AA:6E:83:CD:FD
 Signature algorithm name: SHA1withRSA
 Version: 3
Trust this certificate? [no]: yes
Certificate was added to keystore

We now have a file called kanbier-lan.truststore for use by the zabbix java gateway.

5. Place the truststore on the zabbix server and add options to the startup scripts of zabbix-java-gateway

I copied over the kanbier.lan-truststore file to /etc/zabbix on the zabbix server, now we just have to tell the zabbix-java-gateway to use it.

On the Zabbix Server:

I’m using the RPM version of Zabbix, which if I’m correct doesn’t come with start/stop scripts for the java gateway other than the init script. There is probably a nicer way to do this, but for now I’ve added these lines to the init script to load the truststore and enable SSL:

$ vi /etc/init.d/zabbix-java-gateway

In the start case BEFORE the line:

"COMMAND_LINE="$JAVA $JAVA_OPTIONS -classpath $CLASSPATH $ZABBIX_OPTIONS com.zabbix.gateway.JavaGateway"

Add:

# Enable SSL
 JAVA_OPTIONS="$JAVA_OPTIONS -Djavax.net.ssl.trustStore=/etc/zabbix/kanbier-lan.truststore"
 JAVA_OPTIONS="$JAVA_OPTIONS -Djavax.net.ssl.trustStorePassword=changeit"

Restart zabbix-java-gateway, connections made to JBoss EAP6 on port 7777 should be using SSL now.

To make sure SSL is used you can start the JBoss instance with an added option:

/opt/jboss-eap-6.0/bin/standalone.sh --server-config=standalone-full.xml -b 10.37.129.5 -Djboss.bind.address.management=10.37.129.5 -Djavax.net.debug=all

This will generate a lot of output when a SSL connection is made:

07:42:41,292 INFO [stdout] (Remoting "jboss6:MANAGEMENT" read-1) Remoting "jboss6:MANAGEMENT" read-1, READ: TLSv1 Handshake, length = 113
07:42:41,292 INFO [stdout] (Remoting "jboss6:MANAGEMENT" read-1) *** ClientHello, TLSv1

That rounds up my experience setting up SSL for remote management, happy monitoring!

One thought on “Enable SSL on remoting-jmx connections for JBoss EAP6 and Zabbix

Leave a Reply

Your email address will not be published. Required fields are marked *