mod_cluster Documentation

Complete documentation for legacy versions is archived at https://docs.modcluster.io/legacy/.

The mod_cluster container integration module (implemented in Java) is provided for all the following containers:

Most of the standard distributions contain Apache httpd server. Use your platform’s specific install tools to install. For example, in Fedora, just install as a root:

If you already have a working httpd install that you would prefer to use and your distribution contains mod_cluster/mod_proxy_cluster just install it:

And then you have the files below in your module directory:

httpd version mismatch: [warn] httpd version mismatch detected Please, beware that one cannot simply load the aforementioned modules into an arbitrary httpd installation. These modules were built with a particular minor httpd version, and they cannot be used with an older one.

Unzip the bundle corresponding to your architecture. Change to the bin directory of the subfolder Apache24 where you unzipped the bundle.

You may run httpd directly by using:

or install Apache HTTP Server as a service:

and start the service via net start or using httpd.exe:

or

Assuming $CATALINA_HOME indicates the root of your Tomcat install:

Note that you should copy only the tomcat files corresponding to your tomcat installation.

WildFly already includes mod_cluster integration so no extra installation steps are necessary. The mod_cluster subsystem is pre-configured in the HA configurations. For WildFly-specific configuration refer to WildFly documentation.

No post-installation configuration necessary!

Edit the $CATALINA_HOME/conf/server.xml file, adding the following next to the other <Listener/> elements:

To start httpd do the following:

The ModClusterConfig bean enumerates the configuration properties used by mod_cluster. For the complete list of configuration properties and their default values, see the chapter entitled Server-side Configuration Properties.

e.g.

Like mod_jk and mod_proxy_balancer, mod_cluster requires a connector in your server.xml to which to forward web requests. Unlike mod_jk and mod_proxy_balancer, mod_cluster is not confined to AJP, but can use HTTP as well. While AJP is generally faster, an HTTP connector can optionally be secured via SSL.

Since mod_cluster 1.4, the connector registered with the balancer is no longer automatically chosen by mod_cluster and needs to be specified explicitly. The new attributes connectorPort and/or connectorAddress need to be configured explicitly matching exactly one of the configured connectors, e.g.:

In mod_cluster version 1.3 and older, if multiple possible connectors are defined in your server.xml, mod_cluster uses the following algorithm to choose between them:

Like mod_jk and mod_proxy_balancer, mod_cluster identifies each node via a unique jvm route. By default, mod_cluster uses the following algorithm to assign the jvm route for a given node:

While UUIDs are ideal for production systems, in a development or testing environment, it is useful to know which node served a given request just by looking at the jvm route. In this case, you can utilize the org.jboss.modcluster.SimpleJvmRouteFactory. The factory generates jvm routes of the form:

bind-address:*port*:*engine-name*

The entry point for JBoss Web and Tomcat configuration is the ModClusterListener. All mod_cluster configuration properties are defined as attributes of the <Listener/> element. For the complete list of configuration properties and their default values, see the chapter entitled "Server-side Configuration Properties".

e.g.

mod_cluster uses jboss-logging, which exists in JBoss Web, but not in Tomcat. Consequently, to use mod_cluster with Tomcat, it is necessary to add jboss-logging-spi.jar to $CATALINA_HOME/lib.

The mod_cluster integration is done via the modcluster subsystem.

The minimal configuration is having the modcluster schemaLocation in the schemaLocation list:

and subsystem declaration like:

With that configuration modcluster will listen for advertise on 224.0.1.105:23364.

The simple load provider always sends the same load factor. Its purpose is testing, experiments and special scenarios such as hot stand-by.

The dynamic load provide allows to have load-metric as well as custom-load-metric defined. For example:

The load-metric are the "classes" collecting information to allow computation of the load factor sent to httpd.

The custom-load-metric are for user defined "classes" collecting information. They are like the load-metric except type is replaced by class:

See an Example Custom Load Metric that reads load from a local file.

The load-metric have 4 commands to add and remove metrics

Building mod_cluster’s worker-side components from source requires the following tools:

Steps to build:

The build produces the following output in the target directory:

The tables below enumerate the configuration properties available to an application server node. The location for these properties depends on how mod_cluster is configured.

The following configuration values are sent to proxies during server startup, when a proxy is detected via the advertise mechanism, or during the resetting of a proxy’s configuration during error recovery.

You don’t need to do anything to install the java-side binaries in AS 6.x; it’s part of the AS distribution’s default, standard and all configurations.

Assuming \$JBOSS_HOME indicates the root of your JBoss AS install and that you want to use mod_cluster in the AS’s all config:

Assuming $CATALINA_HOME indicates the root of your Tomcat install:

and additionally for Tomcat6:

and additionally for Tomcat7:

CreateBalancers define how balancers are created in the httpd VirtualHosts. This is to allow directives like:

Default: 2

Check that the Alias corresponds to the ServerName (See Host Name Aliases).

Default: Off Ignore the Alias information from the nodes.

Time interval in seconds for load balancing logic to recalculate the status of a node.

Default: 5 seconds

The actual formula to recalculate the status of a node is:

lbfactor is received for the node via STATUS messages.lbstatus is recalculated every LBstatusRecalTime seconds using the formula: lbstatus = (elected - oldelected) * 1000) / lbfactor

where elected is the amount of time the worker has been elected. oldelected is elected from the last time the lbstatus was recalculated. The node with the lowest status is selected. Nodes with lbfactor ≤ 0 are skipped by the both calculation logics.

Time in seconds before a removed node is forgotten by httpd.

Default: 10 seconds

ProxyPassMatch and ProxyPass are mod_proxy directives that when using ! (instead the back-end url) prevent to reverse-proxy in the path. This could be used allow httpd to serve static information like images.

The above for example will allow httpd to serve the .gif files directly.

Use OPTIONS method to periodically check the active connection. Fulfils the same role as the CPING/CPONG used by AJP but for HTTP/HTTPS connections. The endpoint needs to implement HTTP/1.0.

Default: On

Use AJPSecret your_secret to provide the secret for the AJP back-end. See mod_proxy_ajp and mod_proxy docs for more information. If your_secret doesn’t correspond to the value configured in the back-end the back-end will return 503 to any request coming through the proxy.

Use ws or wss instead of http or https when creating nodes (allows WebSocket proxying).

Use WSUpgradeHeader value to define the value of the upgrade header that is accepted (corresponds to ProxyPass upgrade=value). Accepted values are following:

See mod_proxy_http documentation for more information.

Size in bytes of the HTTP/1.1 buffers of the workers, that limits the header size a webapp can use (Note: In Tomcat there is maxHttpHeaderSize that also limits it in the Connector).

Default: 8192

Time to cache the shared memory information in seconds.

Default: 0 (no-caching)

Set of health check parameters to use with mod_proxy_cluster workers.

When no ProxyPass or ProxyMatch match the URL, pass the raw URL path to the backend.

Default: Off

Response code returned when ProxyPass or ProxyMatch doesn’t have any matching context. If the configured value is not a standard HTTP response status code, the server will return 500 instead (see MODCLUSTER-801). (Since 1.3.20.Final)

Default: 404

Number of threads that should be created for watchdog logic. Must be positive. (Since 2.0)

Default: 16

Controls whether a node upon failover is chosen deterministically.

Default: Off

EnableMCMPReceive – allow the VirtualHost to receive Mod-Cluster Management Protocol (MCMP) messages. You need one EnableMCMPReceive in your httpd configuration to allow mod_proxy_cluster to work, put it in the VirtualHost where you configure advertise.

This directive was added to address the issue of receiving MCMP on arbitrary VirtualHosts which was problematic due to accepting messages on insecure, unintended VirtualHosts.

Default: disabled (presence of the directive enables this functionality)

That is the base name for the names mod_manager will use to store configuration, generate keys for shared memory or lock files. The value may be an absolute path name or a relative one (then it will be relative to the server root); the directories will be created if needed. It is highly recommended that those files are placed on a local drive and not an NFS share. (Context: server config)

Default: $server_root/logs/

The maximum number of application contexts supported by mod_proxy_cluster. (Context: server config)

Default: 100 (If Maxhost is bigger than Maxcontext, then Maxcontext is increased to Maxhost.)

That is the maximum number of nodes supported by mod_proxy_cluster. (Context: server config)

Default: 20

That is the maximum number of hosts (Aliases) supported by mod_proxy_cluster. That is also the max number of balancers. (Context: server config)

Default: 20 (If Maxnode is bigger than Maxhost, then Maxhost is increased to Maxnode.)

Maxsessionid: That is the number of active sessionid we store to give number of active sessions in the mod_cluster-manager handler. A session is inactive when mod_cluster doesn’t receive any information from the session in 5 minutes. (Context: server config)

Default: 0 (the logic is not activated).

MaxMCMPMaxMessSize: Maximum size of MCMP messages. from other Max directives.

Default: calculated from other Max directives. Min: 1024

ManagerBalancerName: That is the name of balancer to use when the JBoss AS/JBossWeb/Tomcat doesn’t provide a balancer name.

Default: mycluster

PersistSlots: Tell mod_cluster_slotmem to persist the nodes, Alias and Context in files. (Context: server config)

Default: Off

CheckNonce: Switch check of nonce when using mod_cluster-manager handler on | off

Default: on (Nonce checked)

AllowDisplay: Switch additional display on mod_cluster-manager main page on | off

Default: off (Only version displayed)

AllowCmd: Allow commands using mod_cluster-manager URL on | off

Default: on (Commands allowed)

ReduceDisplay - Reduce the information the main mod_cluster-manager page to allow more nodes in the page. on | off

Default: off (Full information displayed)

SetHandler mod_cluster-manager: That is the handler to display the node mod_proxy_cluster sees from the cluster. It displays the information about the nodes like INFO and additionally counts the number of active sessions.

When accessing the location you define in httpd.conf you get something like:

Note that:

mod_advertise uses multicast packets to advertise the VirtualHost where it is configured that must be the same VirtualHost where mod_manager is defined. Of course at least one mod_advertise must be in the VirtualHost to allow mod_proxy_cluster to find the right IP and port to give to the ClusterListener.

Default: Off. (Any Advertise directive in a VirtualHost sets it to On in the VirtualHost)

AdvertiseGroup IP:port: That is the multicast address to use (something like 232.0.0.2:8888 for example). IP should correspond to AdvertiseGroupAddress and port to AdvertisePort in the JBoss AS/JBossWeb/Tomcat configuration. Note that if JBoss AS is used and the -u startup switch is included in the AS startup command, the default AdvertiseGroupAddress is the value passed via the -u. If port is missing the default port will be used: 23364.

Default: 224.0.1.105:23364.

AdvertiseFrequency seconds[.miliseconds]: Time between the multicast messages advertising the IP and port.

Default: 10

AdvertiseSecurityKey value: key string used to verify advertisements checksums. If configured on either side the verification is required. Both sides must use the same security key.

Default: No default value.

AdvertiseManagerUrl value: Not used in this version (It is sent in the X-Manager-Url: value header). That is the URL that JBoss AS/JBossWeb/Tomcat should use to send information to mod_cluster

Default: No default value. Information not sent.

AdvertiseBindAddress IP:port: That is the address and port httpd is bind to send the multicast messages. This allow to specify an address on multi IP address boxes.

Default: 0.0.0.0:23364

For mod_proxy_cluster 1.3.x you have to change the slotmem module to:

We assume you already have a functional Apache HTTP Server on Windows. This example works with Apache Lounge HTTP Server. We also assume the system has MS Visual Studio (Community Edition is ample) and CMake installed. The example operates in cmder shell, but it is not mandatory. A simple Windows cmd prompt would work too.

or download zipped main branch directly.

Here comes the only slightly tricky part: Apache Lounge httpd ships all necessary *.lib files with exported symbols but for mod_proxy. Since mod_proxy is our dependency, we have to generate these exported symbols from mod_proxy dll.

Let’s run CMake:

Compile

Directory modules now contains all necessary modules:

Done.

As for Windows, you can download the httpd bundle as well from here. Alternatively, you can use your distribution’s repositories (on Fedora, you can install httpd simply by executing dnf install httpd), or you can build httpd from sources.

To build httpd-2.4.x from its sources see ASF httpd 2.4 doc.

Download the sources and configure httpd with following:

Build (make) and install (make install) httpd as configured.

You need an httpd installation with mod_proxy (--enable-proxy) and ajp protocol (--enable-proxy-ajp) enabled and with dso enabled (--enable-so).

Download the mod_proxy_cluster sources:

or download zipped main branch directly.

Build the mod_proxy_cluster’s modules components, for each subdirectory advertise, mod_manager and mod_proxy_cluster do following:

or alternatively using CMake:

Where $APACHE_DIR is the location of the installed httpd.

The apxs file can be found in your $APACHE_DIR/bin directory.

Once that is done use Apache httpd configuration to configure mod_proxy_cluster.

A minimal configuration for mod_proxy_cluster to work is needed in httpd. A listener must be added in JWS/Tomcat’s conf/server.xml.

The httpd.conf is located in httpd/conf/ directory. To quickly test that everything is in place, add the configuration from the minimal example.

To start httpd do the following:

As the ClusterListener allows to configure httpd it is advised to use SSL for that connection. The most easy is to use a virtual host that will only be used to receive information from JBossWEB. Both side need configuration.

in jboss-as/server/production/deploy/mod-cluster.sar/META-INF/mod-cluster-jboss-beans.xml.

In server.xml, change

to

That happens when Advertise is not working and no Proxy List is configured: The worker nodes do not get the from Apache HTTP Server.

When accessing the mod_cluster-manager console you should get something like: TODO: Image. If not, go to the Minimal Example and add the missing directive(s).

it means that the worker was unable to contact the balancer. Keep in mind that the communication is bidirectional.

You can use telnet hostname/address port to check that it is OK, e.g.:

Check that the address and port are the expected ones you may use ServerAdvertise directive in you mod_cluster httpd configuration:

You need to have something like the undermentioned authentication configured in the EnableMCMPReceive marked VirtualHost: