In my last post about Heartbeat I gave an example of a script to start and stop a cluster service. In that post I omitted to mention that the script goes in the directory /usr/lib/ocf/resource.d/heartbeat.

To actually use the script you need to write some XML configuration to tell Heartbeat which parameters should be passed to it via environment variables and which nodes may be candidates to run it.

In the below example the type of web means that the script /usr/lib/ocf/resource.d/heartbeat/web will be called to do the work. The id attributes are all arbitrary, but you want to decide on some sort of consistent naming scheme. I have decided to name web server instances web-X where X is the IP address used for providing the service.

The nvpair element contains a configuration option that will be passed to the script as an environment variable. The name of ip means that the environment variable will be named OCF_RESKEY_ip. Naming of such variables is arbitrary and a script may take many variables. A well written script (which incidentally does not mean my previous blog post) will have an option meta-data to give XML output describing all the variables that it accepts. An example of this can be seen by the command /usr/lib/ocf/resource.d/heartbeat/IPaddr2 meta-data.

In the XML the resources section (as specified by --obj_type resources on the cibadmin command-line) describes resources that the Heartbeat system will run. The constraints section specifies a set of rules that determine where they will run. If the symmetric-cluster attribute in the cluster_property_set is set to true then resources will be permitted to run anywhere, if it is set to false then a resource will not run anywhere unless there is a constraint specifying that it should do so – which means that there must be at least one constraint rule for every resource that is permitted to run.

In the below example I have constraint rules for the service giving node-0 and node-1 a priority of 9000 for running the service.

In a future post I will describe the cluster_property_set and how it affects calculations of where resources should run.
Continue reading →

A service script for Heartbeat needs to support at least three operations, start, stop, and status. The operations will return 0 on success, 7 on failure (which in the case of the monitor script means that the service is not running) and any other value to indicate that something has gone wrong.

In the second half of this post (not in the feed) I have included an example service script. It is a very brief script and does not support some of the optional parameters (monitor, validate-all, and meta-data). So this script is not of a quality that would be accepted for inclusion in a Heartbeat release but is adequate to demonstrate the concepts.

The XML configuration for a service can have an arbitrary set of name-value pairs, and they are passed to the script as environment variables. For example the below script expects that the XML configuration item named ip will have the IP address used by the service, my script receives this as a variable named OCF_RESKEY_ip. My script doesn’t use the address, it merely allows it to be inherited by the IPaddr2 script (which is part of the Heartbeat distribution) and that script assigns the address to an Ethernet interface.

The script is for testing Heartbeat, it mounts a filesystem and starts Apache (which is configured to serve web pages from the filesystem in question on the IP address supplied by the ip parameter).

For test purposes the script looks for a file named /root/fail, if this file exists then the status check will always abort. An aborting status script means that Heartbeat can not be certain that the node in question has released all resources that it was using for the service. This means that Heartbeat will have to kill the node via the STONITH service. Such test scripts are the only way to test that STONITH works, and I believe that it’s a necessary part of pre-production testing of a Heartbeat cluster.

Update: Made it display error messages in all cases and also reformatted it for better cut/paste.
Continue reading →

Below is a sample script to configure the ssh STONITH agent for the Heartbeat system. STONITH will reboot nodes when things go wrong to restore the integrity of the cluster.

The STONITH test program supports the -n option to list parameters and the -l option to list nodes. The following is an example of using it with the ssh agent:
# stonith -t ssh -n
hostlist
# stonith -t ssh hostlist="node-0 node-1" -l
node-0
node-1

The hostlist tuple is the only configuration option for ssh. It is assumed that you have passwordless ssh logins allowed between root on all the nodes in the cluster so the host name list is all that’s needed.

The important thing to note about the constraint is that you are constraining the parent of the clones (which in this example has an ID of “DoFencing“) not a clone instance.

ssh is the simplest and in many ways least useful method of STONITH, but it’s good for an example as everyone knows it. Once you get ssh going it’ll be trivial to get the other methods working.

See below for the script to insert the XML in the CIB.
Continue reading →