Preparing Hazelcast to Work with Memory Machine

To prepare Hazelcast to work with Memory Machine Cluster Snapshot you must configure Hazelcast on each host; create a user that belongs to the Memory Machine mvmm group; run Hazelcast on each host in Active state as a background process; and register the Hazelcast cluster with Memory Machine.

About this task

Memory Machine uses a REST API to communcate with the Hazelcast cluster.

Do the following to prepare the Hazelcast hosts for snapshot capture by Memory Machine.

Procedure

On each Hazelcast host, add the REST API to the hazelcast.xml configuration file:

    <network>
        <!-- ADD THESE REST API ENDPOINTS-->
        <rest-api enabled="true">
            <endpoint-group name="HEALTH_CHECK" enabled="true"/>
            <endpoint-group name="CLUSTER_WRITE" enabled="true"/>
        </rest-api>
    </network>

On each Hazelcast host, add the following connection properties to the hazelcast.xml file.

    <properties>
          <!-- ADD THESE PROPERTIES -->
            <property name="hazelcast.connection.monitor.interval">1000</property>
            <property name="hazelcast.connection.monitor.max.faults">10</property>
            <property name="hazelcast.heartbeat.interval.seconds">10</property>
            <property name="hazelcast.merge.next.run.delay.seconds">4</property>
            <property name="hazelcast.merge.first.run.delay.seconds">4</property>
    </properties>

Note: These properties have been verified by MemVerge to work in test installations. You might need to adjust some of the values to tune your own Hazelcast cluster.

If you are using the TCP/IP joiner, ensure that all hosts are referenced by IP address. Reference by hostname is not supported. For example:

            <tcp-ip enabled="true">
                <member-list>
                    <!-- Using IP addresses - OK -->
                    <member>10.0.0.62</member>
                    <member>10.0.0.63</member>
                    <!-- Wrong! using hostnames is not supported -->
                    <member>hz-node-a.example.com</member>
                    <member>hz-node-b.example.com</member>
                </member-list>

If a host has multiple network interfaces, use the interface property in the Hazelcast joiner to define the interface on which the hosts reside. For example:
```
<network>
        <interfaces enabled="true">
            <interface>10.0.0.*</interface>
        </interfaces>
```
For more information about interfaces in the Hazelcast joiner, see the Hazelcast Reference Manual here.
The user under which Hazelcast runs must belong to the mvmm group on every host. Create a new user belonging to mvmm or add the mvmm group to an existing user.
```
# # Add an mvmm user:
# useradd -g mvmm -m -d /home/hzuser hzuser
# # Or add a user to the mvmm group:
# usermod -a -G mvmm existing-user-id
```
On every host, use the Memory Machine mm script to start Hazelcast as a background process. This example runs the Hazelcast start.sh script in the background and appends output to a file called hz-node-c1.log:

Important: Run the Hazelcast start.sh script as shown. Running the script without the file redirections as shown can cause restoration of the snapshotted process to fail.
```
$ nohup mm bin/start.sh < /dev/null >> hz-node-c1.log 2>&1 &
$
```
Choose one of the Hazelcast hosts to be the Primary Coordinator host. This is the host on which the cluster snapshot state is persisted. For example, assume that 10.0.0.63:5701 is assigned to be the Primary Coordinator host.
Gather the IP address for each Hazelcast member on each host. For example, assume that all the Hazelcast members are on two hosts, 10.0.0.62 and 10.0.0.63.
Choose a name for the Hazelcast cluster; for example, hz-cluster-1.
On the Primary Coordinator host, register the Hazelcast cluster using the mvmcli cluster register command:
```
# mvmcli cluster register --name=cluster-name
  --contact_member=contact_member_ip:contact_member_port
  --coordinators=host1:5679,host2:5679 
  --profile=hazelcast
  --hazelcast-cluster-name=hz_cluster_name
  --hazelcast-cluster-password=hz_cluster_password
#
```
Where:

cluster-name

is the name of the cluster

coordinators

are the IP addresses of the cluster hosts

--profile=hazelcast

identifies the type of cluster to the mvmcli command

contact_member_ip

is the IP address of one of the cluster members, preferrably a member on the same host as the primary coordinator, for Memory Machine to communicate with the Hazelcast cluster.

contact_member_port

is the port of the contact member.

hz_cluster_name and hz_cluster_password

are the credentials for your Hazelcast cluster as defined in the hazelcast.xml file. These cluster credentials enable Memory Machine to change the cluster state while taking a snapshot.

Note: 5679 is the port number on which the Memory Machine cluster snapshot service listens. It must be appended to every cluster host IP.

Note: If you enable TLS, additional parameters are required. See Working with Clusters .

For example, using the example values from the previous step (the name and password are made up):
```
# mvmcli cluster register --name=hz-cluster-1 --contact_member=10.0.0.63:5701 --coordinators=10.0.0.62:5679,10.0.0.63:5679 --profile=hazelcast
  hazelcast-cluster-name=hzname hazelcast-cluster-password=hzpass
#
```

Verify that the cluster has been registered using the mvmcli cluster list command. The output should look like the following example:

# mvmcli cluster list --list-coordinator
Name                Profile     Cluster Contact Member   Coordinators          Create Time
------------------------------------------------------------------------------------------
------------
hz-cluster-1        hazelcast   10.0.0.63:5701           10.0.0.62:5679        Tue Dec  1 16:09:15 2020
                                                      10.0.0.63:5679
Snapshots:       None
#

Omit the --list-coordinator option to show only the Primary Coordinator.

What to do next

The system is now prepared to capture a snapshot of the cluster.