Containers

Introduction

A container image is a template for creating a container, i.e., a container is the runnable instance of a container image. Applications submitted to Memory Machine Cloud Edition must be packaged in images compliant with the Open Container Initiative (OCI) image format. An example of an OCI-compliant container image is Docker.

Docker

The company Docker was an early developer of Linux container technology and the Docker platform is often considered the de-facto standard. Docker uses a client-server architecture in which the client uses docker commands to interact with the docker daemon (dockerd). Docker is a complete container environment — images are built, and containers are deployed and managed using the docker CLI.

Container Image Registry

A container repository is a collection of container images. A registry is a collection of repositories and also a service that allows users to store, access, and share images in specific repositories. A repository may be public in which case all images in the repository are available to all users. A private repository restricts access so that only users who have a valid username and access token may access the repository.

MemVerge maintains a private repository at Docker Hub to store images that are automatically loaded by Memory Machine CE.

Singularity

Singularity is an alternative to Docker. Singularity is commonly used in HPC (high-performance computing) environments whereas Docker is widely used in VM and cloud deployments. Although similar in many ways, differing design goals lead to significant differences also. Docker requires a separate daemon process that runs with root privileges whereas Singularity runs with user privileges and does not require a separate daemon. Docker stresses isolation from the host system, for example, processes running inside the Docker container do not have easy access to the host file system. Singularity stresses integration, for example, some host file systems are automatically mounted inside the container.

Although it has its own container format called Singularity Image Format (SIF), Singularity can pull images from Docker Hub and convert them to SIF format on the fly. It is possible to convert Singularity images to Docker images but the conversion usually happens the other way.

Kubernetes

A container is a convenient way to package an application along with its dependencies so that the resulting image can be used to start multiple application instances with reproducible results. Deploying large numbers of containers requires a container orchestration and management platform. Originally developed by Google, Kubernetes, also known as k8s, is the most popular of these platforms. Red Hat provides a supported version called OpenShift.

The smallest execution unit in Kubernetes is the pod. A pod includes one or more containers and runs on a worker node. Kubernetes includes workload resources called controllers that create and manage pods in a cluster. For example, the controller can replicate pods to scale up or delete pods to scale down. Controllers can also ensure that a minimum number of pods is running at all times.

Memory Machine CE is not a replacement for Kubernetes. Both platforms start and run Docker containers. Where Kubernetes manages and orchestrates containers in terms of pods running on worker nodes deployed in clusters, Memory Machine CE deploys individual containers and manages at the job level.

Using Containers with Memory Machine CE

If you use existing Docker images (accessible from a public or private repository) to submit workloads to Memory Machine CE, you can complete all the necessary tasks using the float CLI. To build your own custom image and upload it to a repository, you can use docker or another container management tool such as podman.

If you build a container image on your local server, using docker or buildah for example, you can bypass the image registry and upload the image directly to the OpCenter using float commands.