docker

Deploying an HRDMBS test cluster with Docker

Overview

• Currently only in dev-env branch
• HRDBMS nodes are wrapped in Docker containers
• Debugging can be done in any environment (Linux, Windows, etc)
• Tested with Docker version: 1.13; Docker composer: 1.10
• Tested with Java version: 1.8.0_101 (already pushed into docker image)
• Tested with IDE: IntelliJ IDEA 2016.2.5 (free student subscription)

Compiling

• Native IDEA Build (used for debugging)
  • Build -> Make Project (Ctrl + F9)
  • Result is bin directory as shown on the screenshot
  • We can lauch executable from bin directory such coordinator, worker, and cli
• IDEA Plugins: ANTLR v4 grammar plugin, IvyIDEA
• Ant build
  • clean
  • createParser (autogenerated code)
  • createJar

Running Docker on Linux

• Open terminal and go to the docker folder of the project
• docker-compose.yml should be available in the folder
• Run docker-compose down (We do not need to run the command if docker containers are not launched yet)
• Run docker-compose up -d
• Note: if your host OS has SELinux enabled, the docker container will not be able to see the /home/hrdbms/app volume. Append :z to the volumes in the docker-compose.yml to remedy

Running Docker on Windows

• Docker Toolbox+Oracle VM Virtual Box have to be installed
• VM should have sufficient amount of memory (launch is possible when RAM = 8 Gb)

Setup shared folder between Your PC and Docker host (Windows Env)

1. Open Oracle VM VirtualBox Manager
2. Open "Settings" for Docker host VM
3. Open "Shared Folders"
4. Remove all existing shares and add new one:
5. Specify any path on Your PC (referenced further as [LOCAL_SHARED_PATH])
6. Enter any name you will remember, e.g. "HRDBMS" (referenced further as [VM_SHARE_NAME])
7. Create sub-directory under your [LOCAL_SHARED_PATH] for specified container, e.g. 'my-first-container' (referenced further as [CONTAINER_DIR])
8. Start or attach to running Docker host VM (use "Start" or "Show" button respectively)
9. Create new directory inside Docker host VM, e.g. /home/docker/hrdbms (referenced further as [DOCKER_HOST_PATH])
10. sudo mkdir -p /home/docker/
11. Mount [VM_SHARE_NAME] to [DOCKER_HOST_PATH] using 'mount -t vboxsf [VM_SHARE_NAME] [DOCKER_HOST_PATH]' command, e.g.
12. sudo mount -t vboxsf -o defaults,uid=id -u docker,gid=id -g docker HRDBMS /home/docker/hrdbms
13. Create/Run Docker container with volume using '-v [DOCKER_HOST_PATH]/[CONTAINER_DIR]:[CONTAINER_SHARE_PATH] Now you have access to [LOCAL_SHARED_PATH]/[CONTAINER_DIR] inside your docker container using [CONTAINER_SHARE_PATH].

Use below commands in Docker Quickstart terminal

• docker-machine ssh
• docker run -it -d -v /home/docker/hrdbms:/home/hrdbms/app --name coordinator -p 5010:5010 mykh74/hrdbms:v2
• docker run -it -d -v /home/docker/hrdbms:/home/hrdbms/app --name worker -p 5015:5015 mykh74/hrdbms:v2

Start HRDBMS from coordinator docker container

• ssh hrdbms@17.17.0.2 (we need to ssh to docker-machine first in Windows)
• password: hrdbms
• sh hrdbms_start.sh
• Instead of hrdmbs_start.sh we can run the following comand:
  • nohup java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5010 -XX:+UseG1GC -XX:G1HeapRegionSize=2m -XX:+ParallelRefProcEnabled -XX:MaxDirectMemorySize=27772160000 -XX:+AggressiveOpts -classpath /home/hrdbms/app/build/HRDBMS.jar: com.exascale.managers.HRDBMSWorker 0 &
  • Port 5010 is opened port for debugging by IDE
• Hrdbms client can be run locally or in docker containers:
  • java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 -Xmx1G -Xms1G -classpath /home/michael/ms-project/Bitbucket/hrdbms/HRDBMS/build/HRDBMS.jar: com.exascale.cli.CLI - local launch of CLI
  • classpath should be local path to local bin folder of the project
  • java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005 -Xmx1G -Xms1G -classpath /home/hrdbms/app/build/HRDBMS.jar: com.exascale.cli.CLI - launch of HRDBMS client in HRDBMS node