Plugin v2

.travis.yml

@@ -0,0 +1,11 @@
+dist: xenial
+sudo: false
+language: python
+python:
+    - "3.5"
+install: 
+  - sudo apt-get install -y tox
+
+script: 
+  - tox -e py35 -- test.test_hpe_plugin_v2
+  - tox -e pep8

Dockerfile

@@ -1,4 +1,4 @@
-FROM alpine:edge
+FROM alpine:3.8
 
 ENV DEBIAN_FRONTEND=noninteractive
 ENV PYTHONPATH=${HOME}/python-hpedockerplugin:/root/python-hpedockerplugin
@@ -20,6 +20,7 @@ RUN apk add --no-cache --update \
     sg3_utils\
     eudev \
     libssl1.0 \
+    nfs-utils \
 	sudo \
  && apk update \
  && apk upgrade \
@@ -67,6 +68,13 @@ RUN mkdir -p /opt/hpe/data
 RUN chmod u+x /usr/bin/iscsiadm
 RUN chmod u+x /usr/bin/cleanup.sh
 
+# Patch the os_brick, twisted modules
+
+COPY ./patch_os_bricks/linuxscsi.py /usr/lib/python3.6/site-packages/os_brick-1.13.1-py3.6.egg/os_brick/initiator/linuxscsi.py
+COPY ./patch_os_bricks/rootwrap.py /usr/lib/python3.6/site-packages/os_brick-1.13.1-py3.6.egg/os_brick/privileged/rootwrap.py
+COPY ./oslo/comm.py /usr/lib/python3.6/site-packages/oslo.privsep-1.29.0-py3.6.egg/oslo_privsep/comm.py
+COPY ./patch_os_bricks/compat.py /usr/lib/python3.6/site-packages/Twisted-18.7.0rc1-py3.6-linux-x86_64.egg/twisted/python/compat.py
+
 WORKDIR /python-hpedockerplugin
 ENTRYPOINT ["/bin/sh", "-c", "./plugin-start"]
 
@@ -83,4 +91,3 @@ RUN sed -i \
 ENV TAG $TAG
 ENV GIT_SHA $GIT_SHA
 ENV BUILD_DATE $BUILD_DATE
-

buildDockerPlugin.sh

@@ -92,18 +92,10 @@ rc=$?
   exit $rc
 fi
 
-# Patch the os-bricks code
-
-sudo rm ./v2plugin/rootfs/usr/lib/python3.6/site-packages/os_brick-1.13.1-py3.6.egg/os_brick/initiator/linuxscsi.pyc
-sudo rm ./v2plugin/rootfs/usr/lib/python3.6/site-packages/os_brick-1.13.1-py3.6.egg/os_brick/privileged/rootwrap.pyc
-sudo cp ./patch_os_bricks/linuxscsi.py ./v2plugin/rootfs/usr/lib/python3.6/site-packages/os_brick-1.13.1-py3.6.egg/os_brick/initiator/linuxscsi.py
-sudo cp ./patch_os_bricks/rootwrap.py ./v2plugin/rootfs/usr/lib/python3.6/site-packages/os_brick-1.13.1-py3.6.egg/os_brick/privileged/rootwrap.py
-sudo cp ./patch_os_bricks/compat.py ./v2plugin/rootfs/usr/lib/python3.6/site-packages/Twisted-18.7.0rc1-py3.6-linux-x86_64.egg/twisted/python/compat.py
-sudo cp ./oslo/comm.py ./v2plugin/rootfs/usr/lib/python3.6/site-packages/oslo.privsep-1.29.0-py3.6.egg/oslo_privsep/comm.py
-
-
-# end of patch for os-bricks
 # minor modification to remove the .git folder from getting packaged
 # into v2plugin folder
 rm -rf ./v2plugin/rootfs/python-hpedockerplugin/.git
+rm -rf ./v2plugin/rootfs/python-hpedockerplugin/.tox
 docker plugin create ${pluginName} v2plugin
+rm -rf v2plugin
+rm -f unit_tests_run.log

config/create_share_help.txt

@@ -0,0 +1,47 @@
+
+
+===============================================
+HPE 3PAR Share Plug-in For Docker: Create Help
+===============================================
+Create a share in HPE 3PAR using HPE 3PAR volume plug-in for Docker.
+
+-o filePersona          Presence of this flag allows the File Persona driver to process the request
+-o cpg=x                  x specifies the cpg to be used for the share. This parameter can be used with or without
+                            ‘fpg’ option. When used with ‘fpg’, the FPG is created with the specified name if it
+                            does not exist. If it does exist, then share is created under it.
+                            When used without ‘fpg’ option, default FPG under the specified CPG is selected for share
+                            creation. If default FPG does not exist, a new default FPG is created under which the
+                            share is created.
+-o fpg=x                  x is the name of the file provisioning group (FPG). This option must be specified when user wants
+                            to use a non-default FPG or a legacy FPG. The FPG may or may not be an existing one.
+                            For a non-existing FPG x, a new FPG is created using the CPG that is either explicitly
+                            specified with '-o cpg' option or configured in hpe.conf.
+                            If FPG exists, be it a legacy FPG or Docker managed FPG, share is simply created under it.
+                            In case this option is not specified, then a default FPG is created with size 16TiB if it
+                            doesn't exist. Naming convention for default FPG is DockerFpg_n where n is an integer
+                            starting from 0.
+-o size=x                 x is the size of the share in GiB. By default, it is 1024 GiB.
+-o help -o filePersona    When used together, these options display this help content
+-o help=backends -o filePersona  When used together, these options display status of the backends configured for File Persona
+-o fsOwner=x              x is the user id and group id that should own the root directory  of nfs file share in the form of
+                            [userId:groupId]. Administrator also need to make sure that local user and local group with these
+                            ids are present on 3PAR before trying to mount the created share.
+                            For such shares which has userId and groupId specified, mount will succeed only if users and
+                            group with specified ids are present on 3PAR.
+-o fsMode=x               x is 1 to 4 octal digits that represent the file mode to be applied to the root directory of the
+                            file system. Ex: fsMode="0754" , Here 0 before number is mandatory. This ensures specified user
+                            of fsOwner will have rwx permissions, group will have rx permissions and others will have read
+                            permissions.
+                          x can also be ACL string. This also represents ACL permissions that are allowed on share directory.
+                            fsMode contains list of ACEs. Use Commas to separate ACEs. Each ACE here contains 3 values named,
+                            type, flag and permissions. These 3 values are separated by ':'. First ACE represents Owner,
+                            Second ACE represents Group and third ACE represents EveryOne. These has to be represented in
+                            order. Ex: A:fd:rwa,A:g:rwaxdnNcCoy,A:fdS:DtnNcy 
+                            type field can take only one of these values [A,D,U,L] 
+                            flag field can take one or more of these values [f,d,p,i,S,F,g]
+                            permissions field can take one or more of these values [r,w,a,x,d,D,t,T,n,N,c,C,o,y]
+                            Please refer 3PAR cli user guide more details on meaning of each flag.
+                          Note: For fsMode values user can specify either of mode bits or ACL string. Both can not be used 
+                            simultaneously. While using fsMode it is mandatory to specify fsOwner. If Only fsMode is used
+                            User will not be able to mount the share. This is because permissions and ownership changes are
+                            done during the first mount call. 

config/setupcfg.py

@@ -15,12 +15,17 @@
 from hpedockerplugin import configuration as conf
 from hpedockerplugin.hpe import hpe3par_opts as plugin_opts
 from oslo_log import log as logging
+import logging as log
 from oslo_config import cfg
+from logging.handlers import RotatingFileHandler
 
 host_opts = [
     cfg.StrOpt('hpedockerplugin_driver',
                default='hpe.hpe_lefthand_iscsi.HPELeftHandISCSIDriver',
                help='HPE Docker Plugin Driver to use for volume creation'),
+    cfg.StrOpt('mount_prefix',
+               default=None,
+               help='Mount prefix for volume mount'),
     cfg.StrOpt('host_etcd_ip_address',
                default='0.0.0.0',
                help='Host IP Address to use for etcd communication'),
@@ -56,8 +61,7 @@
     cfg.StrOpt('ssh_hosts_key_file',
                default='/root/.ssh/ssh_known_hosts',
                help='File containing SSH host keys for the systems with which '
-                    'the plugin needs to communicate.  OPTIONAL: '
-                    'Default=$state_path/ssh_known_hosts'),
+                    'the plugin needs to communicate'),
 ]
 
 CONF = cfg.CONF
@@ -70,6 +74,40 @@ def setup_logging(name, level):
     logging.setup(CONF, name)
     LOG = logging.getLogger(None)
 
+    # Add option to do Log Rotation
+    handler = RotatingFileHandler('/etc/hpedockerplugin/3pardcv.log',
+                                  maxBytes=10000000, backupCount=100)
+    formatter = log.Formatter('%(asctime)-12s [%(levelname)s] '
+                               '%(name)s [%(thread)d] '
+                               '%(threadName)s %(message)s')
+
+    handler.setFormatter(formatter)
+    LOG.logger.addHandler(handler)
+
+    if level == 'INFO':
+        LOG.logger.setLevel(logging.INFO)
+    if level == 'DEBUG':
+        LOG.logger.setLevel(logging.DEBUG)
+    if level == 'WARNING':
+        LOG.logger.setLevel(logging.WARNING)
+    if level == 'ERROR':
+        LOG.logger.setLevel(logging.ERROR)
+
+
+def setup_logging_for_ut(name, level):
+
+    logging.setup(CONF, name)
+    LOG = logging.getLogger(None)
+
+    # Add option to do Log Rotation
+    handler = log.FileHandler('./3pardcv.log')
+    formatter = log.Formatter('%(asctime)-12s [%(levelname)s] '
+                              '%(name)s [%(thread)d] '
+                              '%(threadName)s %(message)s')
+
+    handler.setFormatter(formatter)
+    LOG.logger.addHandler(handler)
+
     if level == 'INFO':
         LOG.logger.setLevel(logging.INFO)
     if level == 'DEBUG':
@@ -80,6 +118,12 @@ def setup_logging(name, level):
         LOG.logger.setLevel(logging.ERROR)
 
 
+def getdefaultconfig(configfile):
+    CONF(configfile, project='hpedockerplugin', version='1.0.0')
+    configuration = conf.Configuration(host_opts, config_group='DEFAULT')
+    return configuration
+
+
 def get_host_config(configfile):
     CONF(configfile, project='hpedockerplugin', version='1.0.0')
     return conf.Configuration(host_opts)

docs/active-passive-based-replication.md

@@ -0,0 +1,171 @@
+# Active/Passive Based Replication #
+
+In Active/Passive based replication, only one array is in active state 
+at any point of time serving the VLUNs of a given replicated volume.
+
+When a remote copy group (RCG) is failed over manually via 3PAR CLI to the
+secondary array, the secondary array becomes active. However, the VLUNs
+of the failed over volumes are still not exported by the secondary array
+to the host. In order to trigger that, the container/POD running on the 
+host needs to be restarted.
+
+## Configuring replication enabled backend
+**For FC Host** 
+```sh
+host_etcd_port_number=<ETCD_PORT_NUMBER>
+hpe3par_username=<Source-3PAR-Username>
+hpe3par_password=<Source-3PAR-Password>
+hpe3par_cpg=<Source-User-CPG>
+hpedockerplugin_driver=hpedockerplugin.hpe.hpe_3par_fc.HPE3PARFCDriver
+logging=DEBUG
+san_ip=<Source-3PAR-SAN-IP>
+san_login=<Source-3PAR-SAN-Username>
+san_password=<Source-3PAR-SAN-Password>
+host_etcd_ip_address=<IP1>[:PORT1[,IP2[:PORT2]][,IP3[:PORT3]]...]
+hpe3par_api_url=https://<Source-Array-IP>:8080/api/v1
+replication_device = backend_id:<Target-Array-Hostname>,
+                     replication_mode:<synchronous | asynchronous | streaming>,
+                     cpg_map:<Source-User-CPG>:<Target-User-CPG>,
+                     snap_cpg_map:<Source-Snap-CPG>:<Target-Snap-CPG>
+                     hpe3par_api_url:https://<Target-Array-IP>:8080/api/v1,
+                     hpe3par_username:<3PAR-Username>,
+                     hpe3par_password:<3PAR-Password>,
+                     san_ip:<3PAR-SAN-IP>,
+                     san_login:<3PAR-SAN-Username>,
+                     san_password:<3PAR-SAN-Password>
+```
+
+*Note*:
+
+1. In case of asynchronous replication mode, *sync_period* field can optionally be 
+defined as part of *replication_device* entry and it should be between range 300 
+and 31622400 seconds. If not defined, it defaults to 900 seconds.
+2. *cpg_map* and *snap_cpg_map* in *replication_device* section are mandatory. If *snap_cpg_map* is not mentioned then it will be same as *cpg_map*
+3. If password is encrypted for primary array, it must be encrypted for secondary array
+as well using the same *pass-phrase*
+
+
+**For ISCSI Host** 
+```sh
+host_etcd_port_number=<ETCD_PORT_NUMBER>
+hpe3par_username=<Source-3PAR-Username>
+hpe3par_password=<Source-3PAR-Password>
+hpe3par_cpg=<Source-User-CPG>
+hpedockerplugin_driver=hpedockerplugin.hpe.hpe_3par_iscsi.HPE3PARISCSIDriver
+logging=DEBUG
+san_ip=<Source-3PAR-SAN-IP>
+san_login=<Source-3PAR-SAN-Username>
+san_password=<Source-3PAR-SAN-Password>
+host_etcd_ip_address=<IP1>[:PORT1[,IP2[:PORT2]][,IP3[:PORT3]]...]
+hpe3par_api_url=https://<Source-Array-IP>:8080/api/v1
+hpe3par_iscsi_ips=<ISCSI_IP1>[,ISCSI_IP2,ISCSI_IP3...]
+replication_device=backend_id:<Source-Array-Host-Name>,
+replication_device = backend_id:<Target-Array-Name>,
+                     replication_mode:<synchronous | asynchronous | streaming>,
+                     cpg_map:<Source-User-CPG>:<Target-User-CPG>,
+                     snap_cpg_map:<Source-Snap-CPG>:<Target-Snap-CPG>
+                     hpe3par_api_url:https://<Target-Array-IP>:8080/api/v1,
+                     hpe3par_username:<3PAR-Username>,
+                     hpe3par_password:<3PAR-Password>,
+                     san_ip:<3PAR-SAN-IP>,
+                     san_login:<3PAR-SAN-Username>,
+                     san_password:<3PAR-SAN-Password>
+                     hpe3par_iscsi_ips=<ISCSI_IP1>[;ISCSI_IP2;ISCSI_IP3...]
+```
+*Note*:
+
+1. *cpg_map* and *snap_cpg_map* in *replication_device* section are mandatory. If *snap_cpg_map* is not mentioned then it will be same as *cpg_map*
+2. *hpe3par_iscsi_ips* MUST be defined upfront for both source and target arrays.
+3. *hpe3par_iscsi_ips* can be a single ISCSI IP or a list of ISCSI IPs delimited by 
+semi-colon. Delimiter for this field is applicable for *replication_device* section ONLY.
+4. If password is encrypted for primary array, it MUST be encrypted for secondary array
+as well using the same *pass-phrase*.
+5. In case of asynchronous replication mode, *sync_period* field can optionally be
+defined as part of *replication_device* entry and it should be between range 300 
+and 31622400 seconds. If not defined, it defaults to 900 seconds.
+
+
+## Managing Replicated Volumes ###
+### Create replicated volume ###
+This command allows creation of replicated volume along with RCG creation if the RCG
+does not exist on the array. Newly created volume is then added to the RCG.
+Existing RCG name can be used to add multiple newly created volumes to it.
+```sh
+$ docker volume create -d hpe --name <volume_name> -o replicationGroup=<3PAR_RCG_Name> [Options...]
+```
+where,
+- *replicationGroup*: Name of a new or existing replication copy group on 3PAR array
+
+One or more following *Options* can be specified additionally:
+1. *size:* Size of volume in GBs
+2. *provisioning:* Provision type of a volume to be created.
+Valid values are thin, dedup, full with thin as default.
+3. *backend:* Name of the backend to be used for creation of the volume. If not 
+specified, "DEFAULT" is used providied it is initialized successfully.
+4. *mountConflictDelay:* Waiting period in seconds to be used during mount operation
+of the volume being created. This happens when this volume is mounted on say Node1 and
+Node2 wants to mount it. In such a case, Node2 will wait for *mountConflictDelay* 
+seconds for Node1 to unmount the volume. If even after this wait, Node1 doesn't unmount
+the volume, then Node2 forcefully removes VLUNs exported to Node1 and the goes ahead 
+with the mount process.
+5. *compression:* This flag specifies if the volume is a compressed volume. Allowed 
+values are *True* and *False*.
+
+#### Example ####
+
+**Create a replicated volume having size 1GB with a non-existing RCG using backend "ActivePassiceRepBackend"**
+```sh
+$ docker volume create -d hpe --name Test_RCG_Vol -o replicationGroup=Test_RCG -o size=1 -o backend=ActivePassiceRepBackend 
+```
+This will create volume Test_RCG_Vol along with TEST_RCG remote copy group. The volume
+will then be added to the TEST_RCG.
+Please note that in case of failure during the operation at any stage, previous actions
+are rolled back.
+E.g. if for some reason, volume Test_RCG_Vol could not be added to Test_RCG, the volume
+is removed from the array.
+
+
+### Failover a remote copy group ###
+
+There is no single Docker command or option to support failover of a RCG. Instead, following 
+steps must be carried out in order to do it:
+1. On the host, the container using the replicated volume must be stopped or exited if it is running. 
+This triggers unmount of the volume(s) from the primary array.
+
+2. On the primary array, stop the remote copy group manually:
+```sh
+$ stoprcopygroup <RCG_Name>
+```
+
+3. On the secondary array, execute *failover* command:
+```sh
+$ setrcopygroup failover <RCG_Name_On_Secondary_Array>
+```
+
+4. Restart the container. This time the VLUNs would be served by the failed-over or secondary array
+
+### Failback workflow for Active/Passive based replication ###
+There is no single Docker command or option to support failback of a RCG. Instead, 
+following steps must be carried out in order to do it:
+1. On the host, the container using the replicated volume must be stopped or exited if it is running.
+This triggers unmount of the volume(s) from the failed-over or secondary array.
+
+2. On the secondary array, execute *recover* and *restore* commands:
+```sh
+$ setrcopygroup recover <RCG_Name_On_Secondary_Array>
+$ setrcopygroup restore <RCG_Name_On_Secondary_Array>
+```
+
+3. Restart the container so that the primary array exports VLUNs to the host this time.
+
+
+### Delete replicated volume ###
+```sh
+$ docker volume rm <volume_name>
+```
+This command allows the user to delete a replicated volume. If this was the last 
+volume present in RCG then the RCG is also removed from the backend.
+
+
+**See also:**
+[Peer Persistence Based Replication](peer-persistence-based-replication.md)