From b5b5ec250d5dd8a1f7df9790b37089fda97ed9d9 Mon Sep 17 00:00:00 2001 From: Alex Heneveld Date: Tue, 20 Jun 2017 12:20:54 +0100 Subject: [PATCH 1/4] updated versioning rules as per ML proposal --- guide/blueprints/catalog/versioning.md | 124 ++++++++++++++++++++++--- 1 file changed, 112 insertions(+), 12 deletions(-) diff --git a/guide/blueprints/catalog/versioning.md b/guide/blueprints/catalog/versioning.md index 3bec946e..14bd2e43 100644 --- a/guide/blueprints/catalog/versioning.md +++ b/guide/blueprints/catalog/versioning.md @@ -3,18 +3,118 @@ title: Versioning layout: website-normal --- -### Versioning +Brooklyn supports multiple versions of a type to be installed and used at the same time. +Versions are a first-class concept and are often prominently displayed in the UI. -Version numbers follow the OSGi convention. This can have a major, minor, micro and qualifier part. -For example, `1.0`. `1.0.1` or `1.0.1-20150101`. +In order to do this, Brooklyn requires that the `id:version` string be unique across the catalog: +it is normally an error to add a type if a type with the same `id:version` is present. +Exceptions are if the definition is identical, or if the `version` is noted as a `SNAPSHOT`. +In extraordinary circumstances it is possible to delete a given `id:version` definition +and then add the new one, but this is discouraged and the usual practice is to: -The combination of `id:version` strings must be unique across the catalog. -It is an error to deploy the same version of an existing item: -to update a blueprint, it is recommended to increase its version number; -alternatively in some cases it is permitted to delete an `id:version` instance -and then re-deploy. -If no version is specified, re-deploying will automatically -increment an internal version number for the catalog item. +* Use a `-SNAPSHOT` qualifer suffix on your version when developing +* Increase the version number when making a change to a non-SNAPSHOT type + +When adding to the catalog, if no version is supplied, Brooklyn may automatically +increment the version number for the catalog item. + +When deploying a blueprint, if a version number is not specified Brooklyn will typically use +the highest ordered version (see "Ordering" below) in the catalog for the referenced type, +and will thereafter lock the use of that version in that blueprint. +(An exception is where types are co-bundled or an explicit search path is defined; +in the context of evaluating one type, Brooklyn may prefer versions in the same bundle or on the search path.) + + +#### Versioning Syntax + +Version numbers in Brooklyn are recommended to follow the following syntax: + +``` + ( "." ( "." )? )? ( "-" )? +``` + +where the ``, ``, and `` parts are numbers +in accordance with [semver](http://semver.org) semantic versioning, +assumed to be `0` if omitted, +and an `` is made up of letters, numbers, `"-"` and `"_"` +in accordance with [OSGi](https://www.osgi.org/release-4-version-4-3-download/) +(see sections 1.3.2 and 3.2.5). + +Examples: + +* `1.2` +* `2.0.0` +* `3` +* `2.0.0-SNAPSHOT` +* `1.10-rc3-20170619` + + +#### Snapshots and Ordering + +The string `SNAPSHOT` appearing anywhere in the version indicates a pre-release version; +if this string is not present the version is treated as a release version. + +When taking an ordering, for instance to find the highest version, +snapshot versions are always considered lower than release versions. +Next, the natural order is taken on the major, minor, and patch fields. +Next, a version with no qualifier is considered higher than one with a qualifier. +Finally, the qualifier is taken in natural order. + +The natural order here is defined as ASCII-lexicographic comparison +for any non-numeric segments (`"a" < "b"`) but numeric order for digit sequences +(`"9" < "10"`), so it does what is usually expected for versions (`1.9` < `1.10` +and `"1.1-rc9-b" < "1.1-rc10-a"`). + +Thus the _order_ of the list of examples above is: + +* `2.0.0-SNAPSHOT` +* `1.2` +* `1.10-rc3-20170619` +* `2.0.0` +* `3` + + +#### Advanced: Other Version Syntaxes + +Other version syntaxes are supported with the following restrictions: + +* Version strings MUST NOT contain a colon character (`:`) +* Version strings MUST NOT be empty +* Fragments that do not follow the recommended syntax may be ignored + when determining version uniqueness + (e.g. adding both `"1.0.0-v1.1"` and "1.0.0-v1_1" can result in + one bundle _replacing_ the other rather than both versions being loaded) + +This means in practice that almost any common version scheme can be used. +However the recommended scheme will fit more neatly alongside types from other sources. + +Internally in some places, Brooklyn needs to produce OSGi-compliant versions. +For the recommended syntax, this mapping consists of replacing the first +occurrence of `"-"` with `"."` and setting `0` values for absent minor and patch versions. +Thus when looking at the OSGi view, instead of version `1.10-rc3-20170619` +you will see `1.10.0.rc3-20170619`. +This mapping is guaranteed to be one-to-one so no conflicts will occur if the +recommended syntax is used. +(If not using the recommended syntax, the mapping proceeds by treating the first dot-separated fragment +as the qualifer and converts unsupported characters in a qualifier to an underscore; +thus `1.x` becomes `1.0.0.x`, `v1` becomes `0.0.0.v1`, and `"1.0.0-v1.1"` becomes `"1.0.0.v1_1"` +hence the bundle replacement noted above.) + +If you are creating an OSGi `MANIFEST.MF` for a bundle that also contains a `catalog.bom`, +you will need to use the mapped rsult (OSGi version syntax) in the manifest, +but should continue to use the Brooklyn-recommended syntax in the `catalog.bom`. + +For those who are curious, the reason for the Brooklyn version syntax is to reconcile +the popular usage of semver and maven with the internal requirement to use OSGi versions. +Semver, OSGi, and maven conventions agree on up to three numeric dot-separated tokens, +but differ quite significantly afterwards, with Brooklyn adopting what seems to be the +most popular choices in each. + +A summary of the main differences between Brooklyn and other versioning syntaxes is as follows: + +* `SNAPSHOT` treated specially (maven semantics) +* Qualifier preceded by hyphen (maven and semver semantics, different to OSGi which wants a dot) +* Underscores allowed in qualifiers (OSGi and maven semantics, different to semver) +* Periods and plus not allowed in qualifiers (OSGi semantics and maven convention, + different to semver which gives them special meaning) -When referencing a blueprint, if a version number is not specified -the latest non-snapshot version will be loaded when an entity is instantiated. \ No newline at end of file From 80debad88f823f70d131be249bfbdb8a056f326a Mon Sep 17 00:00:00 2001 From: Alex Heneveld Date: Tue, 20 Jun 2017 12:21:15 +0100 Subject: [PATCH 2/4] misc doc changes --- guide/blueprints/catalog/bundle.md | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/guide/blueprints/catalog/bundle.md b/guide/blueprints/catalog/bundle.md index 3f2b899b..78367cf7 100644 --- a/guide/blueprints/catalog/bundle.md +++ b/guide/blueprints/catalog/bundle.md @@ -5,7 +5,25 @@ layout: website-normal ### Bundling Catalog Resources -It is possible to add an OSGi bundle to AMP. This is useful when you have a blueprint that needs to reference external scripts/resources or when you have multiple blueprints that you want to keep in sync. Brooklyn will persist any uploaded bundles so that they are available after a restart, or a HA failover. +It is possible to upload catalog items and associated resources as a single bundle to AMP. +This is useful when you have a blueprint that needs to reference external scripts/resources or +when you have multiple blueprints that you want to keep in sync. Brooklyn will persist any +uploaded bundles so that they are available after a restart or on HA failover. + +The bundle must be a ZIP file including a `catalog.bom` in the root. +(The `br` CLI will create a ZIP from a local folder, for convenience.) +The `catalog.bom` must declare a `bundle` identifier and a `version`, +following Brooklyn's [versioning](versioning.html) rules. +Brooklyn will keep track of that bundle, allow SNAPSHOT-version bundles to be replaced, +and ensure dependent bundles (specified as `brooklyn.libraries` or, for people familiar +with OSGi, the `Required-bundle` manifest header) are available for searching. + +Resources in the bundle can be referenced from the `catalog.bom` by using +the `classpath:` URL protocol, as in `classpath://path/to/script.sh`. +This can also be used to load resources in explicitly declared dependent bundles. + + +### Example In this example, we will create a simple `my-server` catalog item, bundled with a simple script. The script will be run when launching the server. From 0b106a3425db529bc2f7c56788d838059aa598cc Mon Sep 17 00:00:00 2001 From: Alex Heneveld Date: Fri, 23 Jun 2017 12:44:05 +0100 Subject: [PATCH 3/4] PR comments on versioning and note on `0` implicit minor/patch --- guide/blueprints/catalog/bundle.md | 12 ++++++++---- guide/blueprints/catalog/versioning.md | 26 +++++++++++++++++++------- 2 files changed, 27 insertions(+), 11 deletions(-) diff --git a/guide/blueprints/catalog/bundle.md b/guide/blueprints/catalog/bundle.md index 78367cf7..964dee0b 100644 --- a/guide/blueprints/catalog/bundle.md +++ b/guide/blueprints/catalog/bundle.md @@ -6,7 +6,8 @@ layout: website-normal ### Bundling Catalog Resources It is possible to upload catalog items and associated resources as a single bundle to AMP. -This is useful when you have a blueprint that needs to reference external scripts/resources or +This is useful when you have a blueprint that needs to reference external scripts, icons, +config files or other resources, or when you have multiple blueprints that you want to keep in sync. Brooklyn will persist any uploaded bundles so that they are available after a restart or on HA failover. @@ -14,9 +15,12 @@ The bundle must be a ZIP file including a `catalog.bom` in the root. (The `br` CLI will create a ZIP from a local folder, for convenience.) The `catalog.bom` must declare a `bundle` identifier and a `version`, following Brooklyn's [versioning](versioning.html) rules. -Brooklyn will keep track of that bundle, allow SNAPSHOT-version bundles to be replaced, -and ensure dependent bundles (specified as `brooklyn.libraries` or, for people familiar -with OSGi, the `Required-bundle` manifest header) are available for searching. +Brooklyn will keep track of that bundle, allowing items to be added and removed as a group, +and associated resources to be versioned and included alongside them. +With SNAPSHOT-version bundles, it allows replacement of multiple related items at the same time, +and in advanced cases it allows setting up dependent bundles +(specified as `brooklyn.libraries` or, for people familiar with OSGi, the `Required-bundle` manifest header) +which will be searched if a blueprint in one bundle references resources from another bundle. Resources in the bundle can be referenced from the `catalog.bom` by using the `classpath:` URL protocol, as in `classpath://path/to/script.sh`. diff --git a/guide/blueprints/catalog/versioning.md b/guide/blueprints/catalog/versioning.md index 14bd2e43..5077b79e 100644 --- a/guide/blueprints/catalog/versioning.md +++ b/guide/blueprints/catalog/versioning.md @@ -8,8 +8,8 @@ Versions are a first-class concept and are often prominently displayed in the UI In order to do this, Brooklyn requires that the `id:version` string be unique across the catalog: it is normally an error to add a type if a type with the same `id:version` is present. -Exceptions are if the definition is identical, or if the `version` is noted as a `SNAPSHOT`. -In extraordinary circumstances it is possible to delete a given `id:version` definition +The exceptions to this are if the definition is identical, or if the `version` is noted as a `SNAPSHOT`. +In extraordinary circumstances it may be appropriate to delete a given `id:version` definition and then add the new one, but this is discouraged and the usual practice is to: * Use a `-SNAPSHOT` qualifer suffix on your version when developing @@ -73,6 +73,11 @@ Thus the _order_ of the list of examples above is: * `2.0.0` * `3` +For practical purposes, `3`, `3.0`, and `3.0.0` are treated as equivalent, +but if referencing a version you should use the exact version string defined. +The version `3.0-0` is different, as the `-0` indicates a qualifier, and +is ordered before a `3.0.0`. + #### Advanced: Other Version Syntaxes @@ -88,20 +93,27 @@ Other version syntaxes are supported with the following restrictions: This means in practice that almost any common version scheme can be used. However the recommended scheme will fit more neatly alongside types from other sources. -Internally in some places, Brooklyn needs to produce OSGi-compliant versions. +Internally when installing bundles, Brooklyn needs to produce OSGi-compliant versions. For the recommended syntax, this mapping consists of replacing the first occurrence of `"-"` with `"."` and setting `0` values for absent minor and patch versions. Thus when looking at the OSGi view, instead of version `1.10-rc3-20170619` you will see `1.10.0.rc3-20170619`. -This mapping is guaranteed to be one-to-one so no conflicts will occur if the +Apart from the omission of `0` as minor and patch versions, +this mapping is guaranteed to be one-to-one so no conflicts will occur if the recommended syntax is used. -(If not using the recommended syntax, the mapping proceeds by treating the first dot-separated fragment +Bundles `foo:3`, `foo:3.0`, and `foo:3.0.0` would all be installed using OSGi version `3.0.0`, +and so would conflict and block installation if there is any change +(and replace if they have a `-SNAPSHOT` qualifier); +references to bundles can use `3` or `3.0` or `3.0.0`, though as noted above +types contained within would have to be referenced using the exact version string supplied. + +If not using the recommended syntax, the mapping proceeds by treating the first dot-separated fragment as the qualifer and converts unsupported characters in a qualifier to an underscore; thus `1.x` becomes `1.0.0.x`, `v1` becomes `0.0.0.v1`, and `"1.0.0-v1.1"` becomes `"1.0.0.v1_1"` -hence the bundle replacement noted above.) +hence the bundle replacement noted above. If you are creating an OSGi `MANIFEST.MF` for a bundle that also contains a `catalog.bom`, -you will need to use the mapped rsult (OSGi version syntax) in the manifest, +you will need to use the mapped result (OSGi version syntax) in the manifest, but should continue to use the Brooklyn-recommended syntax in the `catalog.bom`. For those who are curious, the reason for the Brooklyn version syntax is to reconcile From 939e2d4dfdbcd88685168c3ae6c781e595f40212 Mon Sep 17 00:00:00 2001 From: Alex Heneveld Date: Wed, 28 Jun 2017 16:04:04 +0100 Subject: [PATCH 4/4] address PR comments, and minor tidies --- guide/blueprints/catalog/versioning.md | 34 ++++++++++++++++++-------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/guide/blueprints/catalog/versioning.md b/guide/blueprints/catalog/versioning.md index 5077b79e..9e92eb80 100644 --- a/guide/blueprints/catalog/versioning.md +++ b/guide/blueprints/catalog/versioning.md @@ -15,14 +15,16 @@ and then add the new one, but this is discouraged and the usual practice is to: * Use a `-SNAPSHOT` qualifer suffix on your version when developing * Increase the version number when making a change to a non-SNAPSHOT type -When adding to the catalog, if no version is supplied, Brooklyn may automatically -increment the version number for the catalog item. +When adding to the catalog, if no version is supplied, Brooklyn will typically use +`0.0.0-SNAPSHOT`, and some clients will automatically create and increment the version number +for the catalog item. When deploying a blueprint, if a version number is not specified Brooklyn will typically use the highest ordered version (see "Ordering" below) in the catalog for the referenced type, and will thereafter lock the use of that version in that blueprint. (An exception is where types are co-bundled or an explicit search path is defined; -in the context of evaluating one type, Brooklyn may prefer versions in the same bundle or on the search path.) +in the context of evaluating one type, Brooklyn may prefer versions in the same bundle +or on the search path.) #### Versioning Syntax @@ -60,10 +62,11 @@ Next, the natural order is taken on the major, minor, and patch fields. Next, a version with no qualifier is considered higher than one with a qualifier. Finally, the qualifier is taken in natural order. -The natural order here is defined as ASCII-lexicographic comparison -for any non-numeric segments (`"a" < "b"`) but numeric order for digit sequences -(`"9" < "10"`), so it does what is usually expected for versions (`1.9` < `1.10` -and `"1.1-rc9-b" < "1.1-rc10-a"`). +The natural order here is defined as +numeric order for digit sequences (`"9" < "10"`) +and ASCII-lexicographic comparison elsewhere (`"a" < "b"`), +which is normally what people will expect for versions +(`1.9` < `1.10` and `"1.1-rc9-b" < "1.1-rc10-a"`). Thus the _order_ of the list of examples above is: @@ -73,7 +76,7 @@ Thus the _order_ of the list of examples above is: * `2.0.0` * `3` -For practical purposes, `3`, `3.0`, and `3.0.0` are treated as equivalent, +For most practical purposes, `3`, `3.0`, and `3.0.0` are treated as equivalent, but if referencing a version you should use the exact version string defined. The version `3.0-0` is different, as the `-0` indicates a qualifier, and is ordered before a `3.0.0`. @@ -106,7 +109,13 @@ and so would conflict and block installation if there is any change (and replace if they have a `-SNAPSHOT` qualifier); references to bundles can use `3` or `3.0` or `3.0.0`, though as noted above types contained within would have to be referenced using the exact version string supplied. - +(If different versions are specified on individual types than for the bundle itself -- +which is not recommended -- then the conversion to OSGi does not apply, +and the versions are not treated as equal; +in such edge cases the ordering obeys numeric then ASCII ordering on segments, +so we have `3` < `3.0` < `3.01` < `3.1` < `3.09` < `3.9` < `3.10` +and `v-1` < `v.1` < `v_1`.) + If not using the recommended syntax, the mapping proceeds by treating the first dot-separated fragment as the qualifer and converts unsupported characters in a qualifier to an underscore; thus `1.x` becomes `1.0.0.x`, `v1` becomes `0.0.0.v1`, and `"1.0.0-v1.1"` becomes `"1.0.0.v1_1"` @@ -124,9 +133,14 @@ most popular choices in each. A summary of the main differences between Brooklyn and other versioning syntaxes is as follows: -* `SNAPSHOT` treated specially (maven semantics) * Qualifier preceded by hyphen (maven and semver semantics, different to OSGi which wants a dot) * Underscores allowed in qualifiers (OSGi and maven semantics, different to semver) * Periods and plus not allowed in qualifiers (OSGi semantics and maven convention, different to semver which gives them special meaning) +* The ordering used in Brooklyn is different to that used in OSGi + (where qualifiers come after the unqualified version and don't do a numeric comparison) +* `SNAPSHOT` treated specially (maven semantics) +* Maven's internal to-OSGi conversion is different for some non-recommended syntax strings + (e.g. `10rc1` becomes `10.0.0.rc1` in Brooklyn but Maven will map it by default to `0.0.0.10rc1`) +