New Release v1.1.0 for OCI Service Broker

- Added Support for Oracle Streaming Service (OSS)
- Bug fixes

Co-authored-by: Shyam Radhakrishnan [email protected]
Co-authored-by: Ashokkumar Kannan [email protected]
Co-authored-by: Jayasheelan Kumar [email protected]
Co-authored-by: Naman Mehta [email protected]

CHANGELOG.md

@@ -5,6 +5,11 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 This project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+[1.1.0]
+
+- Adding support for Oracle Streaming Service (OSS)
+- Bug fixes
+
 [1.0.1]
 
 - Minor Document Fixes

README.md

@@ -14,6 +14,7 @@ The OCI Service Broker is an open source implementation of [Open service broker
 1. [Object Storage](https://docs.cloud.oracle.com/iaas/Content/Object/Concepts/objectstorageoverview.htm)
 1. [Autonomous Transaction Processing](https://www.oracle.com/in/database/autonomous-transaction-processing.html)
 1. [Autonomous Data Warehouse](https://www.oracle.com/in/database/data-warehouse.html)
+1. [Oracle Streaming Service](https://docs.cloud.oracle.com/iaas/Content/Streaming/Concepts/streamingoverview.htm)
 
 ## Installation
 
@@ -25,7 +26,11 @@ See the [Documentation](charts/oci-service-broker/README.md#oci-service-broker)
 
 ## Charts
 
-The OCI Service Broker is packaged as Helm chart for making it easy to install in Kubernetes Clusters. Please refer to [Documentation](#documentation) for detailed instructions.
+The OCI Service Broker is packaged as Helm chart for making it easy to install in Kubernetes Clusters. The chart can be downloaded from below URL.
+
+```
+https://github.com/oracle/oci-service-broker/releases/download/v<REPLACE_LATEST_VERSION>/oci-service-broker-<REPLACE_LATEST_VERSION>.tgz
+```
 
 ## Samples
 

charts/oci-service-broker/Chart.yaml

@@ -5,4 +5,4 @@
 apiVersion: v1
 description: A Helm chart for installing OCI Service Broker into a Kubernetes cluster
 name: oci-service-broker
-version: 1.0.1
+version: 1.1.0

charts/oci-service-broker/README.md

@@ -97,6 +97,17 @@
         - [Delete Service Binding](docs/adw.md#delete-service-binding)
         - [Delete Service Instance](docs/adw.md#delete-service-instance)
       - [Use Secret to pass passwords](docs/adw.md#use-secret-to-pass-passwords)
+  - [OracleStreamingService](docs/oss.md#oracle-streaming-oci-service-broker)
+    - [Introduction](docs/oss.md#introduction)
+    - [Plans](docs/oss.md#plans)
+    - [Service Provision Request Parameters](docs/oss.md#service-provision-request-parameters)
+    - [Service Binding](docs/oss.md#service-binding)
+      - [Request Parameters](docs/oss.md#request-parameters)
+      - [Response Credentials](docs/oss.md#response-credentials)
+    - [Example](docs/oss.md#example)
+      - [Kubernetes](docs/oss.md#kubernetes)
+        - [Provisioning](docs/oss.md#provisioning)
+        - [Binding](docs/oss.md#binding)
 
 ## Introduction  
 

charts/oci-service-broker/docs/installation.md

@@ -68,6 +68,10 @@ brew update && brew install kubernetes-service-catalog-client
 
 The OCI Service Broker is packaged as Helm chart for making it easy to install in Kubernetes. The chart is available at [charts/oci-service-broker](../) directory.
 
+```plain
+https://github.com/oracle/oci-service-broker/releases/download/v<REPLACE_LATEST_VERSION>/oci-service-broker-<REPLACE_LATEST_VERSION>.tgz
+```
+
 ### OCI credentials
 
 The OCI Service Broker needs OCI user credentials details to provision and manage services/resources in the user tenancy. The users required to create a Kubernetes secret as detailed below.
@@ -189,7 +193,8 @@ Replace the values of --set arguments with your appropriate values to install th
 
 ```bash
 helm install  charts/oci-service-broker/. --name oci-service-broker \
- --set ociCredentials.secretName=ocicredentials --set tls.secretName=certsecret \
+ --set ociCredentials.secretName=ocicredentials \
+ --set tls.secretName=certsecret \
  --set storage.etcd.servers=<comma separated list of etcd servers>
 ```
 
@@ -222,9 +227,10 @@ Refer [Restrict access to Service Catalog resources using RBAC](security.md#rest
 
 Sample files for various services are available under [`oci-service-broker/samples`](../samples) directory.
 
-Create a `ClusterServiceBroker` resource.
+Create a `ClusterServiceBroker` resource with OCI Service Broker URL to register the broker. Use the below register yaml file after updating the namespace of the OCI Service Broker.
 
 ```bash
+# Ensure <NAMESPACE_OF_OCI_SERVICE_BROKER> is replaced with the a proper namespace in oci-service-broker.yaml
 kubectl create -f oci-service-broker/samples/oci-service-broker.yaml
 ```
 
@@ -256,6 +262,7 @@ Output:
   atp-service                        Autonomous Transaction Processing Service
   object-store-service               Object Storage Service
   adw-service                        Autonomous Data Warehouse Service
+  oss-service                        Oracle Streaming Service
 ```
 
 Get Service Plans
@@ -274,6 +281,7 @@ Output:
   archive    object-store-service   An Archive type Object Storage
   standard   object-store-service   A Standard type Object Storage
   standard   adw-service            OCI Autonomous Data Warehouse
+  standard   oss-service            Oracle Streaming Service plan
 ```
 
 ## Monitoring OCI Service Broker
@@ -313,4 +321,4 @@ example:
   --set image.repository=<image name> --set image.tag=<image tag> \
   ...
   ...
- ```
+ ```

charts/oci-service-broker/docs/oss.md

@@ -0,0 +1,97 @@
+# Oracle Streaming OCI Service Broker
+
+- [Introduction](#introduction)
+- [Plans](#plans)
+- [OCI User Permission requirement](#oci-user-permission-requirement)
+- [Service Provision Request Parameters](#service-provision-request-parameters)
+- [Service Binding](#service-binding)
+  - [Request Parameters](#request-parameters)
+  - [Response Credentials](#response-credentials)
+- [Example](#example)
+  - [Kubernetes](#kubernetes)
+    - [Provisioning](#provisioning)
+    - [Binding](#binding)
+
+## Introduction
+
+OCI Streaming Service (OSS) provides a fully managed, scalable, and durable storage solution for ingesting continuous, high-volume streams of data that users can consume and process in real time. OSS service is also offered via OCI Service Broker thereby making it easy for applications to provision and integrate seamlessly with OSS.
+
+## Plans
+
+The supported plans for this service are
+
+1. standard
+
+## OCI User Permission requirement
+
+The OCI user for OCI Service Broker should have permission `manage` for resource type `streams`
+
+**Sample Policy:**
+
+```plain
+Allow group <SERVICE_BROKER_GROUP> to manage streams in compartment <COMPARTMENT_NAME>
+```
+
+## Service Provision Request Parameters
+
+The request parameters for Service provisioning are
+
+| Parameter     | Description                                                   | Type   | Mandatory |
+| ------------- | ------------------------------------------------------------- | ------ | --------- |
+| name          | The name of the stream                                        | string | Yes       |
+| compartmentId | The OCID of the compartment to which the stream should belong | string | Yes       |
+| partitions    | The number of partitions of the stream                        | number | Yes       |
+| freeFormTags  | The free form tags of the bucket                              | object | No        |
+| definedTags   | The defined tags of the bucket                                | object | No        |
+
+## Service Binding
+
+### Request Parameters
+
+The Service Binding Request does not have any parameters.
+
+### Response Credentials
+
+| Parameter | Description                                                                    | Type   |
+| --------- | ------------------------------------------------------------------------------ | ------ |
+| streamId  | The unique identifier of the stream, this can be used to connect to the stream | string |
+
+An OCI user credential can be used to connect to the stream using streamId. The binding request does not create the user.
+
+## Example
+
+### Kubernetes
+
+#### Provisioning
+
+Create a stream
+
+```yaml
+apiVersion: servicecatalog.k8s.io/v1beta1
+kind: ServiceInstance
+metadata:
+ name: "InstanceName"
+ namespace: "Namespace"
+spec:
+ clusterServiceClassExternalName: "oss-service"
+ clusterServicePlanExternalName: "standard"
+ parameters:
+   name: "StreamName"
+   compartmentId: "CompartmentOCID"
+   partitions: "5"
+```
+
+#### Binding
+
+Create a Request binding
+
+```yaml
+apiVersion: servicecatalog.k8s.io/v1beta1
+kind: ServiceBinding
+metadata:
+ name: "BindingName"
+ namespace: "Namespace"
+spec:
+ instanceRef:
+   name: "InstanceName"
+```

charts/oci-service-broker/docs/services.md

@@ -4,4 +4,5 @@ The OCI Service Brokers supports the following services
 
 1. [Object Storage Service](object-storage.md#object-storage-oci-service-broker).
 1. [Autonomous Transaction Processing](atp.md#autonomous-transaction-processing-service).
-1. [Autonomous Data Warehouse](adw.md#autonomous-data-warehouse-service).
+1. [Autonomous Data Warehouse](adw.md#autonomous-data-warehouse-service).
+1. [Oracle Streaming Service](oss.md#oracle-streaming-oci-service-broker).

charts/oci-service-broker/docs/troubleshoot.md

@@ -92,12 +92,18 @@ oci-service-broker-oci-service-broker-57b76b66f7-g269x   0/2     ContainerCreati
 ```
 
 **Solution:**
-Ensure the [OCI Credentials Kubernetes secret](installation.md#oci-credentials)  is created and passed correctly in the `helm` install command.
+Ensure the [OCI Credentials Kubernetes secret](installation.md#oci-credentials)  is created and passed correctly in the `helm` install command. Also, ensure that both the OCI credentials and OCI Service Broker are installed in the same namespace.
+
+Command to check the pod for OCI Credentials not found error:
+
+```bash
+kubectl -n <NAMESPACE_OF_OCI_SERVICE_BROKER> describe pod $(kubectl -n <NAMESPACE_OF_OCI_SERVICE_BROKER> get pods | grep 'oci-service-broker-' | cut -d" " -f1) | grep 'secret "ocicredentials" not found'
+```
 
 Command to check the secret:
 
 ```bash
-kubectl get secret ocicredentials -o yaml
+kubectl -n <NAMESPACE_OF_OCI_SERVICE_BROKER> get secret ocicredentials -o yaml
 ```
 
 In the helm install command the secret should have been passed as shown below:

charts/oci-service-broker/samples/adw/adw-demo.yaml

@@ -64,4 +64,4 @@ spec:
           secretName: adw-demo-binding
       # Shared Volume in which initContainer will save the decoded credential files and the db-app container reads.
       - name: creds
-        emptyDir: {}
+        emptyDir: {}

charts/oci-service-broker/samples/atp/atp-demo.yaml

@@ -64,4 +64,4 @@ spec:
           secretName: atp-demo-binding
       # Shared Volume in which initContainer will save the decoded credential files and the db-app container reads.
       - name: creds
-        emptyDir: {}
+        emptyDir: {}

charts/oci-service-broker/samples/oci-service-broker.yaml

@@ -7,4 +7,4 @@ kind: ClusterServiceBroker
 metadata:
   name: oci-service-broker
 spec:
-  url: http://oci-service-broker:8080
+  url: http://oci-service-broker.<NAMESPACE_OF_OCI_SERVICE_BROKER>:8080

charts/oci-service-broker/samples/oss/create-oss-binding.yaml

@@ -0,0 +1,11 @@
+#
+# Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
+#
+apiVersion: servicecatalog.k8s.io/v1beta1
+kind: ServiceBinding
+metadata:
+ name: test-stream-binding
+spec:
+ instanceRef:
+   name: teststream

charts/oci-service-broker/samples/oss/create-oss-instance.yaml

@@ -0,0 +1,15 @@
+#
+# Copyright (c) 2019, Oracle and/or its affiliates. All rights reserved.
+# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
+#
+apiVersion: servicecatalog.k8s.io/v1beta1
+kind: ServiceInstance
+metadata:
+ name: teststream
+spec:
+ clusterServiceClassExternalName: oss-service
+ clusterServicePlanExternalName: standard
+ parameters:
+   name: teststream
+   compartmentId: CHANGE_COMPARTMENT_OCID_HERE
+   partitions: CHANGE_PARTITION_COUNT_HERE

charts/oci-service-broker/values.yaml

@@ -14,7 +14,7 @@ image:
   repository: iad.ocir.io/oracle/oci-service-broker
 
   # Tag of the image
-  tag:  1.0.1
+  tag:  1.1.0
 
   # The image pull policy
   pullPolicy: Always
@@ -75,5 +75,5 @@ tls:
 
 # Log Levels are configured below
 logLevel:
-  ociSDK: DEBUG
-  broker: DEBUG
+  ociSDK: INFO
+  broker: INFO

oci-service-broker/build.gradle

@@ -30,17 +30,18 @@ apply plugin: 'maven-publish'
 archivesBaseName = 'oci-service-broker'
 
 // Sometimes, the version has to be overridden from command line
-version = project.hasProperty('version_num') ? project.getProperty('version_num') : '1.0.1'
+version = project.hasProperty('version_num') ? project.getProperty('version_num') : '1.1.0'
+ext.dockerGroup = 'iad.ocir.io/oci-cnp-dev'
 mainClassName = 'com.oracle.oci.osb.Broker'
 
 sourceCompatibility = 10
 
 ext {
-    ociSdkVersion = "1.3.1-SNAPSHOT"
-    jerseyVersion = "2.24.1"
-    hk2Version = "2.5.0-b05"
+    ociSdkVersion = "1.5.4"
+    jerseyVersion = "2.27"
+    hk2Version = "2.5.0-b42"
     hamcrestVersion = "1.3"
-    jacksonVersion = "2.9.7";
+    jacksonVersion = "2.9.8";
     jacksonDatabindVersion = "2.9.8";
     javaxwsrsVersion = "2.1.1"
     bedrockVersion = "5.0.7"
@@ -71,7 +72,7 @@ dependencies {
     //oci-java-sdk is not published to any public maven repo yet. In order to build the the project users are required
     //to download oci-java-sdk and the dependent libraries to libs directory. Please run script download_SDK_libs.sh
     //before build.
-    compile fileTree(dir: 'libs', include: ['*.jar'])
+    compile fileTree(dir: 'libs', include: ['*.jar'], exclude: ['animal-sniffer-annotations-1.14.jar','guava-25.0-android.jar','javax.ws.rs-api-2.1.jar','checker-compat-qual-2.0.0.jar','error_prone_annotations-2.1.3.jar'])
 
     compile("com.fasterxml.jackson.core:jackson-annotations:$jacksonVersion");
     compile("com.fasterxml.jackson.core:jackson-core:$jacksonVersion");

oci-service-broker/deploy/docker/Dockerfile

@@ -20,4 +20,4 @@ RUN groupadd -g 999 svcbroker && \
     useradd -r -u 999 -g svcbroker svcbroker
 RUN chown -R svcbroker:svcbroker /oci-service-broker
 USER svcbroker
-ENTRYPOINT ["/oci-service-broker/start-broker.sh"]
+ENTRYPOINT ["/oci-service-broker/start-broker.sh"]

oci-service-broker/download_SDK_libs.sh

@@ -9,15 +9,15 @@
 #sdk jars and their dependency jars. The jars are written to libs directory.
 
 SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
-SDK_VERSION="1.3.1"
+SDK_VERSION="1.5.4"
 TEMP_DIR="/tmp/oci-java-sdk"
 rm -rf ${TEMP_DIR}
 mkdir -p ${TEMP_DIR} 
 mkdir -p ${SCRIPT_DIR}/libs
 echo "Downloading oci-java-sdk version v${SDK_VERSION} and the dependent libraries..."
 curl -sSL https://github.com/oracle/oci-java-sdk/releases/download/v${SDK_VERSION}/oci-java-sdk.zip -o ${TEMP_DIR}/oci-java-sdk.zip
 unzip -qq ${TEMP_DIR}/oci-java-sdk.zip -d ${TEMP_DIR}
-cp ${TEMP_DIR}/lib/oci-java-sdk-full-1.3.1.jar ${SCRIPT_DIR}/libs/
+cp ${TEMP_DIR}/lib/oci-java-sdk-full-1.5.4.jar ${SCRIPT_DIR}/libs/
 cp ${TEMP_DIR}/third-party/lib/*.jar  ${SCRIPT_DIR}/libs/
 rm -rf ${TEMP_DIR}
 echo "oci-java-sdk and the dependent libraries are downloaded to ${SCRIPT_DIR}/libs directory"