Friday, 19 January 2018

IBM BPM - Tinkering with Process Application Exports and Imports

In the context of an IBM BPM migration document, I'm just making a few notes about the export of a BPM Process Application snapshot from a Process Center, ready for an offline deployment to a different ( and unconnected ) Process Server.

Which is nice :-)

On the source Process Center

Connect to the AppCluster SOAP Endpoint

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -port 8880 -user wasadmin -password passw0rd

WASX7209I: Connected to process "AppClusterMember1" on node Node1 using SOAP connector;  The type of process is: ManagedProcess
WASX7031I: For help, enter: "print Help.help()"

List Available Process Applications

print AdminTask.BPMListProcessApplications()

Containers: 
Name: System Data
Acronym: TWSYS
Description: IBM BPM System Data
Toolkit: true

Name: System Governance
Acronym: TWSYSG
Description: IBM BPM Governance Data and Services
Toolkit: true

Name: Coaches
Acronym: SYSC
Description: IBM BPM System Coaches
Toolkit: true

Name: Content Management
Acronym: SYSCM
Description: IBM BPM Content Management Data, Services and Coach views.
Toolkit: true

Name: Dashboards
Acronym: SYSD
Description: IBM BPM System Dashboards
Toolkit: true

Name: SAP Guided Workflow
Acronym: SGW
Description: Toolkit of SAP Guided Workflow
Toolkit: true

Name: Process Portal
Acronym: TWP
Description: 
Toolkit: false

Name: Saved Search Admin
Acronym: SSA
Description: Dashboard for creating and managing Saved Searches
Toolkit: false

Name: Hiring Sample
Acronym: HSS
Description: Hiring Sample
Toolkit: false

Name: Hiring Sample Advanced
Acronym: HSAV1
Description: 
Toolkit: false

Name: Procurement Sample
Acronym: STPPS1
Description: 
Toolkit: false

Drill into the details of the target PA

print AdminTask.BPMShowProcessApplication('[-containerAcronym HSS]')

Name: Hiring Sample
Acronym: HSS
Description: Hiring Sample
Toolkit: false
Tracks:

Track Name: Main
Track Acronym: Main
Default: true

Tip: 
Created On: 2018-01-19 16:29:26.083
Created By: User.9
State: State[Inactive]
Capability: Capability[Standard]
No of running instances: 0

List of Snapshots: 
Name: Standard Hiring Sample v8550
Acronym: SHSV855
Created On: 2018-01-19 16:29:26.083
Created By: User.9
Is Default: false
State: State[Inactive]
Capability: Capability[Standard]
No of running instances: 0


List the Process Servers ( the subsequent command requires a server name, even if it's an unconnected Process Server )

print AdminTask.BPMListServers()

Servers:
Name: DMH
URL: jnp://XXXXX:XXXX

Create an Offline Package, ready for export

AdminTask.BPMCreateOfflinePackage('[-containerAcronym HSS -containerSnapshotAcronym SHSV855 -containerTrackAcronym Main -serverName DMH]')

'BPMCreateOfflinePackage passed.'

Export the Process Application snapshot

AdminTask.BPMExtractOfflinePackage('[-containerAcronym HSS -containerSnapshotAcronym SHSV855 -containerTrackAcronym Main -serverName DMH -outputFile /mnt/Apps/StandardHiringSamplev8550.zip]')

'BPMExtractOfflinePackage passed.'

Validate the export

ls -al

total 5368
drwxr-xr-x   4 davidhay  staff      136 19 Jan 17:46 .
drwxr-xr-x  22 davidhay  staff      748  9 Jan 16:36 ..
-rw-r--r--   1 davidhay  staff  1243529  3 Nov 19:22 BAMTST_V4.zip
-rw-r--r--   1 davidhay  staff  1501656 19 Jan 17:46 StandardHiringSamplev8550.zip

On the target Process Server

Connect to the AppCluster SOAP Endpoint

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -port 8880 -user wasadmin -password passw0rd

WASX7209I: Connected to process "AppClusterMember1" on node Node1 using SOAP connector;  The type of process is: ManagedProcess
WASX7031I: For help, enter: "print Help.help()"

AdminTask.BPMInstallOfflinePackage('[-inputFile /mnt/Apps/StandardHiringSamplev8550.zip]')

'BPMInstallOfflinePackage passed.'

List Available Process Applications

print AdminTask.BPMListProcessApplications()

Containers: 
Name: System Data
Acronym: TWSYS
Description: IBM BPM System Data
Toolkit: true

Name: System Governance
Acronym: TWSYSG
Description: IBM BPM Governance Data and Services
Toolkit: true

Name: Coaches
Acronym: SYSC
Description: IBM BPM System Coaches
Toolkit: true

Name: Content Management
Acronym: SYSCM
Description: IBM BPM Content Management Data, Services and Coach views.
Toolkit: true

Name: Dashboards
Acronym: SYSD
Description: IBM BPM System Dashboards
Toolkit: true

Name: SAP Guided Workflow
Acronym: SGW
Description: Toolkit of SAP Guided Workflow
Toolkit: true

Name: Process Portal
Acronym: TWP
Description: 
Toolkit: false

Name: Saved Search Admin
Acronym: SSA
Description: Dashboard for creating and managing Saved Searches
Toolkit: false

Name: Hiring Sample
Acronym: HSS
Description: 
Toolkit: false


print AdminTask.BPMShowProcessApplication('[-containerAcronym HSS]')

Name: Hiring Sample
Acronym: HSS
Description: 
Toolkit: false
Tracks:

List of Snapshots: 
Name: Standard Hiring Sample v8550
Acronym: SHSV855
Created On: 2018-01-19 20:44:18.822
Created By: User.1
Is Default: true
State: State[Active]
Capability: Capability[Standard]
No of running instances: 0

In addition, validatory messages can be seen in AppCluster SystemOut.log e.g.

[19/01/18 20:44:16:768 GMT] 00000195 PALAdminComma I PALAdminCommands processServerOfflineDeploy Entering
[19/01/18 20:44:18:192 GMT] 00000195 wle           I   CWLLG3547I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The snapshot with ID Snapshot.aaa569e9-a57a-4684-b357-34237e4e49ea is being installed. The following toolkits are being installed: []
[19/01/18 20:44:18:212 GMT] 00000195 wle           I   CWLLG3533I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The snapshot is being imported.
[19/01/18 20:44:24:178 GMT] 00000195 wle           I   CWLLG3534I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The snapshot was successfully imported.
[19/01/18 20:44:24:178 GMT] 00000195 wle           I   CWLLG3536I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The installation service for the snapshot is being started.
[19/01/18 20:44:24:197 GMT] 0000019a JMSClassifica W   XJMS0021E: Unable to map destination and bus to workclass. Aborting the xd classification filter for the current request. Request context information is com.ibm.ws.xd.jms.proxy.impl.AsynchJMSServiceContextImpl@5700490c<bus=BPM.De1.Bus,destination=PortalWebMessagingTopicSpace.AppCluster>.
[19/01/18 20:44:24:231 GMT] 0000019b JMSClassifica W   XJMS0021E: Unable to map destination and bus to workclass. Aborting the xd classification filter for the current request. Request context information is com.ibm.ws.xd.jms.proxy.impl.AsynchJMSServiceContextImpl@c5fba99b<bus=BPM.De1.Bus,destination=cacheTopicDestination.AppCluster>.
[19/01/18 20:44:24:236 GMT] 0000019c JMSClassifica W   XJMS0021E: Unable to map destination and bus to workclass. Aborting the xd classification filter for the current request. Request context information is com.ibm.ws.xd.jms.proxy.impl.AsynchJMSServiceContextImpl@557e4439<bus=BPM.De1.Bus,destination=TWServerTopicDestination.AppCluster>.
[19/01/18 20:44:25:285 GMT] 00000195 wle           I   CWLLG3537I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The installation service for the snapshot finished successfully.
[19/01/18 20:44:25:286 GMT] 00000195 wle           I   CWLLG3539I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The data for the snapshot is being migrated.
[19/01/18 20:44:25:408 GMT] 00000195 wle           I   CWLLG3540I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The data for the snapshot was migrated successfully.
[19/01/18 20:44:25:409 GMT] 00000195 wle           I   CWLLG3541I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The instances of the snapshot are being migrated.
[19/01/18 20:44:25:436 GMT] 00000195 wle           I   CWLLG3542I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The instances of the snapshot were migrated successfully.
[19/01/18 20:44:30:683 GMT] 00000195 wle           I   CWLLG3545I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The tracking definitions for the snapshot were updated successfully.
[19/01/18 20:44:30:710 GMT] 00000195 wle_eventmgr  I   CWLLG3550I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The scheduled UCAs for the snapshot are being activated.
[19/01/18 20:44:30:880 GMT] 00000195 wle_eventmgr  I   CWLLG3551I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The scheduled UCAs for the snapshot were activated successfully.
[19/01/18 20:44:30:884 GMT] 00000195 wle_eventmgr  I   CWLLG3550I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The scheduled UCAs for the snapshot are being activated.
[19/01/18 20:44:30:884 GMT] 00000195 wle_eventmgr  I   CWLLG3551I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The scheduled UCAs for the snapshot were activated successfully.
[19/01/18 20:44:31:205 GMT] 00000195 wle           I   CWLLG3548I: Process Application: Hiring Sample, Snapshot: Standard Hiring Sample v8550 : The snapshot was installed successfully.
[19/01/18 20:44:31:243 GMT] 00000195 PALAdminComma I PALAdminCommands processServerOfflineDeploy Exiting




Thursday, 18 January 2018

Docker on Linux - Scripting the removal of images

I'm again going through the process to deploy IBM Cloud Private (ICP) on Ubuntu.

However, I forgot to check the available disk space …..

So, when I ran this command: -

tar xf /tmp/ibm-cloud-private-x86_64-2.1.0.1.tar.gz -O | sudo docker load

to expand the ICP tarball and load the corresponding Docker images, I ended up with: -

97f41382ba74: Loading layer [==================================================>]  4.727MB/4.727MB
Loaded image: ibmcom/vulnerability-annotator:20171011
9f17712cba0b: Loading layer [========================>                          ]  61.28MB/126.6MB
Error processing tar file(exit status 1): write /usr/lib/python2.7/pydoc_data/topics.py: no space left on device


df -kmh

Filesystem      Size  Used Avail Use% Mounted on
udev            1.9G     0  1.9G   0% /dev
tmpfs           393M  6.2M  387M   2% /run
/dev/sda1        46G   29G   15G  67% /
tmpfs           2.0G  196K  2.0G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           393M   52K  393M   1% /run/user/1000

To mitigate this, I wanted to quickly remove the Docker images, and re-do from start.

This is what I have: -

docker images

REPOSITORY                                  TAG                 IMAGE ID            CREATED             SIZE
ibmcom/icp-catalog-ui                       2.1.0.1             24cf2215580a        4 weeks ago         116MB
ibmcom/kubernetes                           v1.8.3-ee           924aee0d6910        5 weeks ago         513MB
ibmcom/icp-datastore                        2.1.0.1             1575ed9083e6        5 weeks ago         1.72GB
ibmcom/icp-platform-ui                      2.1.0.1             25e8146b6fac        5 weeks ago         153MB
ibmcom/iam-policy-decision                  2.1.0.1             ac23eb27e760        5 weeks ago         576MB
ibmcom/iam-token-service                    2.1.0.1             4da681e7e542        5 weeks ago         891MB
ibmcom/unified-router                       2.1.0.1             caf82a5f8a00        5 weeks ago         50.1MB
ibmcom/icp-platform-api                     2.1.0.1             9529dd6d9349        5 weeks ago         142MB
ibmcom/icp-identity-provider                2.1.0.1             7a8784ad58fc        5 weeks ago         164MB
ibmcom/icp-image-manager                    2.1.0.1             c05a213a80e1        5 weeks ago         25.6MB
ibmcom/icp-router                           2.1.0.1             1e2c1cc654b6        5 weeks ago         46.5MB
ibmcom/icp-platform-auth                    2.1.0.1             96e70190d588        5 weeks ago         498MB
ibmcom/icp-identity-manager                 2.1.0.1             54db055b2b1c        5 weeks ago         185MB
ibmcom/icp-helm-api                         2.1.0.1             b72c1d4155b8        5 weeks ago         139MB
ibmcom/icp-helm-repo                        2.1.0.1             2cb2b0c0ca02        5 weeks ago         98.5MB
ibmcom/metering-data-manager                2.1.0.1             01fc6bf87198        6 weeks ago         72.6MB
ibmcom/metering-server                      2.1.0.1             3401a1fe1a65        7 weeks ago         630MB
ibmcom/metering-ui                          2.1.0.1             216ae682db59        7 weeks ago         425MB
ibmcom/metering-reader                      2.1.0.1             56354bef6b82        7 weeks ago         166MB
ibmcom/usncrawler                           20171128            7f9ab5354091        7 weeks ago         481MB
ibmcom/live-crawler                         20171128            5dd18f1681db        7 weeks ago         715MB
ibmcom/sas-mgmt                             20171113            9801e6747666        7 weeks ago         519MB
ibmcom/sas-api                              20171113            750679c0c313        7 weeks ago         519MB
ibmcom/reg-crawler                          20171122            4d1991b4d0af        7 weeks ago         716MB
ibmcom/gluster                              3.12.1              0a644a8465a6        7 weeks ago         347MB
ibmcom/metrics-crawler                      20171122            f25802bd7454        8 weeks ago         715MB
ibmcom/service-catalog-service-catalog      v0.1.2              566a76042660        2 months ago        105MB
ibmcom/livescan-proxy                       20171011            1f28dd781b9a        3 months ago        418MB
ibmcom/rootkit-annotator                    20171011            e0d54f8afa6a        3 months ago        413MB
ibmcom/password-annotator                   20171011            13cbea171a66        3 months ago        1.19GB
ibmcom/compliance-annotator                 20171011            9a6812e9f19f        3 months ago        1.27GB
ibmcom/vulnerability-annotator              20171011            bca818bb6d43        3 months ago        1.26GB
ibmcom/config-parser                        20171011            b69068191cc9        3 months ago        1.22GB
ibmcom/generic-indexer                      20171011            9a03a4f3cc64        3 months ago        785MB
ibmcom/config-indexer                       20171011            6f388d467108        3 months ago        785MB
ibmcom/elasticsearch                        2.4.1-20171011      7317af2ea391        3 months ago        537MB
ibmcom/indices-cleaner                      0.2                 5e092b17e223        3 months ago        210MB
ibmcom/icp-initcontainer                    1.0.0               76da55c8019d        4 months ago        3.97MB
ibmcom/ucarp                                1.5.2               d6f29a895ed9        4 months ago        7.67MB
ibmcom/tiller                               v2.6.0              47011b53bf14        4 months ago        48.6MB
ibmcom/calico-policy-controller             v0.7.0              60d797585fc5        5 months ago        21.9MB
ibmcom/service-catalog-apiserver            v0.0.15             3a30f3472e44        5 months ago        196MB
ibmcom/service-catalog-controller-manager   v0.0.15             708755fff078        5 months ago        193MB
ibmcom/kube-state-metrics                   v1.0.0              8bbbe7eb4704        5 months ago        43.9MB
ibmcom/calico-node                          v2.4.1              7643422fdf0f        5 months ago        277MB
ibmcom/grafana                              4.4.3               49e2eb4da222        5 months ago        287MB
ibmcom/curl                                 3.6                 f73fee23ac74        5 months ago        5.35MB
ibmcom/coredns                              010                 27e7aa6e71ce        5 months ago        43.5MB
ibmcom/kibana                               5.5.1               7bf53f710f44        5 months ago        629MB
ibmcom/filebeat                             5.5.1               af80995a4fcf        5 months ago        271MB
ibmcom/logstash                             5.5.1               e8f5c2ec30f2        5 months ago        577MB
ibmcom/elasticsearch                        5.5.1               74ef44f69db6        5 months ago        544MB
ibmcom/heketi                               5                   5851537645df        5 months ago        343MB
ibmcom/alertmanager                         v0.8.0              901c70144da4        6 months ago        17.7MB
ibmcom/heapster                             v1.4.0              749531a6d2cf        6 months ago        73.4MB
ibmcom/k8s-dns-sidecar                      1.14.4              38bac66034a6        6 months ago        41.8MB
ibmcom/k8s-dns-kube-dns                     1.14.4              a8e00546bcf3        6 months ago        49.4MB
ibmcom/k8s-dns-dnsmasq-nanny                1.14.4              f7f45b9cb733        6 months ago        41.4MB
ibmcom/prometheus                           v1.7.1              4da113bb6ae3        7 months ago        74.5MB
ibmcom/etcd                                 v3.1.5              3f6d64a8fae8        8 months ago        152MB
ibmcom/node-exporter                        v0.14.0             bb4a6b774658        10 months ago       18.9MB
ibmcom/kafka                                0.10.0.1            223037679e4f        10 months ago       260MB
ibmcom/k8szk                                v2                  2fd25e05d6e2        13 months ago       284MB
ibmcom/configmap-reload                     v0.1                b70d7dba98e6        17 months ago       4.78MB
ibmcom/mariadb                              10.1.16             3b5159f610ed        18 months ago       392MB
ibmcom/registry                             2                   8ff6a4aae657        19 months ago       172MB
ibmcom/pause                                3.0                 99e59f495ffa        20 months ago       747kB

Sadly, there's no command such as: -

docker rmi —all

or similar.

So I did this: -

docker rmi `docker images | awk '{print $3}'`

which is just as good :-)

And now we're back to normal

df -kmh

Filesystem      Size  Used Avail Use% Mounted on
udev            1.9G     0  1.9G   0% /dev
tmpfs           393M  6.2M  387M   2% /run
/dev/sda1        46G   13G   31G  30% /
tmpfs           2.0G  196K  2.0G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
tmpfs           393M   52K  393M   1% /run/user/1000


Having moved the ICP tarball onto a different box ( NFS server ), I'm trying again ….

tar xf /mnt/ibm-cloud-private-x86_64-2.1.0.1.tar.gz -O | sudo docker load 

Monday, 15 January 2018

ADML0004E: An exception occurred when attempting to expand variable ${IBMSCMX} com.ibm.wsspi.runtime.variable.UndefinedVariableException: Undefined variable IBMSCMX

Having installed BPM 8.6 CF2017-12, as part of my ongoing testing of IBM BPM 8.5.5 to 8.6 side-by-side upgrades, I started seeing: -
Continuing with unexpanded genericJvmArguments '-Xscmx120m -Xgcpolicy:gencon -Xjit:iprofilerMemoryConsumptionLimit=67108864 ${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC}'
ADML0004E: An exception occurred when attempting to expand variable ${IBMSCMX} com.ibm.wsspi.runtime.variable.UndefinedVariableException: Undefined variable IBMSCMX
ADML0004E: An exception occurred when attempting to expand variable ${IBMGCPOLICY_GENCON} com.ibm.wsspi.runtime.variable.UndefinedVariableException: Undefined variable IBMGCPOLICY_GENCON
ADML0004E: An exception occurred when attempting to expand variable ${IBMJITPMC} com.ibm.wsspi.runtime.variable.UndefinedVariableException: Undefined variable IBMJITPMC
This manifested itself whenever I ran a Java process such as startManager.sh or startNode.sh or DBUpgrade.sh
For the record, this is what I had installed: -
com.ibm.cic.agent_1.8.8000.20171130_1105
com.ibm.bpm.ADV.v85_8.6.201712.20171211_1726
com.ibm.websphere.ND.v85_8.5.5012.20170627_1018
8.5.5.12-WS-WASND-IFPI82630_8.5.5012.20170718_1447
As part of my testing, I'm using the Deployment Environment exported from a BPM 8.5.5 environment as input into the 8.6 DE creation: -
/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -create -de ~/BPM855DE/De1.properties
Interestingly, this was NOT a problem when I had the base 8.6 binary ( com.ibm.bpm.ADV.v85_8.6.0.20170918_1207 ) installed.
In essence, the problem was that WAS was looking for three variables: -
IBMJITPMC
IBMGCPOLICY_GENCON
IBMSCMX
none of which existed within the WAS cell configuration.
I "cheated" by adding the following three lines to the resources.xml : -
vi /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/nodes/Dmgr/variables.xml
vi /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/nodes/Node1/variables.xml
appending: -
<entries xmi:id="VariableSubstitutionEntry_1515008456894" symbolicName="IBMJITPMC" value="-Xjit:iprofilerMemoryConsumptionLimit=67108864"/>
<entries xmi:id="VariableSubstitutionEntry_1515008456717" symbolicName="IBMGCPOLICY_GENCON" value="-Xgcpolicy:gencon"/>
<entries xmi:id="VariableSubstitutionEntry_1515008456476" symbolicName="IBMSCMX" value="-Xscmx120m"/>
in both cases.
However, that felt too much like a "hack" so I dug further.
Looking back at the De1.properties file that I'd generated from the 8.5.5 environment: -
/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -export -profile Dmgr01 -de De1 -outputDir ~/BPM855DE
I noticed this: -
##################
# JVM properties #
##################
# Do not modify this value. If running on z/OS, the name of the process definition these settings are for. On non-z/OS platforms, this value should be empty.
bpm.dmgr.jvmSettings.1.name=
bpm.dmgr.jvmSettings.1.jvmArgs=${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC}
bpm.dmgr.jvmSettings.1.initialHeapSize=768
bpm.dmgr.jvmSettings.1.maximumHeapSize=2048
bpm.dmgr.jvmSettings.1.verboseModeGarbageCollection=false
##################
# JVM properties #
##################
# Do not modify this value. If running on z/OS, the name of the process definition these settings are for. On non-z/OS platforms, this value should be empty.
bpm.de.node.1.clusterMember.1.jvmSettings.1.name=
bpm.de.node.1.clusterMember.1.jvmSettings.1.jvmArgs=${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC}
bpm.de.node.1.clusterMember.1.jvmSettings.1.initialHeapSize=
bpm.de.node.1.clusterMember.1.jvmSettings.1.maximumHeapSize=
bpm.de.node.1.clusterMember.1.jvmSettings.1.verboseModeGarbageCollection=false
##################
# JVM properties #
##################
# Do not modify this value. If running on z/OS, the name of the process definition these settings are for. On non-z/OS platforms, this value should be empty.
bpm.de.node.1.clusterMember.2.jvmSettings.1.name=
bpm.de.node.1.clusterMember.2.jvmSettings.1.jvmArgs=${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC} -Dsun.net.http.allowRestrictedHeaders=true
bpm.de.node.1.clusterMember.2.jvmSettings.1.initialHeapSize=768
bpm.de.node.1.clusterMember.2.jvmSettings.1.maximumHeapSize=2048
bpm.de.node.1.clusterMember.2.jvmSettings.1.verboseModeGarbageCollection=false
##################
# JVM properties #
##################
# Do not modify this value. If running on z/OS, the name of the process definition these settings are for. On non-z/OS platforms, this value should be empty.
bpm.de.node.1.clusterMember.3.jvmSettings.1.name=
bpm.de.node.1.clusterMember.3.jvmSettings.1.jvmArgs=${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC}
bpm.de.node.1.clusterMember.3.jvmSettings.1.initialHeapSize=768
bpm.de.node.1.clusterMember.3.jvmSettings.1.maximumHeapSize=2048
bpm.de.node.1.clusterMember.3.jvmSettings.1.verboseModeGarbageCollection=false
In other words, the variables had been generated by the -export process.
For some reason, these didn't cause a problem for the September 2017 release of 8.6, BUT did cause a problem with the December 2017 release.
I'm not 100% sure why.
However, the mitigation was to remove the references in the De1.properties, and re-do the DE creation process.

*UPDATE* Having checked with a colleague in IBM, this is a known issue, which should be mitigated in the CF2018-03 Cumulative Fix. In the meantime, the circumvention is to manually amend the exported DE configuration, and remove references to the IBMSCMX, IBMGBPOLICY_GENCON and IBMJITPMC variables is the way to go.

If in doubt, please raise a PMR with IBM Support *UPDATE*

Thursday, 4 January 2018

IBM BPM 8.6 - Performance Dashboard - Who Knew ?

So I've been tinkering with BPM 8.6 since it was first released in September.

And yet I've just discovered this: -

Performance > Performance Dashboard

View statistics (both real time and historical) that are relevant for BPM database performance and performance indicators that influence the performance of your BPM system. Get housekeeping tips for improving performance and generate wsadmin commands for snapshot cleanup. To get started, click Load Data.


within the Process Admin UI: -






etc.

which is AWESOME

Wednesday, 3 January 2018

VMware - Driving it from the command-line

I blogged about this a while back: -

but I've rediscovered the VMware CLI.

On my Ubuntu box, I'm using VMrun: -

vmrun list

Total running VMs: 3
/home/hayd/vmware/icpboot/icpboot.vmx
/home/hayd/vmware/icpproxy/icpproxy.vmx
/home/hayd/vmware/icpworker/icpworker.vmx


and: -

vmrun suspend /home/hayd/vmware/icpproxy/icpproxy.vmx
vmrun suspend /home/hayd/vmware/icpworker/icpworker.vmx
vmrun suspend /home/hayd/vmware/icpboot/icpboot.vmx

and then on the Mac, I'm doing this: -

vmrun start "/Users/davidhay/Documents/Virtual Machines.localized/BPM855.vmwarevm/BPM855.vmx"

and: -

vmrun stop "/Users/davidhay/Documents/Virtual Machines.localized/BPM855.vmwarevm/BPM855.vmx"

and: -

vmrun clone "/Users/davidhay/Documents/Virtual Machines.localized/BPM855.vmwarevm/BPM855.vmx" "/Users/davidhay/Documents/Virtual Machines.localized/BPM86.vmwarevm/BPM86.vmx" full

and: -

vmrun start "/Users/davidhay/Documents/Virtual Machines.localized/BPM86.vmwarevm/BPM86.vmx"

which is, as ever, always nice

Tuesday, 2 January 2018

IBM DataPower Gateway on IBM Cloud Private - Can you say "Doofus" ?

Following my earlier post: -

I'm having MORE fun with the IBM DataPower Gateway (IDG) pattern on IBM Cloud Private (ICP).

Having finally managed to instantiate my container ( I had to drop the resources.requests.cpu and resources.requests.memory values down, as I'm running on a relatively small estate, with a single Worker node with 1 CPU and 8 GB RAM ) : -


I waited for the container to start ( there's quite a bit going on, including the bit where the Worker node has to pulled the IDG Docker image ).

Once it was running, I followed the guidance to get back the endpoint details: -

export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services davidhay-ibm-datapower-dev)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo https://$NODE_IP:$NODE_PORT


and hit the endpoint

and……

got the IBM website.

So I appeared to be proxying IBM.COM, which was somewhat worrying.

I cleaned my configuration, and tried again, and then saw this: -


Yes, you've guessed it, the default behaviour ( within the Helm chart ) is to act as a Web Application Proxy for …. IBM.COM :-)

There's even a clue: -


in the Helm chart.

So I deployed another Helm chart, to spin up a container instance of WebSphere Liberty Profile, deployed the Ferret app to it: -

docker ps -a|grep -i websphere-liberty

1a9a6536444e        websphere-liberty   "/opt/ibm/docker/doc…"   20 minutes ago      Up 20 minutes                           k8s_ibm-websphere-liberty_davehaywlp-ibm-websphere-689cfdbb6c-ffq75_default_a8e5ac8e-efdb-11e7-93d6-000c29651885_0

docker exec -i -t 1a9a6536444e /bin/bash

docker cp ferret-1.2.war 1a9a6536444e://opt/ibm/wlp/usr/servers/defaultServer/dropins

docker logs 1a9a6536444e -f

[AUDIT   ] CWWKF0011I: The server defaultServer is ready to run a smarter planet.
[AUDIT   ] CWWKT0016I: Web application available (default_host): http://davehaywlp-ibm-websphere-689cfdbb6c-ffq75:9080/ferret/
[AUDIT   ] CWWKZ0001I: Application ferret-1.2 started in 1.448 seconds.


and confirmed that I could hit Ferret via the normal ICP proxy: -

https://192.168.1.200:31585/ferret/

and, finally, redeployed my DataPower container, but this time specifying the WLP URL: -


I used the same approach to get the proxied endpoint of the DataPower URL: -

export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services davehaywlp-ibm-websphere)
export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
echo https://$NODE_IP:$NODE_PORT



and was then able to hit the Ferret servlet, via the IDG Web Application Proxy: -


So I've got DataPower running as a Web Application Proxy against Liberty, which is nice :-)

Finally, for reference, if I understand it correctly, I've got four additional containers running on my Worker node: -

docker ps -a

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
b46395546542        ibmcom/datapower    "/bin/drouter"           35 minutes ago      Up 35 minutes                           k8s_ibm-datapower-dev_davehayidg-ibm-datapower-dev-585554d78c-pzbcj_default_211ade0c-efe0-11e7-93d6-000c29651885_0
7a81a4f421ab        ibmcom/pause:3.0    "/pause"                 35 minutes ago      Up 35 minutes                           k8s_POD_davehayidg-ibm-datapower-dev-585554d78c-pzbcj_default_211ade0c-efe0-11e7-93d6-000c29651885_0
1a9a6536444e        websphere-liberty   "/opt/ibm/docker/doc…"   About an hour ago   Up About an hour                        k8s_ibm-websphere-liberty_davehaywlp-ibm-websphere-689cfdbb6c-ffq75_default_a8e5ac8e-efdb-11e7-93d6-000c29651885_0
0e8f9dbb49b9        ibmcom/pause:3.0    "/pause"                 About an hour ago   Up About an hour                        k8s_POD_davehaywlp-ibm-websphere-689cfdbb6c-ffq75_default_a8e5ac8e-efdb-11e7-93d6-000c29651885_0
a77aee47c7fb        bb4a6b774658        "/bin/node_exporter …"   3 days ago          Up 3 days                               k8s_nodeexporter_monitoring-prometheus-nodeexporter-amd64-x5268_kube-system_ce4afa9c-ecc7-11e7-93d6-000c29651885_0
4494b0042b5d        ibmcom/pause:3.0    "/pause"                 3 days ago          Up 3 days                               k8s_POD_monitoring-prometheus-nodeexporter-amd64-x5268_kube-system_ce4afa9c-ecc7-11e7-93d6-000c29651885_0
4ef59603592a        56354bef6b82        "/bin/sh -c /startup…"   3 days ago          Up 3 days                               k8s_metering-reader-amd64_metering-reader-amd64-qf472_kube-system_8697c507-ecc7-11e7-93d6-000c29651885_0
852b06a7a54c        ibmcom/pause:3.0    "/pause"                 3 days ago          Up 3 days                               k8s_POD_metering-reader-amd64-qf472_kube-system_8697c507-ecc7-11e7-93d6-000c29651885_0
261e111c20c2        af80995a4fcf        "filebeat -e"            3 days ago          Up 3 days                               k8s_filebeat_filebeat-ds-amd64-x5bsp_kube-system_6e494fa5-ecc7-11e7-93d6-000c29651885_0
40767f0230db        ibmcom/pause:3.0    "/pause"                 3 days ago          Up 3 days                               k8s_POD_filebeat-ds-amd64-x5bsp_kube-system_6e494fa5-ecc7-11e7-93d6-000c29651885_0
fee15f1d965a        88ca805c8ddd        "/install-cni.sh"        3 days ago          Up 3 days                               k8s_install-cni_calico-node-amd64-2d2hk_kube-system_95bdffaa-ecc6-11e7-93d6-000c29651885_0
2b89043639c3        7643422fdf0f        "start_runit"            3 days ago          Up 3 days                               k8s_calico-node-amd64_calico-node-amd64-2d2hk_kube-system_95bdffaa-ecc6-11e7-93d6-000c29651885_0
2fbc29422aa7        ibmcom/pause:3.0    "/pause"                 3 days ago          Up 3 days                               k8s_POD_calico-node-amd64-2d2hk_kube-system_95bdffaa-ecc6-11e7-93d6-000c29651885_0
dac6368cc3f9        924aee0d6910        "/hyperkube proxy --…"   3 days ago          Up 3 days                               k8s_proxy_k8s-proxy-192.168.1.201_kube-system_8312bd0a1d5cee59daf19f2df2bdf2fe_0
631f76b15478        ibmcom/pause:3.0    "/pause"                 3 days ago          Up 3 days                               k8s_POD_k8s-proxy-192.168.1.201_kube-system_8312bd0a1d5cee59daf19f2df2bdf2fe_0

via the two Helm releases: -




 



IBM DataPower on IBM Cloud Private - Wrapping it up

I'm using IBM Cloud Private (ICP) 2.1.0.1 to spin up a container running IBM DataPower Gateway (IDG), via a Helm chart.

Right now, I'm cheating using the ICP admin UI: -


rather than the Helm command-line interface, as per my normal practice.

The Helm chart has two mandatory values - frontsideKey and frontsideCert - which contain a private key and a personal certificate.

I generated the key and certificate using openssl : -

openssl req -subj '/C=GB/O=IBM/CN=davehay.uk.ibm.com' -new -newkey rsa:2048 -days 365 -nodes -x509  -sha256 -keyout server.key -out server.crt

When I clicked the button to instal the Helm chart, I saw this exception in the UI : -

Internal service error : release DaveH failed: Secret in version "v1" cannot be handled as a Secret: v1.Secret: Data: decode base64: illegal base64 data at input byte 64, parsing 1177 ...X375XW1M="... at {"apiVersion":"v1","data":{"cert.pem":"MIIDQzCCAiugAwIBAgIJAMNFpd9mxNckMA0GCSqGSIb3DQEBCwUAMDgxCzAJBgNV BAYTAkdCMQwwCgYDVQQKDANJQk0xGzAZBgNVBAMMEmRhdmVoYXkudWsuaWJtLmNv 
G6rTAsZcFxBukcRxfCk4CmRR1C16dlsiz64rKVkdGO9IUTF0GchNAobYcAEK+dbW v+jU2J13dMnmzYfOz9QZkpM="},"kind":"Secret","metadata":{"labels":{"app":"DaveH-ibm-datapower-dev","chart":"ibm-datapower-dev-1.0.3","heritage":"Tiller","release":"DaveH"},"name":"DaveH-ibm-datapower-dev-secret","namespace":"default"}}

Reading this: -



I realised that the requisite values should be base64-encoded.

However, it's my understanding that X509 certificates already base-64 encoded.

I fiddled about for a bit …. and then realised where I was going wrong.

The .cer and .key files that are generated by the openssl command are written to files that are automatically formatted as …. 64-column files :-)

That may be the base64 encoding :-)

However, the ICP/Helm UI doesn't like the resulting wrapped text in the frontsideKey and frontsideCert parameters.

So I manually stripped away the CR/LF characters at the end of each 64-bit row, and pasted the result into each of the requisite fields.

This got me further, but I then saw this: -

Internal service error : release DaveH failed: Secret "DaveH-ibm-datapower-dev-secret" is invalid: metadata.name: Invalid value: "DaveH-ibm-datapower-dev-secret": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')

which is easier to fix.

This is what ICP says: -


and this is what I had for my Release Name: -

DaveH

Can you say "Doofus" ?

Once I changed it to daveh, we're good to go: -