Please note the following!
Beginning with version 3.0 the upgrade from DB2 to version 10.5 FP2 is mandatory.
Therefore, first of all please download this DB2 version from the IBM web page and then perform the upgrade. For instructions, please click the following link: DB2 upgrade from version 9.7 to 10.5 Fixpack2
Starting from version 2.9 a new license file is recommended.
However, it is also possible to keep the old license file further. In this case, new features in the license dialogue will appear as 'missing' and are deactivated. Otherwise, this will not have any further negative influence on the function of BVQ.
Analysis, monitoring and reporting:
- The module 'alerting' allows now the definition of threshold values in the GUI and thier surveillance (monitoring).
- 'HD' high definition statistics: support for any interval widths of the SVC performance statistics down to 1min.
- Looking back into the past: 'Retrospective PIs' transfer the load conditions to a past date in the treemap.
- 'SVC cache partition analysis': get an overview of the cache utilization of each IO group.
- Object-specific functions are now available everywhere in the same way by using the new 'Common context menu'.
- Acquisition of the selected objects between lists and tree maps - 'Show in treemap', 'Show in list'.
- Improved communication with IBM support: performance statistics in the original format can now by kept an unlimited period of time and can be sent to IBM in case of problems.
- New possibilities: create reports directly from the database with self-defined queries (views), managed by a customized report installer.
- Accelerated and simplified path selection in the topology view by using the new 'Text mode'.
- Massive click-reduction with the new object clone function 'Copy ...'
- Significantly fewer delays in the performance view because of multi threading and less memory usage.
- Significantly simplified procedure to remove SVC and Storwize clusters.
- Acceleration of database accesses by using advanced functions of the IBM DB2 10.5 Advanced Workgroup Server Edition.
Some highlights in detail
The BVQ Alerting Module automatically warns the user, if the configurable threshold values for attributes of specific object types are exceeded.
This module has already been published in BVQ version 2.9 and has now been significantly expanded. The configuration of thresholds and injury reactions can now also be done via the GUI.
Alert report configurations were significantly simplified by the introduction of status codes like 'all', 'better' and 'worse' .
- SVC MDisk group
- SVC VDisk
Walkthrough: alert definition
The functioning of the module is described here as a walkthrough along a practical example to make it easier to understand.
The aim is to define a signal for a VDisk, if the response time of 10ms is exceeded (at a data rate under 50MiB/s).
To reach this aim, the following steps have to be performed in the BVQ GUI to configure the alerting module:
- Step 1: open the VDisk editor
- Step 2: creating a new 'Alert rule'
- Step 3: define alert rule 'RT>10ms'
- Step 4: assigning the alert rule 'RT>10ms' to a VDisk
- Step 5: checking the alert state of an object:
Step 1: open the VDisk editor
Click with the right mouse button on a VDisk object and select 'Edit ...' in the context menu. The rule can be assigned to multiple objects of different types.
Step 2: creating a new 'Alert rule'
A BVQ Alert rule combines conditions (alert conditions) and notifications (alert reports) .
In the VDisk editor a new configuration section can be found for assigning and generating alert rules:
In the left field already defined alert rules which are not assigned to this object can be found. The right side contains already allocated alert rules.
First of all an alert rule has to be created. Click with the right mouse button in one of the two fields, and select 'Add ...' from the menu.
Step 3: define alert rule 'RT>10ms'
In the following dialogue, click 'Edit Alert rule', enter the name 'RT> 10 ms' into the field 'Name:' and the alert rule is created by clicking 'Apply'.
Step 3.1: creation of filter condition 'vdisk_realtime_dr < 50'
Now click the button (Add...) beneath the field 'Filter conditions'.
The dialogue 'Edit Alert condition' appears, and should be completed as the following picture illustrates:
After the button '...' behind the field 'Attributes' is clicked, the dialogue 'Choose attribute' appears, in which the name 'vdisk_realtime_dr' can be chosen comfortably:
If both dialogues are closed by clicking 'OK', the system returns to the dialogue 'Edit Alert rule', which now contains the new filter condition 'vdisk_realtime_dr <50'.
Step 3.2: creation of alert condition 'vdisk_realtime_rt > 10 --> Error'
At first the button (Add...) has to be clicked, but this time under the field 'Alert conditions'.
Again the dialogue 'Edit Alert condition' appears. Please complete the fields as follows:
In the dialogue 'Choose attribute', please choose the attribute name 'vdisk_realtime_rt' from:
A 'result' in the form of an alert level must be configured in the alert condition, contrary to the filter condition.
After the confirmation of the dialogue 'Edit Alert condition' with 'OK', return to the dialogue 'Edit Alert rule', which now contains the new alert condition 'vdisk_realtime_rt> 10 -> Error'.
The condition´s definition for 'exceeding the response time of 10ms at a data rate below 50MiB / s' for the alert level 'Error' is now completed.
Step 3.3: creation of alert report config 'all -> Error : E-Mail'
Now only the definition is missing in which ways notifications (report) should be made.
To do this click the button (Add...) beneath the field 'Report config'.
The dialogue 'Edit Alert report config' appears. Please complete the fields as follows:
The option 'All' in the field 'From' is new in this version.
Step 3.4: activation of alert rule
Click the checkbox behind 'Enabled' to activate the alert rule. The whole dialogue should now look as follows:
Close the configuration of the alert rule by clicking 'OK' and thus get back to the dialogue 'Edit VDisk' from step 1.
Step 4: assigning the alert rule 'RT>10ms' to a VDisk
The alert rule 'RT> 10ms' is still on the left side of the configuration area 'Alert rules' in the VDisk editor. Please shift it to the right side ('>>' button) to assign it to this VDisk:
Close the dialogue with 'OK'. The alert rule is now active and complete .
Step 5: checking the alert state of an object:
The current object´s alert level can be checked in the treemap via the color scheme 'Alert level → Object specific → Overall':
More details get visible, if the object path '/BVQ/alert rule/alert rule mapping/VDisk' is used with this color scheme:
When the alert service is run for the first time with the newly defined mapping of the alert rule, the state will change from 'undefined' to 'OK' and the objects will appear green colored in the treemap.
The object type 'Alert rule mapping' is always dyed in other color schemes, based on the current alert level.
More information about alerting can be found in the BVQ Release Notes 2.9.
In a SVC / Storwize system the interval how often the internal statistics are retrieved can be set manually. Integer interval widths of 1 to 60 minutes can be chosen, which are now supported by BVQ too.
Periods of time, which require a particularly closer examination, can now be analyzed in a five times higher time resolution.
CPU workload with a 5min interval width:
The same CPU workload with a 1min interval width:
In standard cases it is recommended to leave this interval width of 5min unchanged and only to decrease it in specific periods of time. Just change it back when the analysis is finished. The BVQ scanner services are automatically adjusted to the time set in the SVC interval width. A change of the configuration or a restart of the services is not required!
In the performance view of the BVQ GUI, the most accurate interval width is automatically displayed. But via the options dialogue under 'Fixed interval' it can be adjusted manually:
Retrospective performance indicators
The retrospective performance indicators show past load conditions in the treemap.
This is particularly interesting, if load peaks or performance bottlenecks occurred long before the date the user checked the BVQ GUI. If, for example, an interesting peak load appeared at a known time in the past, so now the performance indicators can be loaded into the GUI, which were calculated at the 5 minutes interval, including this 'failure-occurance' moment.
To simplify the selection of this indicator time it was relocated from the aspect and color-scheme menu into a common PI-time-selection menu:
If the menu item 'performance date ...' is chosen, it is possible to adjust the known moment in time of an incident in the following dialogue and the loading process can be started:
All performance indicator-values visible in the treemap, tooltips and object detail-views now indicate the aggregated values for the most recent time interval, which also contains the specific moment in time .
Example: adjusted: PI time: 10:00 o´clock, interval: 'Real-time (5 min)' → averages apply e.g. for the interval from 09:56 to 10:01 o´clock.
SVC /Storwize cache partition analysis
SVC and Storwize systems limit the shared write cache per IO group, which is available for the volumes of a pool by using the so-called cache partitioning. Now BVQ shows the number of cache partitons per IO group and the available maximum cache per volume, pool or IO group.
High IO response times are often caused by an overflowing write cache, arising from various circumstances, such as: too slow backend storage, too much frontend write data, too small dimensioned cache , unfavorable load-distribution on the IO groups. The illustrated cache usage in BVQ supports the investigation of such problems.
Representation in the treemap
In the topology path 'BVQ → IO group → MDisk group' the treemap only shows the MDisk groups (pools) which are using the IO group-surrounding cache .
In the topology path 'BVQ → MDisk group → IO group' the treemap only shows the IO groups, which are providing cache to the surrounding MDisk group .
Representation in the object detail-view
In the object details view of the object types cluster, MDisk group and VDisk, the maximum size of the write-cache partitions per IO group is shown now.
Here is an example for the object type MDisk group:
Representation in the performance view
In the performance view the curves of the cache partition metrics are now presented separately per IO group. Thus, the overflow in a single IO group can easily recognized.
In the object list of the performance view´s option dialogue this gets obvious:
In the performance view for the object type VDisk, it is possible now to compare directly the metric "Cache size W (CM)" with the threshold, which is specified in the object detail view .
Please note here that CM already includes the cache consumption of both nodes in total, so generally it has to be divided by 2 in order to be set in relation to the threshold!
Unified context menu
No matter in which context an object is visible in the BVQ GUI, all object-specific standard features are now available in a single context menu.
This simplifies operation and supports the intuitive approach to find problem causes.
The following screenshot shows the context menu of a VDisk in the treemap (right-click on a VDisk):
Now basically there is a chapter with specific functions of each view, but starting from the first hyphen the menu is designed equally throughout the whole BVQ GUI, divided into the following three chapters:
- Editing functions
- Links to represent the object in other views
- Performance views of the object
For example, if the load curve of a VDisk is analyzed in the performance view, the user can easily switch now to the performance view of the underlying MDisks by selecting them in the object context menu of the options dialogue´s object list .
Show in Treemap
Contrary to the existing 'Show in list' there is now also a 'Show in Treemap' in the object context menu. If it is used the current object selection of a table view, of an performance view´s object list or of a treemap can be transferred to another existing treemap.
This allows to emphasize especially the functional strengths of the tables- and topology view. For example the table view is best for sorting and filtering. An object selection done with it, can now simply transferred into a treemap. Or the selection of a particular subset of objects in an appropriate treemap object-path can be transferred into a treemap with a completely different path and aspect.
'Show in treemap' can be found directly under 'show in list' in the unified object context menu:
Now it is possible to transfer the properties of existing BVQ objects into new ones (cloning) by using the function 'Object copy'.
In the past the creation of a variety of similar disk array objects, for example, was associated with high manual editing operations. This is now much simpler and thus less time-consuming.
'Copy ...' is found directly under 'Add ...' in the unified object context menu:
For example if such an object copy array is executed on an existing disk, the disk array editor is opened with pre-filled attribute fields. Additionally the system tries to detect a sequence number in the object´s name which is automatically incremented and inserted into the name-field of the new object. In general, the dialogue has only to be completed with 'OK' .
SVC statistics in original format
Performance statistics can now be kept in the original file format of an SVC or Storwize system almost any period of time.
A SVC support snap usually only contains statistics up to a maximum of 1.5 hours in the past. This is often too short for a problem which occured at midnight or at a weekend. Now the statistics retained by BVQ with valuable measured values of the critical moment in time, can additionally be sent to IBM. This helps to improve the communication with the IBM support and accelerates their root-cause analysis.
In the next dialogue of the scanner installer a different value for the storage-duration of each instance can be set:
It should be ensured that enough free storage is available in the configured queue directory for this feature!
The files are usually located in the done-folder of the BVQ SVC scanner´s configured queue directory:
Topology navigator - Text mode
In the panel ot the topology view there is still the familiar menu-based topology navigator. However, also a text-based method to enter the object path is available now.
Often it was difficult to find the desired entries in the meanwhile rather large menus of the topology navigator. If already some of the object types´ names are known, the path input is significantly simplified by the text mode.
The well-known topology navigator is now headed by a button.
A first click on this button switches the topology navigator to text mode, with a second click it returns to menu mode.
The path-input via text mode is supported by auto-completion a s soon as the path can be clearly identified. Pressing the [Tab] key completes the name as much as possible (tab completion).
With this BVQ version a tool for deleting SVC or Storwize systems from the BVQ environment is available.
If an SVC or Storwize system is no longer needed, it can be easily removed from the BVQ environment.
cluster-deletion.exe can be found in the directory:
The tool opens a simple dialogue which searches the BVQ DB for known SVC / Storwize clusters and displays them for selection:
After clicking 'OK', the tool marks the selected cluster in the BVQ DB as deleted. It will not be displayed in the GUI anymore. Nevertheless, the data of the cluster are still available in the BVQ DB to use them, for example, for historical reports. The data of the performance measurement are automatically deleted by the normal cleanup-processes in the BVQ scheduler.
More new features
Topology auto reload
Now it is possible to adjust whether and at what interval the GUI´s display is automatically updated with the data from the managed environment.
With this , for example, a permanently updated display of BVQ on a monitor in the operating can be realised.
To accomplish this, the button 'Reload from DB' was expanded and moved to the status bar next to the button to reload the GUI´s performance indicators:
Statistics of the SVC / Storwize compression CPU cores
Since code version 220.127.116.11 SVC and Storwize systems also provide differentiated values about the utilization of the system and compression CPU cores.
These values are displayed in the new BVQ performance graph type "Node CPU core". By using this, a detailed analysis of the CPU utilization is possible.
Meaning-illustration of aspects and colorings
The meaning of aspects and object color schemes is now be explained in a tooltip for each menu.
This eliminates the time-consuming looking up in the BVQ documentation.
All BVQ modules are now installed and updated by using a new installation tool (install4j). This has the following advantages:
- The installation procedure is easier and safer.
- The names of current running system processes correspond to the module names (not 'java.exe').
- Now all installation packages are 'exe'-files. The dependence of the association of 'jar' files to the installed Java Runtime system is no longer in effect.
- Each installed BVQ module is now also included in the standard Windows list of installed programs and can be uninstalled from there too.
- The installation procedure can be automated via script.
- Log, config and data files are now stored in standard Windows directories.
Performance-scan: allow any intervals >1min
Enable configurable retention period of the iostat files in the installer dialogue
Enable deletion of a cluster from the BVQ DB
Enable SSH authentication also via user name / password
Upgrade from DB2 9.7 Express to DB2 Enterprise 10
Custom reports: relocate custom DB view schemas into custom report installer
VMware: execution of functions depending on the license modules 'Topology-scan' and 'Gui-toplogy'
Alerting: execution of functions depending on the license module 'Alerting'
Table view: function 'Show in treemap': copy current table selection into the treemap
Topology view: treemap: show legends for meaning of the area-size and staining
Topology view: performance indicators: retrospective PIs - display past PIs
Topology view: automatic reload of the topology and the performance indicators with adjustable repetition rate
Topology view: path browser: improved operation by using text mode
Object detail view: MDisk group, node, Iogroup, cluster: display of the MDisk group cache partition distribution
Performance view: display MDisk group cache partition metrics per IO group
Performance view: integrate SVC 18.104.22.168 CPU core performance statistics (CPU busy system vs. compression)
Performance view: VDisk: options: separation of 'SCSI layer' and 'Above cache' metrics
CLI: record output in file 'bvq-gui-cli.log'
Alerting: integrate editors for alert conditions
Alerting: integrate editors for alert report config
Alerting: topology view: treemap: integration of 'Alert rule'
Alerting: topologie view: treemap: integration of "Alert rule map"
Alert rule: integrate flag 'Enabled' to enable / disable an alert rule
Alert report config: introduce from/to status values 'all', 'better', 'worse'
Alert report config: integration of type 'Text file reporter'
Alert condition: allow text comparisons
Set Windows process-names corresponding to the service / application names
Email notification: user should not be mandatory
Save log and configuration files compliant to UAC and Windows security
Allow to display the command-help with -? etc.
Command addmdisktodiskarray: mdisk.name ohne prefix "svc_cluster.name:" should be accepted, if it is clear
Scheduler / DB
DB Cleanup: perf-tables: switching to MDC Rollout Delete and Deferred Async Index Cleanup
Extend Alerting-Info.log with output: found(xxx) alertconditions for rulexxx
CLI: mkalertcondition '-compared_to_value' is not outputted as missing parameter
CLI: mkalertrule: improve testing and help text for the options '-default_level' and '-fallback_level'
Upgrade Eclipse BIRT engine to version 4.3
Tabs: accelerate the appearance of the 'tab preview windows'
Tabs: clearly naming of tabs by prefixing an ID
Tabs: allow closing of all / all other tabs in the tab bar by mouse click
Performance view: parallelize the loading of raw performance-statistics from DB
Performance view: parallelize calculation of 'real' performance-values based on counter values
Performance view / table view: integrate common object context menu
Performance view / table view: note system time and number format for data export into CSV file
Performance view: shift bold-printing of objects into common object context menu
Performance view: fixed interval: harmonize reference start time
Performance view: options: allow 'Fixed Interval' selection 'None' (Rwa data as in DB)
Performance view: options: limit "Fixed interval" to maximum chart width
Performance view: checkmark dropdowns also for 'Duration', 'Fixed interval' and 'Moving average'
Editors: field lables should never be displayed shortened with '...'
Editors: move table views to the front after opening
Editors: integrate merge/demerge/multi-edit for boolean values
Editors: ensure easier reading of validation messages
Editors: mark validation messages with consistent icon
Editors: optimize behaviour at window resize
Editors: support any reduced screen heights by using ScrollPane
Editors: integrate MultiEdit also for (m:n multi-selects)
Editors: disk type: bring out messages more clearly
Tables view - change export function to 'export selected only'
Tables view: add column 'Alert rule' to VDisk & MDisk groups
CLI: switch 'execute on enter' to single line and prevent input-clear
Common object context menu: make grouping of entries more meaningful
Common object context menu: relocate menu item 'cluster CLI scripts' into common object context menu
Common object context menu: integrate function 'copy...'
Common object context menu: also available in editors
Topology view: simplify the selection of PI-interval by separate dropdown
Topology view: coloring: IO density (heat): add additional gradation between 20% and 80% (turquoise bei 50%)
Topology view: additional paths: 'VDisk -> extent' and 'MDisk group -> IO group'
Topology view: define and add legend for color scheme 'Alerting'
Topology view: treemap: enable the display of VDisk copies in performance aspects
Objekt detail view: various: rename field 'Host port count' in 'Host port defined count'
Objekt detail view: various: detail 'VDisk copy count' according to prim / sec
Objekt detail view: MDisk group: attributes: SVC: revising of staining / sorting
Scan should be stopped at 'Error writing cluster identification file!' (cluster_id.txt)
Closing a SSH session takes an unusually long time, sometimes some remain opened
Old 'Topo' files cannot be deleted if persisting
Editors: changes in input-fields which are undone before 'save' or 'apply' is executed, cause an update and thus a duplicate history entry
Performance view: VDisk: systematic error in the calculation of CTRH, CTRHP and SLAV/CTRM
Customize DB application name
Error deleting iostats folder / cluster_id.txt
Table view: export: columns with counts are exported as lists
Table view: selected lines are preserved even though the sorting sequence was changed (column function)
Performance view: user favorites: NPE when opening user performance graphs being older than 100 days
Performance view: calling node to node perf graph: NPE
Topology view: cluster script dialogue: calling from context menu entry does not work anymore
Scanner status dialogue: call takes too long (more than 1min)
Editors: columns of embedded tables do not spread when the width is increased
Topology view: color selection dialogue: color brightness control: gradient reversed to set color
Topology view: at mobility zones the coloring 'BVQ Qualified' is displayed correctly just after a second update
Performance view: options dialogue: object marking is lost when scrolling with tooltip display
Performance view: options: double repaint after selection of a new object
Performance view: option panel loses control after favorites storing
Performance view: charting does not load enough data in advance to provide an sufficient amount of lines for the set time interval
Table view: edit button is always greyed out
Favorites/tabs: tab rename does not work anymore, if favorites saving is applied multiple times
Tabs: previews: when tabs are in scroll mode the position is too far to the right
Tabs: sometimes magnifying glass not visible in scroll mode .
Scanner status dialogue: detail error dialogue: red exclamation mark under dialogue
|IBM PMR 91765,033,724||SVC 22.214.171.124|
Strong impairment of the VDisk latency caused by the SVC CLI command 'lsmdiskextent'. Seen in a cluster with 8 nodes (8G4) under 126.96.36.199.
|IBM PMR unidentified||SVC 188.8.131.52|
Massive occurence of reaching the SVC limit of 10 simultaneously competing SSH sessions may trigger a soft reboot of the node.
Multiple mappings of a VM virtual disk (and/or different snapshot-versions of a single mapping ) to a VM virtual machine are not supported yet.
|BVQ-6623||BVQ VMware||BVQ does yet not support the scanning and display of snapshots and their capacities. The used capacity of a VM virtual disk currently corresponds to the used apacity of the original VMDK file.|
|IBM PMR ?||SVC performance stats|
Starting from version 184.108.40.206 SLAV, PLAV and DLAV refer to the consolidated backend IO sizes up to 256k. But not the IO counter for CTRM and CTP. This means their quotient may be a factor of 8 too low. Therefore, the VDisk below cache metrics SLAV and PLAV cannot be used directly to assess the backend latencies. Also their transaction size is not the same as that of the backend (max 32k).
CTD, however, was adjusted to the consolidated IO sizes and continuesly provides useful values.
DB Schema changes
|220.127.116.11||N/A||Raising schema-level to prevent further persistence in running scanners|
|18.104.22.168||ALERT_RULE||Adding column 'ENABLED' for BVQ-6859|
|22.214.171.124||ALERT_REPORT_CONFIG||Migration of contents for the introduction of BVQ-6873|
|126.96.36.199||PERF_SVC_NODE||Adding columns 'COMP' and 'SYSTEM' for BVQ-6126|
|188.8.131.52||PERF_SVC_NODE_CPU_CORE||New table for BVQ-6126|
|184.108.40.206||SCANNERLOG||Improved index to speed up the GUI display|
|220.127.116.11||Alle PERF_SVC_*||Changing deletion to rollout delete for BVQ-7109|
Requirements and limitations
Requirements of the HW / SW environment
Minimum required BVQ version to perform an update
DB2 Express V10.5 FP2
|Java Runtime minimum version||18.104.22.168|
- When configuring a topology scanner interval >15 min, an adaptation in a GUI configuration file is necessary, regarding the scanner status checks. Please contact the BVQ support in this case.
- The Excel-format generated by the BVQ reporting is compatible with Excel, starting from version 2003.
It is the format 'XML Spreadsheet 2003' (please see Microsoft article).
The default file extension for this format is: '.xml'. If it is despite saved as '.xls' , a warning message is shown when it is opened with Excel.
- Licensing changes will take effect in the scanner and scheduler after about 2 hours or after reboot.
- Currently unsupported VMware functions:
- Partitioned VM SCSI volumes, with partitions which are assigned to multiple VM datastores
- VM SCSI volumes with UUID which are not unique.
Short description: update steps
Note about upgrade
- During the BVQ scanner upgrade from version 2.7 to 2.8 or higher, it should be noted that besides the SVC scanner also a VMware scanner exists. Therefore it is installed into a new default directory bvq-svc-scanner (previously bvq-scanner). To use the existing scanner configuration again please proceed as follows before carrying out updating step 3 :
- Stop BVQ scanner services manually
- Rename directory bvq-scanner in bvq-svc-scanner
- Continue with update step 3.
- To be able to use new features, a new license file is required.
After the import of the new license a GUI restart is required.
- The GUI is equipped with new Predefined favorites. Only User favorites are retained.
We strongly recommend to follow the sequence of update steps listed below:
- Check for correct function of the environment, especially the scanner.
- Stop BVQ scanner, alerting service, scheduler services, reporting-clients and all GUIs manually
- Backup BVQ installation directory and BVQ DB (e.g. DB offline backup with DB2 control center)
- Restart BVQ scanner
- Update DB schema:
- Update BVQ SVC scanner:
- Update BVQ VMware scanner:
- Update BVQ alerting:
- Update BVQ scheduler:
- Update BVQ GUI:
- Update BVQ reporting engine:
- Update BVQ report module:
- Standard reports
- Customized reports
- Standard reports
- Perform functional tests
- If necessary, inform all colleagues about to the needed update of locally installed BVQ GUIs / report client