Information Store processing after you upgrade i2 Analyze
After you upgrade a deployment of some versions of i2 Analyze with an Information Store, extra processing of the data in the Information Store might take place. Analysts can continue to use the system while data is processed in these online upgrade operations.
Upgrade tasks
A number of different processes for updating the contents of the Information Store can occur after you upgrade i2 Analyze. Some of these processes are mandatory and automatic, while you can initiate others through toolkit commands.
The tasks that the server might perform depend on the versions of i2 Analyze that you are upgrading from and to. Each task can place additional restrictions on the system until it is complete.
At this version of i2 Analyze, there are no tasks that run automatically when you upgrade from a supported version. However, there is an optional task for Improving Visual Query performance for multi-line string property types when the Information Store uses an IBM Db2 database:
Update multi-line property columns data type from CLOB to VARCHAR
Blocks analysts from uploading to the Information Store
Blocks ingestion and deletion
Status and progress reports
The Information Store provides information about the progress of upgrade processes through a set of views that offer different levels of granularity.
Status view
The IS_Public.Upgrade_Status view in the Information Store database shows a complete list of tasks that are Complete or Pending (or have Failed):
task_code | description | status | start_time | end_time |
|---|---|---|---|---|
VQ_MLS | Converts record details stored in Db2 as CLOBs to VARCHARS to enable visual queries on them | Pending | 2019-08-13 15:30:04.49 |
|
The possible values for the status column in this and other views are:
Pending
The task is either not started, or started but not complete.
Note: If you stop i2 Analyze while some tasks are Pending, those tasks continue when you next start i2 Analyze.
Complete
The task completed successfully.
Failed
The task did not complete successfully.
If a task fails, check the messages.log file for messages or stack traces related to online upgrade. Resolve the issues, and then restart the Liberty server.
Progress view
For tasks that are Pending or Failed in the Upgrade_Status view, the IS_Public.Upgrade_Progress view shows progress by item type:
task_code | description | schema_type_id | display_name | status | start_time | end_time | latest_log_end_time | rows_processed |
|---|---|---|---|---|---|---|---|---|
VQ_MLS | Converts record details stored in Db2... | ET1 | Address | Complete | 2023-08-13 15:30:05.12 | 2023-08-13 15:32:48.89 |
|
|
VQ_MLS | Converts record details stored in Db2... | ET5 | Person | Pending | 2023-08-13 15:30:05.12 |
| 2023-08-13 15:41:12.20 | 100000 |
VQ_MLS | Converts record details stored in Db2... | ET6 | Document | Pending | 2023-08-13 15:30:05.12 |
|
|
|
You can use this view to check how many item types processing is complete for, and how many are still to be processed. When a task fails, the view marks the type being processed at the time - and all subsequent types - as Failed.
Even with upgrade tasks broken down on a type-by-type basis, the processing for a type that has a large number of rows can still take a significant amount of time. In such cases, the latest_log_end_time and rows_processed columns are updated after every 100,000 records. For more information still, you can examine the log view.
Log view
When a task is Pending in the Upgrade_Progress view, and it has data in the latest_log_end_time and rows_processed columns, IS_Public.Upgrade_Log provides the following information:
task_code | schema_type_id | display_name | start_time | end_time | duration_seconds | rows_processed | rows_processed_per_second |
|---|---|---|---|---|---|---|---|
VQ_MLS | ET5 | Person | 2023-08-13 15:30:05.12 | 2023-08-13 15:41:12.20 | 667.08 | 100000 | 149.91 |
The Upgrade_Log view gains a row for each 100,000 processed records, and for the last batch of records that completes processing for a particular type.
You can examine the Upgrade_Log view to confirm that the upgrade process is progressing, and at what rate. If you know roughly how many records are in the Information Store, you can also estimate how long the whole process might take.
Note: Rows in the log view for a particular type are removed when processing for that type ends.