Preparing for ingestion

You must complete three tasks before the Information Store can ingest data from an external source. You must identify exactly which data to load, transform the data to align with the active i2 Analyze schema, and augment the data with extra information that the Information Store requires.

About this task

The only way to add and modify large volumes of data in the i2 Analyze Information Store is to enable and then instruct the Information Store to ingest it. The enablement process involves creating and populating staging tables for the data, and then supplying the metadata that is crucial to the analytical capabilities of i2 Analyze.

Procedure

You can plan and run the Information Store data ingestion process in a series of discrete steps. This diagram illustrates the approach.

  1. Decide which entity types and link types in the active i2 Analyze schema best represent the data that you want the Information Store to ingest.
  2. Create staging tables in the database for the types that you identified. Create more than one staging table for some link types.
  3. Use external tools, or any other appropriate technique, to transform your data and load the staging tables with the data for ingestion.
  4. Add information about your data source to the list of ingestion sources that the Information Store maintains.
  5. Write the ingestion mappings that govern the ingestion process and provide additional information that the Information Store requires.
  6. Run the ingestion command separately for each of the ingestion mappings that you wrote.

Example

The examples\data\law-enforcement-data-set-1 and \signal-intelligence-data-set-1 directories in the deployment toolkit contain files that i2 Analyze uses when you run the setup -t ingestExampleData command to populate the Information Store during deployment. These files provide demonstrations of many of the steps in the standard approach to ingestion. The following topics describe those steps in more depth as they detail the Information Store ingestion process.