Using standard import mode

After you create and populate your staging tables, write your ingestion mappings, and validate your work, instruct the Information Store to ingest your data.

Before you begin

If you are using correlation, you must ensure that the application server is started before you run the ingestion commands.

About this task

When you instruct the Information Store to ingest the data that you loaded into the staging tables, you do it one ingestion mapping (and one staging table) at a time. The i2 Analyze deployment toolkit and the ETL toolkit both have a command for ingesting the data that is associated with a particular ingestion mapping in a particular mapping file. For more information about the running the command and any arguments, see The ingestInformationStoreRecords task.

Procedure

The procedure for instructing the Information Store to ingest your data is similar to many others in this process. You start with one type or one staging table, and build from there.

  1. Choose an entity staging table that you populated with data and provided with an ingestion mapping.

  2. Run the ingestInformationStoreRecords command in STANDARD mode to instruct the Information Store to ingest your data. For example:

    setup -t ingestInformationStoreRecords
          -p importMappingsFile=ingestion_mapping_file
          -p importMappingId=ingestion_mapping_id
          -p importLabel=ingestion_label
          -p importConfigFile=ingestion_settings_file
          -p importMode=STANDARD

    For more information about the running the command and any arguments, see The ingestInformationStoreRecords task.

    Note: You can improve the performance of entity ingestion by running ingestInformationStoreRecords for different entity types at the same time when you use the standard import mode. It is recommended that you use the ETL toolkit to run concurrent ingestInformationStoreRecords commands. Do not attempt to run the command for data of the same type at the same time.

    Due to the increased number of operations and comparisons that are required, using correlation can make the ingestion process take longer.

  3. Repeat steps 1 and 2 for the other staging table and ingestion mapping combinations that you created. Take care to run the command for entity types before you run it for link types.

    The ingestion process for links verifies that the entities at each end of the link are already ingested. If it fails to find them, the process fails.

Results

At the end of this procedure, all the external data that you populated the staging tables with is in the Information Store. To add or update records, you can repopulate the staging tables and rerun the ingestInformationStoreRecords command.