Importing Data

You may already have some data you wish to analyze in electronic form, for example in another iBase database, a spreadsheet, some other database, or as a text file. You use an import specification to define how the source data is interpreted during the import.

You will require an import specification for each entity and link type in the source data. You can run the import specifications singly or in a batch (if you first set up an import batch specification).

Planning imports

Before you create an import specification, it is always worthwhile to compare the data with the database to see what type of entity will be imported, which fields are mandatory, and which fields will be identifiers (used to decide if records match those already in the database). You may also find it useful to print out a database design report and a sample of the data that you want to import.
Note: If you are importing entities or links that use multi-line text (append only) fields, you may want to test the import first using a copy of the database. This is because you can only add text to the end of fields of this type; you cannot delete or edit existing text.

Validating and protecting the data

During the import you can check that values imported into Selected from Code List type fields are valid; any invalid values are reported as errors during the import, and you can then correct them before re-importing the data.

You can protect existing data by turning on Do not update existing field values with blank values in order to prevent existing data from being overwritten by blank or empty values in the source data.

Comprehensive record matching also enables you to control how records are created or updated. For details, see the iBase help.

Manipulating the data

When importing data, you can transform field values in source records prior to assigning them to iBase fields in order to change the record structure and discard unwanted data. You can:
  • Copy a source field in order to assign it to more than one iBase field.
  • Merge two or more source fields in order to assign them to a single iBase field.
  • Split a source field in order to assign parts of the field value to several iBase fields.
  • Update specified values with new ones (for example where the source data uses a different code list) by creating a substitution file.
  • Trim unwanted space from the start or end of a source field.

Recording the results of the import

You can record the results of the import by saving new and modified records in a set. If required, you can also log the numbers of new and modified records to a file.
Note: If you use auditing with an Access database and the audit level is set to 4 then the audit log will record only the start and end of the import. It does not log the individual records.

Handling errors

You can save the errors that may occur during the import to a file. You can then fix the problems with the source data by editing the error file, and re-import the remainder of the data using the error file as the source for the import. For further details, see the iBase help.

Importing into a database with case control

If your database is case enabled, you have to specify the case into which you want to import the data.

You can only import data into one case.

When you run the import, the Select Case dialog is displayed. Select a single case to which all the imported records will be added.

Matching during the import will only be carried out against records in the specified case.

Bulk imports and XML imports

A bulk import allows significantly faster importing of large quantities of data, including XML data, without user intervention. You set up a bulk import in the same way as any other import, using an import specification, although there are a few minor differences between a standard and a bulk import.

For further details, see:
Note: After importing large numbers of records, you may want to compact your database.