<file pattern>must be local on the server. The file pattern can contain wildcards if you want to load multiple files. In addition to CSV, TSV, and TXT files, you can import compressed files in TAR, ZIP, 7-ZIP, RAR, GZIP, BZIP2, or TGZ format.
omnisql) but it is significantly slower. For large files, OmniSci recommends that you first
scpthe file to the server, and then issue the COPY command.
COPY FROMto import geo files. You can create the table based on the source file and then load the data:
WITHoptions are available for geo file imports from all sources.
geo='true'option. When importing WKT strings from CSV files, you must create the table first. The geo column type and encoding are specified as part of the DDL. For example, for a polygon with no encoding, try the following:
SQLImporteris a Java utility run at the command line. It runs a SELECT statement on another database through JDBC and loads the result set into OmniSciDB.
SQLImportercreates it. If the target table in OmniSciDB does not match the SELECT statement metadata,
SQLImportertruncates the table in OmniSciDB before transferring the data. If the truncate flag is not used,
SQLImporterappends the results of the SQL statement to the target table in OmniSciDB.
-iargument provides a path to an initialization file. Each line of the file is sent as a SQL statement to the remote database. You can use
-ito set additional custom parameters before the data is loaded.
COPY FROMstatement to import files stored on Amazon Web Services Simple Storage Service (AWS S3) into a OmniSci table, in much the same way you would with local files. In the
WITHclause, specify the S3 credentials and region information of the bucket accessed.
WITHclause of a
COPY FROMstatement; for example:
tripsis created with the following statement:
KafkaImporteron the command line:
KafkaImporterto use your target table.
KafkaImporterlistens to a pre-defined Kafka topic associated with your table. You must create the table before using the
KafkaImporterutility. For example, you might have a table named
customer_site_visit_eventsthat listens to a topic named
KafkaImporterlistens to the topic, validates records against the target schema, and ingests topic batches of your designated size to the target table. Rejected records use the existing reject reporting mechanism. You can start, shut down, and configure
KafkaImporterindependent of the OmniSciDB engine. If KafkaImporter is running and the database shuts down, KafkaImporter shuts down as well. Reads from the topic are nondestructive.
KafkaImporteris not responsible for event ordering; a streaming platform outside OmniSci (for example, Spark streaming, flink) should handle the stream processing. OmniSci ingests the end-state stream of post-processed events.
KafkaImporterdoes not handle dynamic schema creation on first ingest, but must be configured with a specific target table (and its schema) as the basis. There is a 1:1 correspondence between target table and topic.
StreamImporteron the command line.
StreamImporterto use your target table.
StreamImporterlistens to a pre-defined data stream associated with your table. You must create the table before using the
StreamImporterlistens to the stream, validates records against the target schema, and ingests batches of your designated size to the target table. Rejected records use the existing reject reporting mechanism. You can start, shut down, and configure
StreamImporterindependent of the OmniSciDB engine. If StreamImporter is running but the database shuts down, StreamImporter shuts down as well. Reads from the stream are non-destructive.
StreamImporteris not responsible for event ordering - a first class streaming platform outside OmniSci (for example, Spark streaming, flink) should handle the stream processing. OmniSci ingests the end-state stream of post-processed events.
StreamImporterdoes not handle dynamic schema creation on first ingest, but must be configured with a specific target table (and its schema) as the basis.
--connectparameter is the address of a valid JDBC port on your OmniSci instance.