MemSQL Fast Loader (deprecated)
Supported pipeline types:
|
In pipelines that insert data into MemSQL or MySQL, you can use the MemSQL Fast Loader destination rather than the JDBC Producer destination for faster performance. In pipelines that update or delete data, however, you must use a JDBC Producer destination. The MemSQL Fast Loader destination does not process CDC records; the destination applies error handling to records with a CDC operation in a header attribute.
To configure a MemSQL Fast Loader destination, you specify connection information, table name, and optionally define field mappings and other properties that your driver requires. By default, the MemSQL Fast Loader destination writes data to the table based on the matching field names. You can override the default field mappings by defining specific mappings. To use a JDBC version older than 4.0, you can specify the driver class name and define a health check query.
Before you use the MemSQL Fast Loader destination, you must install the MemSQL stage library and complete the other prerequisite tasks. The MemSQL stage library is an Enterprise stage library. Releases of Enterprise stage libraries occur separately from Data Collector releases. For more information, see Enterprise Stage Libraries in the Data Collector documentation.
Prerequisites
- Install the MemSQL stage library.
- To use MemSQL Fast Loader with either a MySQL or MemSQL database, install the JDBC driver for MySQL.
-
To use MemSQL Fast Loader with a MySQL database, you must enable local data loading in MySQL. See the MySQL topic, Security Issues with LOAD DATA LOCAL.
Install the MemSQL Stage Library
You must install the MemSQL stage library before using the MemSQL Fast Loader destination.
You can install Enterprise stage libraries using Package Manager for a tarball Data Collector installation or as custom stage libraries for a tarball, RPM, or Cloudera Manager Data Collector installation.
Supported Versions
Data Collector Version | Supported Stage Library Version |
---|---|
Data Collector 3.8.x and later | MemSQL Enterprise Library 1.0.1 |
Data Collector 3.7.x | MemSQL Enterprise Library 1.0.0 |
Installing with Package Manager
You can use Package Manager to install the MemSQL stage library on a tarball Data Collector installation.
Installing as a Custom Stage Library
You can install the MemSQL Enterprise stage library as a custom stage library on a tarball, RPM, or Cloudera Manager Data Collector installation.
Installing the JDBC Driver for MemSQL Fast Loader
Configuring a MemSQL Fast Loader Destination
Before you use MemSQL Fast Loader destination in a pipeline, complete several prerequisite tasks.