Advanced Features & Extensions

📄️ User Defined Processors

By customizing the node function, you have the flexibility to organize your JavaScript script into reusable processing nodes. Once created, these custom nodes can be easily referenced in your data transformation tasks without the need for rewriting the script. This significantly reduces the development workload.

📄️ Manage External Storage

To facilitate the quick reading of task-related information subsequently, TapData stores necessary configurations, incremental logs of source tables, and other information related to the task in its internal MongoDB database. To store more data, you can create an external database to store relevant data.

📄️ Manage Functions

TapData supports a wide range of functions to facilitate the definition of processing steps, allowing for their use in JavaScript (JS) nodes. Additionally, you can freely define custom functions or import third-party JAR packages to introduce new functions as needed.

📄️ Create Live Cache

Live Cache is primarily designed to alleviate the pressure on the source database by multiple tasks processing some hot data. By placing these data in the cache, it can be used by multiple tasks.

📄️ Manage CDC Log Cache

To alleviate the pressure on the source database during increments, TapData supports shared mining of change data capture (CDC) logs. Once the CDC Log Cache is activated, it will not start mining immediately. Instead, it begins when you create a task for the table belonging to that data source. Regardless of whether the mining task is paused or encounters errors, it will not affect the normal operation of the synchronization task.