Data Capture Service

(WorldData DCS V9a)

 

Introduction

​​​​​Data Capture Service allows collecting data from different database management systems of the softwares in near real-time before transmitting the data collected to datawarehouse, thereby assisting the analysis and direction giving.

Data Capture Service is installed by the databases of the softwares used in the system. The software allows gathering all changes to the databases such as data adding, editing, deleting, then transfers such changes to the Datawarehouse for storage (including history data) of the database. Such stored data will be used for analyzing, planning strategies and operating the whole system. 

Features

Flexible connection with different databases

- Allow connecting to almost all databases like Relational database (MSSQL, MySQL, Oracle, etc.) and Non-relational database (MongoDB, Hbase, and so on).

- Allow configuring to collect partial or full changes to the databases.

Data connectivity via APIs    

- Connect to APIs of the softwares to collect data in standardized formats.

Flexible configuration     
- Allow configuring source databases, assisting users to choose the types for the databases that store source data, enter equivalent parameters of the connection to the database.
- Allow configuring frequency for getting samples so that to see whether the source databases’ tables are changed or not. 
- Allow configuring rules for getting samples, so that data is get from pre-assigned tables, from all tables for from some data fields in the tables. 
- Allow configuring mapping Source and Destination so that when there is new data, such data will be transferred from source data to destination data.
- Allow configuring Destination so that the software can send the data got from source databases, do the mapping and packaging before sending such data to Source.
- Allow merging, aggregating the data collected before sending to Datawarehouse.
- Allow configuring output formats, configuring source for output like via API, to relational database.
- Allow building dynamic processes for collected data streams. Create multiple data collection processes for the same database. The process is built by dragging and dropping.

Visual administration    
- Allow displaying of processes being performed, processes listening to data, processes paused.
- Allow starting or pausing data collection process on graphical interface.
- Allow setting time, frequency of getting data of changes of each data collection process.

System connection     
- Allow direct connection with databases to transfer collected data to other databases.
- Connect to the API to transfer data to another system.
- Connect with Organizations/Units platform to transfer collected data.

Model for connection/integration with other systems​​​​​​​

 

Technical solutions used

- Programming language: Java
- Database: MySQL, SQL Server, PostgreSQL
- Operating system: Windows server 2019, x64; Ubuntu

Hardware requirements

- CPU: 2 CPU cores
- Ram: 8GB
- Disk: 250GB
- OS: Ubuntu Server 18.04 x64

Price

To learn about the price, contact us here.