An example explaining CDC (change data capture) use case with SnappyData streaming sink.
For CDC use case following two conditions should match:
1) The target table must be defined with key columns (for column tables) or primary keys ( for
row table).
2) The input dataset must have an numeric column with name _eventType indicating type of the
event. The value of this column is mapped with event type in the following manner:
0 - insert
1 - putInto
2 - delete
Based on the key values in the incoming dataset and the value of _eventType column the sink
will decide which operation need to be performed for each record.
To run this on your local machine, you need to first run a Netcat server:
$ nc -lk 9999
Example input data. Note that the last value from CSV record indicates the _eventType:
An example explaining CDC (change data capture) use case with SnappyData streaming sink.
For CDC use case following two conditions should match: 1) The target table must be defined with key columns (for column tables) or primary keys ( for row table). 2) The input dataset must have an numeric column with name
_eventType
indicating type of the event. The value of this column is mapped with event type in the following manner:0 - insert 1 - putInto 2 - delete
Based on the key values in the incoming dataset and the value of
_eventType
column the sink will decide which operation need to be performed for each record.To run this on your local machine, you need to first run a Netcat server:
$ nc -lk 9999
Example input data. Note that the last value from CSV record indicates the
_eventType
:1,user1,23,0 2,user2,45,0 1,user1,23,2 2,user2,46,1
To run the example in local mode go to your SnappyData product distribution directory and execute the following command:
bin/run-example snappydata.structuredstreaming.CDCExample