Give me a specific example for system.destination.identityPartitioningEnabled ,thanks about brooklin HOT 4 CLOSED

Don't know how to add it, in this place :
Datastream.pdsc
{
"name": "metadata",
"doc": "Generic metadata for Datastream (e.g. owner, expiration, etc). Metadata is stored as user defined name/value pair.",
"type": {
"type": "map",
"values": "string"
},
"optional": true
}

from brooklin.

You can set this option in datastream metadata, e.g.

bin/brooklin-rest-client.sh -o CREATE -m '{"system.destination.identityPartitioningEnabled":"true","owner":"test-user","system.reuseExistingDestination":"false"}' -u http://localhost:32311/ -n first-mirroring-stream -s "kafka://localhost:9093/^(first|second)-topic$" -c kafkaMirroringConnector -t kafkaTransportProvider 2>/dev/null

from brooklin.

My test steps are as follows: first, create a topic with three partitions in the source kafka and execute it as you do："./brooklin-rest-client.sh -o CREATE -u http://localhost:32311/ -n first-mirroring-stream-nine -s "kafka://10.124.201.51:9092,10.124.201.52:9092,10.124.201.53:9092/bigdata_test_topic_nine" -c kafkaMirroringConnector -t kafkaTransportProvider -m '{"system.destination.identityPartitioningEnabled":"true","owner":"test-user","system.reuseExistingDestination":"false"}' 2>/dev/null
",

but the topic of the target kafka has only one partition.
Should three partitions be created in advance in the destination kafka?

from brooklin.

Yes, topic needs to be created with correct partition count in advance.

from brooklin.