4

I want to read meta data for transaction (which is supported in Kafka 0.11.0.1) so that I can figure out that whether the transaction for a particular transactional ID has been committed or not. Currently I am getting the key and value from _transactional_state topic but it is in some encoded format. Below are some same key/value I received when I polled __transaction_state topic: key = 10000000mmm, value = � �����+'���������)

Shrinivas Kulkarni
  • 492
  • 2
  • 5
  • 17

2 Answers2

5

just like what has been done in Kafka how to read from __consumer_offsets topic

echo "exclude.internal.topics=false" > consumer.config

kafka-console-consumer --consumer.config consumer.config --formatter "kafka.coordinator.transaction.TransactionLog\$TransactionLogMessageFormatter" --bootstrap-server localhost:9092 --topic __transaction_state --from-beginning

Chris
  • 71
  • 1
  • 1
4

You can look to source code of TransactionLogMessageParser class inside kafka/tools/DumpLogSegments.scala file as an example. It uses readTxnRecordValue function from TransactionLog class. The first argument for this function could be retrieved via readTxnRecordKey function of the same class.

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • Hey @alex, thanks for help. I got the required code will try to integrate it with my module. – Shrinivas Kulkarni Dec 06 '17 at 10:51
  • You can import and use `kafka.coordinator.transaction.TransactionLog.readTxnRecordValue` directly by depending on the `org.apache.kafka:kafka_2.11` artifact. – Raman Dec 18 '18 at 05:42