1

I am totally a squat in Kafka land If I run the command 

/cfintools/confluent-4.0.0/bin/kafka-avro-console-consumer --topic $t  --bootstrap-server  $bt  --consumer.config /cfintools/avro_consumer_ssl_cfin_prod.properties --property schema.registry.url=$regd --from-beginning

the thing just takes forever to return. No error but no data either.
All I want is the latest 10 messages in that topic and I read about partition and offsets . Now I dont know how n the world should I go about getting this info Some other place I read about this command 

./bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group count_errors --describe

But I dont know my consumer group. I just know a topic name and other parameters mentioned in the 1st command So with this ltd info how do I get the last 10 odd messages in the topic searching only a specific partition and offset or any other way

I tried to use the kt tool 

$  ./kt topic -brokers $bk -filter $t -partitions -leaders -replicas
failed to create client err=kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

I know the brokers are valid because I m using them

What I want most kindly :

  • get partition and offsets in most straightforward manner without any additional info like consumer group
  • Any other approach to automatically search last 10 messages - that partition and offset and fetch
  • any GUI based tool that will prevent me from becoming mad
user1874594
  • 2,277
  • 1
  • 25
  • 49
  • If you want to consume from a topic, you have to have a consumer group. If you don't have any ACL in your cluster you can try `test-consumer-group` – Amin Feb 23 '19 at 08:57
  • I guess I am a little ignorant about this so please help me out get me a clear picture so when we r weaving the contents of a topic we do not use the group parameter we need the bootstrap topic name et cetera but with this command (one that ends in from the beginning) I don't know how many partitions are there and they are offsets ( so I can query just that one partition that has the latest data. ) ... show with the same extent of information that is the topic name bootstrap-server and schema registry I wanted to know how I can get the partition name and offset that is what I am trying to get – user1874594 Feb 23 '19 at 13:38
  • if I can run consumer avro command that you just saw it means I should have a consumer group? then how do I find it out – user1874594 Feb 23 '19 at 13:56
  • [this can help you to see consumer groups](https://stackoverflow.com/a/41645130/5687237) but remember, consumer avro command automatically sets consumer group – Amin Feb 23 '19 at 14:18

0 Answers0