bash - Check whether a Kafka topic exists in Python -

i want create kafka topic if not exist. know how create topic via bash, don't know how check whether exists.

topic_exists = ?????? if not topic_exists:     subprocess.call([os.path.join(kafkabin, 'kafka-topics.sh'),         '--create',           '--zookeeper', '{}:2181'.format(kafkahost),         '--topic', str(self.topic),          '--partitions', str(self.partitions),         '--replication-factor', str(self.replication_factor)]) 

you can use --list (list available topics) option kafka-topics.sh , see if self.topic exists in topics array, shown below.

depending on number of topics have approach might bit heavy. if case, might able away using --describe (list details given topics) likely return empty if topic doesn't exist. haven't thoroughly tested this, can't sure how solid solution (--describe) is, might worth investigate bit further.

wanted_topics = ['host_updates_queue', 'foo_bar']  topics = subprocess.check_output([os.path.join(kafkabin, 'kafka-topics.sh'),         '--list',         '--zookeeper', '{}:2181'.format(kafkahost)])  wanted in wanted_topics:     if wanted in topics:         print '\'{}\' topic exists!'.format(wanted)     else:         print '\'{}\' topic not exist!'.format(wanted)      topic_desc = subprocess.check_output([os.path.join(kafkabin, 'kafka-topics.sh'),         '--describe',         '--topic', wanted,         '--zookeeper', '{}:2181'.format(kafkahost)])      if not topic_desc:         print 'no description found topic \'{}\''.format(wanted) 

output:

root@dev:/opt/kafka/kafka_2.10-0.8.2.1# ./t.py 'host_updates_queue' topic exists! 'foo_bar' topic not exist! no description found topic 'foo_bar' 

there broker configuration available don't have take of these steps:

auto.create.topics.enable | true | enable auto creation of topic on server. if set true attempts produce data or fetch metadata non-existent topic automatically create default replication factor , number of partitions.

i take approach if possible.

note should set topic configs (server.properties) on broker num.partitions , default.replication.factor match settings in code snippet.