Spark Streaming Kafka Integration direct Approach EOFException -
when run spark streaming example org.apache.spark.examples.streaming.javadirectkafkawordcount,i caught eofexception follow,how can resolve it
exception in thread "main" org.apache.spark.sparkexception: java.io.eofexception: received -1 when reading channel, socket has been closed. java.io.eofexception: received -1 when reading channel, socket has been closed. java.io.eofexception: received -1 when reading channel, socket has been closed. @ org.apache.spark.streaming.kafka.kafkautils$$anonfun$createdirectstream$2.apply(kafkautils.scala:413) @ org.apache.spark.streaming.kafka.kafkautils$$anonfun$createdirectstream$2.apply(kafkautils.scala:413) @ scala.util.either.fold(either.scala:97) @ org.apache.spark.streaming.kafka.kafkautils$.createdirectstream(kafkautils.scala:412) @ org.apache.spark.streaming.kafka.kafkautils$.createdirectstream(kafkautils.scala:528) @ org.apache.spark.streaming.kafka.kafkautils.createdirectstream(kafkautils.scala)
the direct stream uses low level kafka consumer underneath therefore 1 needs provide list of brokers. haven't may want set metadata.broker.list property in form bhost1:9092,bhost2:9092,...,bhostn:9092.
see kafka simpleconsumer cannot connect zookeeper : received -1 when reading channel.
Comments
Post a Comment