Skip to content

Latest commit

 

History

History
17 lines (11 loc) · 670 Bytes

README.md

File metadata and controls

17 lines (11 loc) · 670 Bytes
Raw

confluent-platform-spark-streaming

Example of consuming Avro data encoded with Confluent serialziers from Kafka with Spark Streaming.

Build it

$ sbt assembly

Running it

Export env variables for required configuration.

$ export KAFKA_TOPIC=example_topic
$ export BOOTSTRAP_SERVERS=example.server1.net:9092,example.server2.net:9092,example.server3.net:9092
$ export SCHEMA_REGISTRY_URL=http://kafka-schema.registry.net:8081

Spark submit it

$ spark-submit --class example.StreamingJob --driver-java-options "-Dconfig.file=conf/application.conf -Dlog4j.configuration=file:conf/log4j.properties" target/scala-2.10/StreamingExample-assembly-1.0.jar