The rooster crows immediately before sunrise, the rooster causes the sun to rise.

Using GraphX to detect possible causes and effects between time related events. We observe a true causation signal by generating random correlations over same events at different time and back propagate these scores to their most connected events. Finally, we extract the most probable causes and effects together with a score of aggressiveness (how likely an event could explain downstream effects) and sensitivity (how likely an event results from an upstream cause).

Getting Started

Pathogen project is built for Scala 2.11.x and Spark 2.1.0.

Maven

Pathogen is available on Maven Central. Add below dependency to your pom.xml file

<dependency>
  <groupId>io.pathogen.spark</groupId>
  <artifactId>pathogen-core</artifactId>
  <version>x.y.z</version>
</dependency>

SBT

If you are using SBT, simply add the following to your build.sbt file

libraryDependencies ++= Seq(
  "io.pathogen.spark" % "pathogen-core" % "x.y.z"
)

Spark Packages

Available as a spark package, include this package in your Spark Applications as follows

> $SPARK_HOME/bin/spark-shell --packages io.pathogen.spark:pathogen-core:x.y.z

Authors

Antoine Amend - [antoine.amend@gmail.com]

License

Apache License, version 2.0