-
|
This doc indicates that a workaround is required to enable lineage tracking when running Notebooks on Databricks (adding a couple lines of Scala to the top of each notebook). Is something similar required when running PySpark jobs (via "Python" or "Spark Submit" jobs) on Databricks? I'll be attempting this in the next couple weeks, just wondering how easy/hard/impossible this is going to be... |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
The mentioned workaround is related the Databricks notebook. When using |
Beta Was this translation helpful? Give feedback.
-
|
Excellent, thanks for the quick response. |
Beta Was this translation helpful? Give feedback.
The mentioned workaround is related the Databricks notebook. When using
pyspark,spark-shellorspark-submitthere shouldn't be any issues.