-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #19 from chris9692/master
Add avro converter format properties and job commit policy check
- Loading branch information
Showing
8 changed files
with
158 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# converter.avro.date.format.md | ||
|
||
**Tags**: | ||
[gobblin](categories.md#gobblin-properties) | ||
|
||
**Type**: string | ||
|
||
**Default value**: none | ||
|
||
**Related**: | ||
|
||
## Description | ||
|
||
`converter.avro.date.format` indicates how date values are formatted in the user data. This property | ||
is used by the JSON to AVRO converter in converting fields of type "date". | ||
|
||
This property accepts multiple formats, separated by comma (,), if date values come in with several forms. | ||
|
||
For example: | ||
- `converter.avro.date.format=MM/dd/yyyy HH:mm,dd-MMM-yyyy HH:mm:ss` | ||
|
||
[back to summary](summary.md#essential-gobblin-core-properties) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# converter.avro.time.format.md | ||
|
||
**Tags**: | ||
[gobblin](categories.md#gobblin-properties) | ||
|
||
**Type**: string | ||
|
||
**Default value**: none | ||
|
||
**Related**: | ||
|
||
## Description | ||
|
||
`converter.avro.time.format` indicates how time values are formatted in the user data. This property | ||
is used by the JSON to AVRO converter in converting fields of type "time". | ||
|
||
This property accepts multiple formats, separated by comma (,), if time values come in with several forms. | ||
|
||
For example: | ||
- `converter.avro.time.format=HH:mm:ss,HH:mm:ss.000'Z'` | ||
|
||
[back to summary](summary.md#essential-gobblin-core-properties) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# converter.avro.timestamp.format.md | ||
|
||
**Tags**: | ||
[gobblin](categories.md#gobblin-properties) | ||
|
||
**Type**: string | ||
|
||
**Default value**: none | ||
|
||
**Related**: | ||
|
||
## Description | ||
|
||
`converter.avro.timestamp.format` indicates how timestamp values are formatted in the user data. This property | ||
is used by the JSON to AVRO converter in converting fields of type "timestamp". | ||
|
||
This property accepts multiple formats, separated by comma (,), if timestamp values come in with several forms. | ||
|
||
For example: | ||
- `converter.avro.timestamp.format=MM/dd/yyyy HH:mm,dd-MMM-yyyy HH:mm:ss` | ||
|
||
[back to summary](summary.md#essential-gobblin-core-properties) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# job.commit.policy | ||
|
||
**Tags**: | ||
[gobblin](categories.md#gobblin-properties) | ||
|
||
**Type**: string | ||
|
||
**Default value**: full | ||
|
||
**Related**: | ||
|
||
## Description | ||
|
||
`job.commit.policy` specifies how the job state will be committed when some of its tasks failed. Valid values are: | ||
- full: Commit output data of a job if and only if all of its tasks successfully complete. | ||
- successful: Commit output data of tasks that successfully complete. | ||
- partial: Deprecated, the replacement is "successful" | ||
|
||
[back to summary](summary.md#essential-gobblin-core-properties) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters