Skip to content

Spark client update to use the new lakefs sdk package #9218

Spark client update to use the new lakefs sdk package

Spark client update to use the new lakefs sdk package #9218

Triggered via pull request October 9, 2023 22:27
Status Success
Total duration 11m 55s
Artifacts 3

esti.yaml

on: pull_request
Check if secrets are available.
2s
Check if secrets are available.
Generate code from latest lakeFS app
12s
Generate code from latest lakeFS app
Build lakeFS HadoopFS
55s
Build lakeFS HadoopFS
Prepare Spark Apps
1m 49s
Prepare Spark Apps
Matrix: Build metadata client for Spark 3.x
Build and push rclone export Docker image
30s
Build and push rclone export Docker image
login-to-amazon-ecr
7s
login-to-amazon-ecr
Build and push Docker image
2m 2s
Build and push Docker image
Matrix: Test lakeFS metadata client export with Spark 3.x
Test lakeFS rclone export functionality
1m 19s
Test lakeFS rclone export functionality
Test lakeFS Hadoop FileSystem
5m 16s
Test lakeFS Hadoop FileSystem
Test lakeFS multipart upload with Hadoop S3A
2m 45s
Test lakeFS multipart upload with Hadoop S3A
Test unified gc
2m 30s
Test unified gc
Test metastore client commands using trino
1m 55s
Test metastore client commands using trino
Run latest lakeFS app on AWS S3 DynamoDB KV
4m 43s
Run latest lakeFS app on AWS S3 DynamoDB KV
Run latest lakeFS app on AWS S3
4m 42s
Run latest lakeFS app on AWS S3
Run latest lakeFS app on Google Cloud Platform and Google Cloud Storage
5m 10s
Run latest lakeFS app on Google Cloud Platform and Google Cloud Storage
Run latest lakeFS app on Azure with Azure blobstore
5m 40s
Run latest lakeFS app on Azure with Azure blobstore
Run latest lakeFS app on Azure with Azure Data Lake Storage Gen2 and CosmosDB
9m 13s
Run latest lakeFS app on Azure with Azure Data Lake Storage Gen2 and CosmosDB
Matrix: spark
Fit to window
Zoom out
Zoom in

Annotations

2 warnings
login-to-amazon-ecr
Your docker password is not masked. See https://github.com/aws-actions/amazon-ecr-login#docker-credentials for more information.
Build lakeFS HadoopFS
Error: Path Validation Error: Path(s) specified in the action for caching do(es) not exist, hence no cache is being saved.

Artifacts

Produced during runtime
Name Size
generated-code Expired
15.3 MB
lakefs-hadoopfs Expired
5.96 MB
spark-apps Expired
4.48 MB