Skip to content

Deployment of databricks resources with cdk

License

Notifications You must be signed in to change notification settings

dan1elt0m/databricks-cdk

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

npm version

databricks-cdk

Overview

databricks-cdk is an open source library that allows you to deploy Databricks workspace and resources on AWS using the AWS Cloud Development Kit (CDK) version 2 and TypeScript.

Prerequisites

  • AWS CLI
  • Node.js
  • AWS CDK v2
  • Databricks account
  • AWS Systems Manager (SSM) parameters:
    • /databricks/deploy/user
    • /databricks/deploy/password (secure parameter)
    • /databricks/account-id (not AWS account ID)

Installation

To install databricks-cdk, you can use npm:

npm install databricks-cdk

Usage

Here's an example of how you can use databricks-cdk to deploy a Databricks workspace:

Initialize your AWS CDK project:

cdk init

Import the DatabricksDeployLambda class into your AWS CDK app:

import {Construct} from "constructs";
import {Stack, StackProps} from "aws-cdk-lib";
import {DatabricksDeployLambda} from "databricks-cdk";

Create a Simple stack containing the DatabricksDeployLambda, and a Databricks Workspace and User.

export class SimpleStack extends Stack {

    constructor(scope: Construct, id: string, props: StackProps) {
        super(scope, id, props);

        const deployLambda = new DatabricksDeployLambda(this, "DeployLambda", {
            region: this.region,
            accountId: this.account
        });

        const credentials = deployLambda.createCredential(this, "Credentials", {
            credentialsName: "credentials-name",
            roleArn: "role-arn",
        });
        const storageConfig = deployLambda.createStorageConfig(this, "StorageConfig", {
            storageConfigName: "storage-config-name",
            bucketName: "bucket-name",
        });

        const workspace = deployLambda.createWorkspace(this, "Workspace", {
            workspaceName: "databricks-workspace",
            awsRegion: this.region,
            credentialsId: credentials.credentialsId(),
            storageConfigurationId: storageConfig.storageConfigId(),
        });

        deployLambda.createUser(this, "UserTest", {
            workspaceUrl: workspace.workspaceUrl(),
            userName: "test@test.com",
        });


    }
}

Create an instance of the SimpleStack class:

const app = new SimpleStack(app, "databricks-cdk");

Deploy the stack to your AWS account:

cdk deploy

Note: The AWS SSM parameters /databricks/deploy/user, /databricks/deploy/password, and /databricks/account-id are required for the deployment to succeed.

See also the simple-workspace and multi-stack examples in examples

Contributing

We welcome contributions to databricks-cdk! If you'd like to contribute, please follow these steps:

  1. Fork the repository.
  2. Create a branch for your feature or bug fix.
  3. Make your changes.
  4. Test your changes thoroughly.
  5. Submit a pull request.

License

  • databricks-cdk is licensed under the MIT License. See LICENSE for more information

About

Deployment of databricks resources with cdk

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 71.8%
  • TypeScript 27.1%
  • Other 1.1%