Skip to content

Commit

Permalink
Add CIROH JupyterHub File System and Storage documentation (#191)
Browse files Browse the repository at this point in the history
  • Loading branch information
arpita0911patel authored Aug 16, 2024
2 parents 0177ba7 + c257840 commit eb7a294
Show file tree
Hide file tree
Showing 5 changed files with 92 additions and 7 deletions.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
sidebar_position: 3
title: "Instructions for requesting custom images"
sidebar_position: 4
title: "Request custom images"
description: "2i2c JupyterHub is a cloud-based JupyterHub environment specifically designed for hydrological researchers. It is powered by 2i2c JupyterHub, a cloud-based JupyterHub environment specifically on Google Cloud"
tags:
- 2i2c
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
sidebar_position: 1
title: "JupyterHub User Directory"
description: "2i2c JupyterHub File System"
tags:
- 2i2c
- JupyterHub
- GCP
- notebook
- directory
- filesystem
---

This is a guide for understanding the File System in CIROH JupyterHub. You can find detailed explanation on [2i2c docs site](https://docs.2i2c.org/user/topics/data/filesystem/).

### 1. `/home/jovyan`
This is your home directory and is same for all JupyterHub users. **Only you can access files in your home directory.** Any files you place in your home directory persists between sessions. It is recommended to use only for notebooks and code since it is not suitable for large datasets.

### 2. `/home/jovyan/shared`
This is the shared **readonly** directory. All users can access and read from the shared directory. Only the hub admins can add and delete data from this directory.

### 3. `/tmp`
This is a non persistient directory. This means any files you add under /tmp direcotry will be deleted once you log out. This directory can be used to store data temporary data.

Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
sidebar_position: 2
title: "Manage files in GCP bucket"
description: "2i2c JupyterHub Google Cloud Buckets"
tags:
- 2i2c
- JupyterHub
- GCP
- notebook
- custom images
- tutorial
---

This guide is for managing objects in GCP buckets available on 2i2c CIROH JupyterHub. For more detailed explanation, you can visit [2i2c docs site](https://docs.2i2c.org/user/topics/data/object-storage/manage-object-storage-gcp/).

### 1. Overview
CIROH JupyterHub uses object Google Cloud Storage to store data in buckets (containers for objects). Currently, there are two buckets available to use on CIROH JupyterHub.
- **Scratch Buckets**: It is intended for storing temporary files since any files in scratch buckets get deleted after seven days. Open a terminal in CIROH JupyterHub and run this command to display your scratch bucket name:

```
echo $SCRATCH_BUCKET
gs://awi-ciroh-scratch/<username>
```
- **Persistent Buckets**: It is recommended to use for storing files that you will be using for a longer period of time. Open a terminal in CIROH JupyterHub and run this command to display your persistent bucket name:

```
echo $PERSISTENT_BUCKET
gs://awi-ciroh-persistent/<username>
```

### 2. Copying file to a bucket
You can copy files on your CIROH JupyterHub to an available bucket using the following command.

```
gcloud storage ls cp <filepath> $PERSISTENT_BUCKET/<filepath>
```

### 3. Copying file from a bucket to CIROH JupyterHub
You can copy files from an accessible bucket to your CIROH JupyterHub using the following command.

```
gcloud storage ls cp $PERSISTENT_BUCKET/<filepath> <destination-filepath>
```

### 4. Deleting file from a bucket
You can delete a file in a bucket with the following command:

```
gcloud storage ls rm $PERSISTENT_BUCKET/<filepath>
```


:::note
Anyone can access each other's files in persistent buckets on the hub. Please be careful not to delete other user's files. Your actions impact the entire organization's storage. If unsure, consult with the team lead or ciroh IT support.
:::

## Where to go for help:

- Email ciroh-it-admin@ua.edu UA CIROH Cloud Team
- CIROH Cloud Slack Channel - #ciroh-ua-it-admin
- CIROH Infrastructure Support Slack Channel - #ciroh-infrastructure-support
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
sidebar_position: 2
title: "Push from 2i2c JupyterHub to GitHub"
sidebar_position: 3
title: "Push and Pull to GitHub"
description: "2i2c JupyterHub is a cloud-based JupyterHub environment specifically designed for hydrological researchers. It is powered by 2i2c JupyterHub, a cloud-based JupyterHub environment specifically on Google Cloud"
tags:
- 2i2c
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
sidebar_position: 1
title: "Documentation and Tutorials"
sidebar_position: 5
title: "Documentation and Tutorial"
description: "2i2c JupyterHub is a cloud-based JupyterHub environment specifically designed for hydrological researchers. It is powered by 2i2c JupyterHub, a cloud-based JupyterHub environment specifically on Google Cloud"
tags:
- 2i2c
Expand Down Expand Up @@ -28,7 +28,7 @@ tags:
[2i2c Infrastructure Docs](https://infrastructure.2i2c.org/)
:::

### awi-ciroh-image repository on 2i2c GitHub:
### awi-ciroh-image repository on CIROH GitHub:

:::info
[Github URL for AWI-CIROH 2i2c Hub Image](https://github.com/2i2c-org/awi-ciroh-image)
Expand Down

0 comments on commit eb7a294

Please sign in to comment.