Skip to content

Commit

Permalink
Merge pull request #115 from PermanentOrg/per-9673_add_record_thumbna…
Browse files Browse the repository at this point in the history 
…il_handler

Add lambda handler for attaching thumbnails to records
  • Loading branch information
@liam-lloyd
liam-lloyd authored Aug 28, 2024
2 parents 2f255a6 + 748a973 commit ace41c1
Show file tree
Hide file tree
Showing 23 changed files with 3,118 additions and 1,371 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,5 @@ jobs:
- run: (cd stela/packages/api; docker compose run stela npm run test-ci)
- run: (cd stela; npm run test -w @stela/account_space_updater)
- uses: codecov/codecov-action@v2
- run: (cd stela; npm run test -w @stela/record_thumbnail_attacher)
- uses: codecov/codecov-action@v2
29 changes: 29 additions & 0 deletions Dockerfile.record_thumbnail_attacher
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
FROM public.ecr.aws/lambda/nodejs:18 as builder
WORKDIR /usr/local/apps/stela/

COPY package.json ./
COPY tsconfig.build.json ./
COPY tsconfig.json ./
COPY jest.config.js ./
COPY packages ./packages

RUN npm install -g npm@8.19.3
RUN npm install
RUN npm install -ws
RUN npm run build -ws


FROM public.ecr.aws/lambda/nodejs:18 as final
WORKDIR ${LAMBDA_TASK_ROOT}

COPY --from=builder /usr/local/apps/stela/packages/record_thumbnail_attacher/dist ./packages/record_thumbnail_attacher/dist
COPY --from=builder /usr/local/apps/stela/packages/record_thumbnail_attacher/package.json ./packages/record_thumbnail_attacher/package.json
COPY --from=builder /usr/local/apps/stela/packages/logger/dist ./packages/logger/dist
COPY --from=builder /usr/local/apps/stela/packages/logger/package.json ./packages/logger/package.json
COPY --from=builder /usr/local/apps/stela/package.json ./package.json
COPY --from=builder /usr/local/apps/stela/package-lock.json ./package-lock.json

RUN npm install -g npm@8.19.3
RUN npm install --workspace @stela/record_thumbnail_attacher

CMD ["packages/record_thumbnail_attacher/dist/index.handler"]
82 changes: 59 additions & 23 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,31 +30,34 @@ npm install -ws
Depending on the work being done, some environment variable will not be required for the service to run.
For these, simply fill in any fake value to prevent `require-env-variable` from throwing errors.

| Variable | Default | Notes |
| --------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| ENV | local | Tells stela what environment it's running in |
| DATABASE_URL | postgres://postgres:permanent@database:5432/permanent | Run tests to generate default database |
| PORT | 8080 | Tells stela what port to run on |
| FUSIONAUTH_HOST | <none. needs to be set> | Fusionauth's host URL. Should be different between prod and other envs. |
| FUSIONAUTH_API_KEY | <none. needs to be set> | Find it in Fusionauth admin panel -> settings -> API keys -> the one called "back-end (local)" |
| FUSIONAUTH_TENANT | <none. needs to be set> | Find it in Fusionauth admin panel -> Tenants -> the one called "Local" |
| FUSIONAUTH_BACKEND_APPLICATION_ID | <none. needs to be set> | Find it in Fusionauth admin panel -> Applications -> the one called "back-end (local)" |
| FUSIONAUTH_ADMIN_APPLICATION_ID | <none. needs to be set> | Find it in Fusionauth admin panel -> Applications -> the one called "admin-local" |
| Variable | Default | Notes |
| --------------------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| ENV | local | Tells stela what environment it's running in |
| DATABASE_URL | postgres://postgres:permanent@database:5432/permanent | Run tests to generate default database |
| PORT | 8080 | Tells stela what port to run on |
| FUSIONAUTH_HOST | <none. needs to be set> | Fusionauth's host URL. Should be different between prod and other envs. |
| FUSIONAUTH_API_KEY | <none. needs to be set> | Find it in Fusionauth admin panel -> settings -> API keys -> the one called "back-end (local)" |
| FUSIONAUTH_TENANT | <none. needs to be set> | Find it in Fusionauth admin panel -> Tenants -> the one called "Local" |
| FUSIONAUTH_BACKEND_APPLICATION_ID | <none. needs to be set> | Find it in Fusionauth admin panel -> Applications -> the one called "back-end (local)" |
| FUSIONAUTH_ADMIN_APPLICATION_ID | <none. needs to be set> | Find it in Fusionauth admin panel -> Applications -> the one called "admin-local" |
| LEGACY_BACKEND_HOST_URL | http://load_balancer:80/api |
| LEGACY_BACKEND_SHARED_SECRET | none | Can be found in `back-end`'s library/base/constants/base.constants.php |
| MAILCHIMP_API_KEY | none | Can be found in `back-end`'s library/base/constants/base.constants.php |
| MAILCHIMP_TRANSACTIONAL_API_KEY | none | Can be found in `back-end`'s library/base/constants/base.constants.php, where it is called `MANDRILL_API_KEY` |
| LEGACY_BACKEND_SHARED_SECRET | none | Can be found in `back-end`'s library/base/constants/base.constants.php |
| MAILCHIMP_API_KEY | none | Can be found in `back-end`'s library/base/constants/base.constants.php |
| MAILCHIMP_TRANSACTIONAL_API_KEY | none | Can be found in `back-end`'s library/base/constants/base.constants.php, where it is called `MANDRILL_API_KEY` |
| MAILCHIMP_DATACENTER | us12 |
| MAILCHIMP_COMMUNITY_LIST_ID | 2736f796db | The default value corresponds to the `dev` list |
| SENTRY_DSN | none | Can be found in Sentry under Projects > stela > Settings > Client Keys (DSN) |
| DEV_NAME | none | Only set in local environments. Should be your given name. all lowercase. Used to create Sentry envs for developers |
| AWS_REGION | us-west-2 | |
| AWS_ACCESS_KEY_ID | none | The same one you use in `devenv` |
| AWS_SECRET_ACCESS_KEY | none | The same one you use in `devenv` |
| LOW_PRIORITY_TOPIC_ARN | test | Doesn't need to be set to a real ARN unless your work touches it specifically |
| MIXPANEL_TOKEN | none | Found in Mixpanel at Settings > Project Settings > Project Token |
| ARCHIVEMATICA_BASE_URL | none | It is the url of the EC2 instance on which archivematica is running |
| ARCHIVEMATICA_API_KEY | none | Found in Bitwarden, not needed unless you're running the cleanup cron |
| MAILCHIMP_COMMUNITY_LIST_ID | 2736f796db | The default value corresponds to the `dev` list |
| SENTRY_DSN | none | Can be found in Sentry under Projects > stela > Settings > Client Keys (DSN) |
| DEV_NAME | none | Only set in local environments. Should be your given name. all lowercase. Used to create Sentry envs for developers |
| AWS_REGION | us-west-2 | |
| AWS_ACCESS_KEY_ID | none | The same one you use in `devenv` |
| AWS_SECRET_ACCESS_KEY | none | The same one you use in `devenv` |
| LOW_PRIORITY_TOPIC_ARN | test | Doesn't need to be set to a real ARN unless your work touches it specifically |
| MIXPANEL_TOKEN | none | Found in Mixpanel at Settings > Project Settings > Project Token |
| ARCHIVEMATICA_BASE_URL | none | It is the url of the EC2 instance on which archivematica is running |
| ARCHIVEMATICA_API_KEY | none | Found in Bitwarden, not needed unless you're running the cleanup cron |
| CLOUDFRONT_URL | none | Can be found as `CDN_URL` in `back-end`'s library/base/constants/base.constants.php. Not required for API server |
| CLOUDFRONT_KEY_PAIR_ID | none | Can be found as `CLOUDFRONT_KEYPAIR` in `back-end`'s library/base/constants/base.constants.php. Not required for API server |
| CLOUDFRONT_PRIVATE_KEY | none | Can be found in `back-end`'s library/static/certs/pk-APKAJP2D34UGZ6IG443Q.pem. Not required for API server |

## Linting

Expand Down Expand Up @@ -113,6 +116,39 @@ Outside a container: Run
npm run start -w @stela/api
```

## Running Lambdas Locally

1. Build the lambda image

```bash
docker build --platform linux/amd64 -t <LAMBDA NAME>:test -f Dockerfile.<LAMBDA NAME> .
```

2. Run the lambda container. Add additional --env arguments as needed. Note that the `CLOUDFRONT_PRIVATE_KEY` required
by the `record_thumbnail_attacher` must include literal newlines; docker does not appear to interpret `\n` correctly.

```bash
docker run --platform linux/amd64 -p 9001:8080 --env DATABASE_URL=postgres://postgres:permanent@database:5432/permanent <LAMBDA NAME>:test
```

3. Find the container name

```bash
docker ps
```

4. Connect to the local env docker network

```bash
docker network connect devenv_default <YOUR CONTAINER NAME>
```

5. Trigger the lambda

```bash
curl "http://localhost:9001/2015-03-31/functions/function/invocations" -d '<YOUR PAYLOAD>'
```

## Deployment

### To `dev`
Expand Down
Loading

0 comments on commit ace41c1

Please sign in to comment.