-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
added local minio s3 and backup:mongodb script #649
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good. I'd also like to see a restore script (with parameters for the collections to restore) so we can test this end-to-end before merge.
Also, would it be possible to include lifecycle configurations so that the backups automatically delete after a certain time frame? See https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketLifecycleConfiguration.html
backend/src/config.ts
Outdated
S3_ENDPOINT: env("S3_ENDPOINT"), | ||
S3_PORT: parseInt(env("S3_PORT")), | ||
S3_ACCESS_KEY_ID: env("S3_ACCESS_KEY_ID"), | ||
S3_SECRET_ACCESS_KEY: env("S3_SECRET_ACCESS_KEY"), | ||
S3_MONGO_BACKUP_BUCKET: env("S3_MONGO_BACKUP_BUCKET"), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe put these under an s3 key like config.mongoDB
@kevinzwang added your requested changes |
@kevinzwang should be good to go |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Resolve your conflicts and then pls squash and merge!
* added local minio s3 and backup:mongodb script * included metadata.json files to backup * added download code * added lifecycle and changed config names * removed minio container, added lifecycle rule check --------- Co-authored-by: ARtheboss <advayratan@yahoo.com>
commit 48acfe9 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Jun 25 10:56:11 2024 -0700 fix: Allow frontend to build commit 6b54422 Merge: 7e730ef 419ed5b Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Jun 25 09:43:07 2024 -0700 Merge pull request #711 from asuc-octo/frontend-rewrite Merge frontend work into GQL branch commit 419ed5b Merge: 735aaf4 7e730ef Author: Matthew Rowland <contact@mathhulk.com> Date: Mon Jun 24 16:44:56 2024 -0700 Merge branch 'gql' into frontend-rewrite commit 735aaf4 Author: Matthew Rowland <contact@mathhulk.com> Date: Mon Jun 24 11:56:26 2024 -0700 feat: Winter, GZIP compression, getCourseList commit 7e730ef Author: Max Wang <maxmwang@berkeley.edu> Date: Sun Jun 23 22:50:10 2024 -0700 feat: use redis for user session cookies (#710) commit 271469a Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Jun 18 12:58:05 2024 -0700 feat: Optional query persistence for Browser, internal Browser API fetching commit 5e5c905 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 16 13:02:12 2024 -0700 feat: Init schedules commit 56c7676 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 16 01:14:54 2024 -0700 feat: Log in and log out with redirectURI via base64-encoded state commit 5ed2234 Merge: 81e9329 3b5ee90 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 16 00:22:55 2024 -0700 Merge branch 'gql' into frontend-rewrite commit 3b5ee90 Author: maxmwang <orangemax888@gmail.com> Date: Sun Jun 16 00:05:15 2024 -0700 fix user auth commit 7a40c8b Author: maxmwang <orangemax888@gmail.com> Date: Sat Jun 15 23:58:55 2024 -0700 revert commit ef5a8ad Author: maxmwang <orangemax888@gmail.com> Date: Sat Jun 15 23:40:55 2024 -0700 fix: add placeholder value for refresh_token commit 81e9329 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Jun 11 18:13:54 2024 -0700 feat: Filter by day commit b35bdcb Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 9 19:53:24 2024 -0700 feat: Tooltip commit bc02bf4 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 9 14:49:10 2024 -0700 feat: Capacity color commit 756b343 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 9 14:45:05 2024 -0700 feat: Schedule context (units, classes, term) commit 6af982a Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 9 11:50:25 2024 -0700 feat: Manage commit eee78be Author: Matthew Rowland <contact@mathhulk.com> Date: Fri Jun 7 17:51:27 2024 -0700 feat: Compare commit 41ed7e6 Author: Matthew Rowland <contact@mathhulk.com> Date: Fri Jun 7 12:26:14 2024 -0700 feat: Schedule, Calendar commit f701d01 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Jun 2 13:58:40 2024 -0700 feat: Filtering commit 5f34a05 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue May 7 10:03:58 2024 -0700 feat: Enrollment, Grades commit 8be7703 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun May 5 22:12:08 2024 -0700 feat: In-progress migration to expanded data formats commit 1a54ba5 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Apr 28 21:59:11 2024 -0700 feat: Reserved seats commit 4297d42 Author: Matthew Rowland <contact@mathhulk.com> Date: Sat Apr 27 22:09:18 2024 -0700 feat: Sorting commit ed8020d Author: Matthew Rowland <contact@mathhulk.com> Date: Thu Apr 25 00:09:37 2024 -0700 feat: Filters commit d1968a9 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 23 21:24:43 2024 -0700 fix: uri commit 7bc1fff Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 23 21:19:12 2024 -0700 chore: gitignore commit e55ea83 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 23 21:16:36 2024 -0700 fix: dist commit 297755c Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 23 21:14:01 2024 -0700 fix: start commit dcd6a49 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 23 21:04:24 2024 -0700 fix: types commit 647c706 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 23 20:58:27 2024 -0700 feat: anyPrintInScheduleOfClasses commit b221fc0 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 23 20:21:42 2024 -0700 feat: usePrefersColorScheme commit 110c962 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Apr 21 21:41:27 2024 -0700 feat: Track markers commit 4b1da35 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Apr 21 21:35:25 2024 -0700 feat: Preview sections, Itinerary commit 12530e2 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Apr 21 14:39:41 2024 -0700 feat: section commit 7a81c36 Author: Matthew Rowland <contact@mathhulk.com> Date: Sat Apr 20 12:01:38 2024 -0700 feat: Search commit 2e660cf Author: ARtheboss <advayratan@yahoo.com> Date: Tue Apr 16 21:50:26 2024 -0700 fixes for course/catalog: added sections list, list type corrections, course grades commit ce16be2 Author: Matthew Rowland <contact@mathhulk.com> Date: Fri Apr 19 16:23:50 2024 -0700 feat: Search commit 3a41c7e Merge: 6f891fe 1145edd Author: Matthew Rowland <contact@mathhulk.com> Date: Wed Apr 17 15:32:38 2024 -0700 Merge branch 'gql' into frontend-rewrite commit 6f891fe Author: Matthew Rowland <contact@mathhulk.com> Date: Wed Apr 17 15:31:48 2024 -0700 feat: Init plan commit 1145edd Merge: 78b8cb2 a7a95a4 Author: Max Wang <maxmwang@berkeley.edu> Date: Tue Apr 16 21:22:36 2024 -0700 Merge pull request #705 from asuc-octo/gql-apollo-caching New Backend Caching commit a7a95a4 Author: Kevin Wang <kev@kevwang.dev> Date: Tue Apr 16 21:01:41 2024 -0700 remove extranneous cacheControl directive commit bcd9ede Merge: a9c289a 78b8cb2 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 16 10:01:55 2024 -0700 Merge branch 'gql' into frontend-rewrite commit a9c289a Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Apr 16 10:01:09 2024 -0700 feat: Support old terms, move filtering to Browser, Dockerfile, etc. commit 8727662 Author: Matthew Rowland <contact@mathhulk.com> Date: Mon Apr 15 00:49:47 2024 -0700 feat: Units, responsive Details, etc. commit cc05d01 Author: Kevin Wang <kev@kevwang.dev> Date: Sun Apr 14 23:57:19 2024 -0700 add caching to apollo queries commit 5f51ac2 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Apr 14 21:13:07 2024 -0700 feat: Plan, responsive Browser, responsive Catalog, responsive Calendar commit 9f5cb2d Author: Matthew Rowland <contact@mathhulk.com> Date: Sat Apr 13 15:23:38 2024 -0700 feat: Dark theme, search for and add classes to a schedule, etc. commit a933adf Author: Matthew Rowland <contact@mathhulk.com> Date: Fri Apr 12 17:02:01 2024 -0700 feat: Location, Time, Details, CCN, Tooltip; sections, etc. commit 78b8cb2 Merge: 492759b 363a02b Author: Max Wang <maxmwang@berkeley.edu> Date: Thu Apr 11 14:24:58 2024 -0700 Merge pull request #691 from asuc-octo/mathhulk-patch-1 Update controller.ts commit 363a02b Author: maxmwang <orangemax888@gmail.com> Date: Thu Apr 11 14:23:48 2024 -0700 TODO(production) flag commit 492759b Author: maxmwang <orangemax888@gmail.com> Date: Thu Apr 11 14:22:38 2024 -0700 TODO(production) flag commit b1ceba7 Author: maxmwang <orangemax888@gmail.com> Date: Thu Apr 11 12:40:11 2024 -0700 allow introspection on stanfurdtime commit 2b11afa Author: maxmwang <orangemax888@gmail.com> Date: Tue Apr 9 21:10:19 2024 -0700 fix github actions commit 643ff8b Author: maxmwang <orangemax888@gmail.com> Date: Tue Apr 9 21:07:50 2024 -0700 update github actions from old repo configs commit f334238 Merge: 5ed4ccc 4269b7e Author: Max Wang <maxmwang@berkeley.edu> Date: Tue Apr 9 20:35:51 2024 -0700 Merge pull request #704 from asuc-octo/gql-updated/updatedat-fix added resiliency against variance between updated and updatedAt commit 4269b7e Author: ARtheboss <advayratan@yahoo.com> Date: Thu Apr 4 19:51:19 2024 -0700 added resiliency against variance between updated and updatedAt commit e0e667a Author: Matthew Rowland <contact@mathhulk.com> Date: Thu Mar 21 12:49:51 2024 -0700 feat: Init map markers, look into directions commit fcb7760 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Mar 19 16:06:51 2024 -0700 feat: campus.geojson commit 2150698 Author: Matthew Rowland <contact@mathhulk.com> Date: Tue Mar 19 00:04:56 2024 -0700 feat: Buildings commit c27889e Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Mar 17 23:15:32 2024 -0700 feat: Itinerary commit ee9084c Merge: 1ba33c2 5ed4ccc Author: maxmwang <orangemax888@gmail.com> Date: Sun Mar 17 14:24:13 2024 -0700 Merge branch 'gql' into mathhulk-patch-1 commit 5ed4ccc Merge: bc514c0 1f3829c Author: Max Wang <maxmwang@berkeley.edu> Date: Sun Mar 17 14:23:26 2024 -0700 Merge pull request #692 from asuc-octo/revert-649-mongodb-backup Revert "added local minio s3 and backup:mongodb script" commit 1f3829c Author: Max Wang <orangemax888@gmail.com> Date: Sun Mar 17 14:23:13 2024 -0700 Revert "added local minio s3 and backup:mongodb script (#649)" This reverts commit bc514c0. commit 1ba33c2 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Mar 17 01:07:24 2024 -0700 Update controller.ts commit 496e023 Author: Matthew Rowland <contact@mathhulk.com> Date: Thu Mar 14 23:46:54 2024 -0700 feat: Init dynamic calendar commit 45276e1 Author: Matthew Rowland <contact@mathhulk.com> Date: Thu Mar 14 19:58:54 2024 -0700 feat: Init events commit 11ca8c1 Author: Matthew Rowland <contact@mathhulk.com> Date: Thu Mar 14 17:22:55 2024 -0700 feat: Init Schedule commit bc514c0 Author: ARtheboss <30683624+ARtheboss@users.noreply.github.com> Date: Wed Mar 6 14:37:19 2024 -0800 added local minio s3 and backup:mongodb script (#649) * added local minio s3 and backup:mongodb script * included metadata.json files to backup * added download code * added lifecycle and changed config names * removed minio container, added lifecycle rule check --------- Co-authored-by: ARtheboss <advayratan@yahoo.com> commit c099703 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Mar 3 18:15:52 2024 -0800 feat: Tooltips, Saturday + Sunday commit 59f8e00 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Mar 3 15:54:44 2024 -0800 feat: Empty placeholder, mock Calendar commit 02a26a3 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Mar 3 13:23:34 2024 -0800 feat: Filtering commit 1040a16 Author: Matthew Rowland <contact@mathhulk.com> Date: Sat Mar 2 18:42:45 2024 -0800 feat: Class commit f657db8 Merge: eaf4312 76eb98c Author: maxmwang <orangemax888@gmail.com> Date: Thu Feb 29 01:19:34 2024 -0800 Merge branch 'gql-proxy-auth' into gql commit eaf4312 Author: maxmwang <orangemax888@gmail.com> Date: Thu Feb 29 01:12:29 2024 -0800 clean up root commit 76eb98c Author: maxmwang <orangemax888@gmail.com> Date: Thu Feb 29 01:08:00 2024 -0800 express trust proxy (nginx) commit 5577f2b Author: maxmwang <orangemax888@gmail.com> Date: Thu Feb 29 00:54:28 2024 -0800 fix catalog query with course model change commit 3889271 Merge: 16fb4f9 87dc2a9 Author: Max Wang <maxmwang@berkeley.edu> Date: Thu Feb 29 00:25:09 2024 -0800 Merge pull request #682 from asuc-octo/gql-cors Allow local frontend to authenticate commit e07b62c Author: Matthew Rowland <contact@mathhulk.com> Date: Thu Feb 29 00:20:23 2024 -0800 feat: Initialize AccountContext, Features images, etc. commit 87dc2a9 Author: Matthew Rowland <contact@mathhulk.com> Date: Wed Feb 28 19:39:16 2024 -0800 feat: localhost:3001 commit 16fb4f9 Author: maxmwang <orangemax888@gmail.com> Date: Mon Feb 26 19:25:12 2024 -0800 add collections name to mongo models commit 9c31b41 Author: Matthew Rowland <contact@mathhulk.com> Date: Sun Feb 4 19:41:46 2024 -0800 feat: Course commit 6464bc5 Author: Matthew Rowland <contact@mathhulk.com> Date: Sat Feb 3 17:26:07 2024 -0800 feat: Landing commit dabc4da Author: Matthew Rowland <contact@mathhulk.com> Date: Sat Feb 3 14:36:59 2024 -0800 feat: Initialize commit fc1af7a Author: maxmwang <orangemax888@gmail.com> Date: Mon Nov 13 19:16:48 2023 -0800 revert: use POST for logout commit c95328c Author: Chengming-Li <115936688+Chengming-Li@users.noreply.github.com> Date: Mon Nov 13 18:36:30 2023 -0800 Rewrite schedule schema to better track courses(#644) * fix setSelectedSections bug * change mongo schema and typedefs * more progress * even more progress * finish redesign * housekeeping * fix bugs commit 262bba2 Author: maxmwang <orangemax888@gmail.com> Date: Mon Nov 13 18:26:58 2023 -0800 fix: remove Makefile commit 2d6259e Merge: 4d22a5f 12738eb Author: Max Wang <maxmwang@berkeley.edu> Date: Thu Nov 9 04:11:14 2023 -0800 Merge pull request #651 from asuc-octo/gql-logout Logout route fix commit 12738eb Author: maxmwang <orangemax888@gmail.com> Date: Mon Nov 6 20:23:19 2023 -0800 fix: change logout route to GET
* added local minio s3 and backup:mongodb script * included metadata.json files to backup * added download code * added lifecycle and changed config names * removed minio container, added lifecycle rule check --------- Co-authored-by: ARtheboss <advayratan@yahoo.com>
No description provided.