The initial challenge can be found at: PECOS_data_download_steps_ver_1.0.pdf
.
This is a custom cloud-native solution deployed on AWS with the following specifications:
- Infrastructure as Code with AWS CDK
- Source Code with AWS Lambda Functions built with Python Runtime
- Deployment Strategies: CDK commands (please read the
important_commands.sh
script).
The information of this repository is based on many online resources, so feel free to use it as a guide for your future projects!.
AWS Cloud Development Kit is an amazing open-source software development framework to programmatically define cloud-based applications with familiar languages.
My personal opinion is that you should learn about CDK when you feel comfortable with cloud-based solutions with IaC on top of AWS Cloudformation. At that moment, I suggest that if you need to enhance your architectures, it's a good moment to use these advanced approaches.
The best way to start is from the Official AWS Cloud Development Kit (AWS CDK) v2 Documentation.
The dependencies are explained in detail for each project, but the most important ones are Python and AWS-CDK.
My advice is to primary understand the basics on how CDK works, and then, develop amazing projects with this incredible AWS tool!.
-
Visual Studio Code
Visual Studio Code is my main code editor for high-level programming. This is not absolutely necessary, but from my experience, it gives us a great performance and we can link it with Git and GitHub easily. -
Docker
Docker is a platform designed to help developers build, share, and run modern applications. It uses OS-level virtualization techniques to deliver software packages known as "containers". It is used in this project as one of the building tools for creating the Lambda-function Docker images that will run the automation. -
Python
Python is an amazing dynamic programming language that let us work fast, with easy and powerful integration of different software solutions.
-
CDK CLI (Toolkit)
To work with the CDK, it is important to install the main toolkit as a NodeJs global dependency. Then, feel free to install the specific language AWS-CDK library (for example: aws-cdk.core. -
Selenium (Python-based)
The selenium package is used to automate web browser interaction from Python. It is one of the core components for Web-Scrapping tools. My advice is to only use Selenium if it is really needed (e.g. advanced web interactions).
All projects are well commented/documented and most of them have specifications and remarks for their purpose and I/O.
I will be uploading most of the files, and try to keep it as clean as possible.
- Thanks to all contributors of the great OpenSource projects that I am using.
Senior DevOps Engineer passionate about advanced cloud-based solutions and deployments in AWS. I am convinced that today's greatest challenges must be solved by people that love what they do. |