Skip to content

Commit

Permalink
create structure and first documentation pages
Browse files Browse the repository at this point in the history
  • Loading branch information
Galileo-Galilei committed Nov 8, 2024
1 parent b9cd2d0 commit 192ccb2
Show file tree
Hide file tree
Showing 22 changed files with 101 additions and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ kedro boot fastapi <kedro_run_args>
These production-ready features would be natively included in your FastAPI apps:

- Embedded [Gunicorn web server](https://gunicorn.org/) (only for Linux and macOS)
- [Pyctuator](https://github.com/SolarEdgeTech/pyctuator) that report some service health metrology and application states. Usually used by service orchestrators (kubernetes) or monitoring to track service health and ensure it's high availability
- [Pyctuator](https://github.com/SolarEdgeTech/pyctuator) that report some service health metrology and application states. Usually used by service orchestrators (kubernetes) or monitoring to track service health and ensure its high availability
- Multiple environments configurations, leveraging kedro's ``OmegaConfigLoader``. ``["fastapi*/"]`` config pattern could be used to configure the web server. Configs could also be passed as CLI args (refer to ``--help``)

You can learn more by testing the [spaceflights Kedro FastAPI example](examples/README.md#rest-api-with-kedro-fastapi-server) that showcases serving multiples endpoints operations that are mapped to differents pipeline namespaces.
Expand Down
29 changes: 29 additions & 0 deletions docs/source/01_introduction/01_motivation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Overview

``kedro-boot`` tries to solve 3 differents issues of kedro (as of ``kedro==0.19.X``) which occur when deploying kedro projects. This is described in details in this [design document](https://github.com/kedro-org/kedro/issues/3540).

## The standalone mode - Programmatically call the Kedro Session

### The key feature : running the session multiple times

We want to use the ``KedroSession`` object programmatically *outside a kedro project* to control its behaviour in another application.

The key feature is to make the ``KedroSession`` runnable multiple times, so that we do not have to rebuilt the entire objects (which is slow and complicated) for each execution. This needs a couple of optimization for speed and flexibility:

- runtime data injection: we should be able to pass data in memory to the session to override a dataset, and not force the dataset to read it from a persisted source
- runtime params injection: we should be able to modify parameters between each execution to modify the behaviour
- artifacts preloading: we should be able to preload and cache once for all session exeuxtions some input data with a high I/O cost (e.g. a machine learning model should not be hot reloaded on eeach predict, but only once )

Basically, we would like to be able to execute the following code:

```python
# PSEUDO CODE - Non runnable in plain kedro, this is a fictitious API
session=KedroSession.create(project_path)
session.preload(artifacts=["my_model", "my_encoder"]) # :sparkles: FEATURE 1: preload artifacts, cache them and not not release them between runs for speed
session.run(pipeline="my_pipeline", runtime_data={"data": data1}) # :sparkles: FEATURE 2: inject data at runtime from the session
session.run(pipeline="my_pipeline", runtime_params= {"customer_id": id}) # :sparkles: FEATURE 3: run the same session mulitple times + :sparkles: FEATURE 4: inject runtime params at... runtime (as the name says!) instead of instantation time
```

```tip
Running the session programatically is very convenient when you want to call kedro in a programm that owns the entrypoint (e.g. you launch the app with a command different from ``kedro run`` like ``streamlit run``, ``uvicorn run``...)
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
```{tip}
- If you want to change the behaviour of ``kedro run`` (e.g. to loop over the session multiple times), see the [embedded mode]()
- If you want to call a kedro pipeline programatically in a third party application launched with its own command (eg. ``streamlit run``, ``uvicorn run``, datarobot...), see the [standalone mode]()
- if you want to serve a kedro pipeline as an API, see the [KedroBootFastapiApp]()
```
File renamed without changes.
Empty file.
File renamed without changes.
6 changes: 6 additions & 0 deletions docs/source/04_standalone_mode/01_concepts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

3 changes: 3 additions & 0 deletions docs/source/04_standalone_mode/02_advanced_configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Preload artifacts

# Passing itertime params
6 changes: 6 additions & 0 deletions docs/source/05_embedded_mode/01_concepts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

3 changes: 3 additions & 0 deletions docs/source/05_embedded_mode/02_advanced_configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Preload artifacts

# Passing itertime params
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

6 changes: 6 additions & 0 deletions docs/source/07_example_gallery/standalone_mode/02_fastapi.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Kedro Boot objects

## KedroBootSession

## CompilationSpec

0 comments on commit 192ccb2

Please sign in to comment.