From e5f4b9b61c453e8ee1c86133e82d96afaff5aa95 Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Sat, 14 Sep 2024 17:43:04 +0000 Subject: [PATCH] build based on c0f59cf --- dev/.documenter-siteinfo.json | 2 +- dev/algorithm/index.html | 2 +- dev/index.html | 2 +- dev/installation/index.html | 2 +- dev/theory/index.html | 2 +- dev/usage/index.html | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 5930198..5f9a0d7 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-14T17:42:51","documenter_version":"1.7.0"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.5","generation_timestamp":"2024-09-14T17:43:01","documenter_version":"1.7.0"}} \ No newline at end of file diff --git a/dev/algorithm/index.html b/dev/algorithm/index.html index ec40542..1d0a51f 100644 --- a/dev/algorithm/index.html +++ b/dev/algorithm/index.html @@ -1,2 +1,2 @@ -Algorithm · DividedRectangles.jl
GitHub

Algorithms

Important Note: The content below is borrowed from Chapter 7, ***"Direct Methods,"*** of the forthcoming second edition of ***"Algorithms for Optimization"*** by Mykel Kochenderfer and Tim Wheeler.

The DIRECT algorithm incrementally refines a rectangular partition of the design space, using a heuristic inspired by potential Lipschitz constants to guide the refinement. This partitioning allows for global optimization by balancing exploration and exploitation of the design space.

Normalization:

To simplify the mathematics and avoid oversensitivity to dimensions with larger domains, DIRECT first normalizes the search space to be the unit hypercube.

If we are minimizing $f(\mathbf{x})$ in the interval between lower and upper ranges ($\mathbf{a}$) and ($\mathbf{b}$) DIRECT will instead minimize:

\[g(\mathbf{x}) = f(\mathbf{x} \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a})\]

After finding the minimum $f(\mathbf{x})$ of ($g$), the minimum of ($f$) is:

\[\mathbf{x}^* \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a}\]

Partitioning the Search Space:

DIRECT maintains a partition of this unit hypercube into hyperrectangular intervals. Each interval has:

  • A center ($c^{(i)}$)
  • An associated objective function value $f(\mathbf{c}^{(i)})$
  • A radius ($r^{(i)}$), which is the distance from the center to a vertex.

DIRECT begins every iteration by identifying intervals to be split with additional function evaluations. It splits the intervals for which a Lipschitz constant ($\ell$) exists such that that interval contains the Lipschitz lowerbound.

Splitting Intervals

DIRECT splits the identified intervals along their longest dimensions, dividing each interval into thirds. These splits refine the resolution in areas of interest, increasing accuracy near the minimum while avoiding excessive evaluations in less promising regions. The heuristic ensures that intervals are split in a way that balances the need for exploring the global space and refining local areas of interest.

Lipschitz Lower Bound

The Lipschitz lower bound for an interval is a circular cone extending downwards from its center ($\mathbf{c}^{(i)}$):

\[f(\mathbf{x}) \geq f(\mathbf{c}^{(i)}) - \ell \|\mathbf{x} - \mathbf{c}^{(i)}\|_2\]

This lower bound helps guide the decision of which intervals to split. The intervals for which a Lipschitz constant ($\ell$) exists such that the interval contains the Lipschitz lower bound are selected for further subdivision.

Visulizations:

To better understand the optimization process, here are some visualizations that represent different stages and aspects of the DIRECT algorithm's progress:

Image 13

The Lipschitz lower bound for different Lipschitz constants. Not only does the estimated minimum change locally as the Lipschitz constant is varied, the region in which the minimum lies can vary as well.

Image 14

The DIRECT lower bound for different Lipschitz constants `. The lower bound is not continuous. The minimum does not change locally but can change regionally as the Lipschitz constant changes

Image 12

The Lipschitz lower bound is an intersection of cones, which creates complicated surfaces in multidimensional space. The divided rectangle lower bound isolates each lower-bound cone toits own hyper-rectangular region, making it trivial to compute the minimum value in each region given a Lipschitz constant.

The left contour plot shows such a lower bound using five function evaluations. The right contour plot shows the approximation made by DIRECT, which divides the region into hyper-rectangles—one centered about each design point. Making this assumption allows for the rapid calculation of the minimum of the lower bound.

Image 21

The potentially optimal intervals for the DIRECT method form a piecewise boundary that encloses all intervals along the lower-right. Each dot corresponds to a hyper-rectangle.

Image 20

Potentially-optimal hyper-rectangle identification for a particular Lipschitz constant. Black dots represent DIRECT hyper-rectangles and their location in $(f(\mathbf{c}), r)$ space. A black line of slope is drawn through the dot belonging to the best interval. The dots for all other hyper-rectangles must lie on or above this line.

Image 18

Interval splitting in multiple dimensions (using DIRECT) requires choosing an ordering for the split dimensions.

Image 17

Consider the function $f(x) = sin(x) + sin(2x) + sin(4x) + sin(8x)$ on the interval [−2, 2] with a global minimizer near −0.272. Optimization is difficult because of multiple local minima. The figure above shows the progression of the univariate DIRECT method on this objective function, with intervals chosen for splitting rendered in blue.

Important Note: The content above is borrowed from Chapter 7, ***"Direct Methods,"*** of the forthcoming second edition of ***"Algorithms for Optimization"*** by Mykel Kochenderfer and Tim Wheeler.

+Algorithm · DividedRectangles.jl
GitHub

Algorithms

Important Note: The content below is borrowed from Chapter 7, ***"Direct Methods,"*** of the forthcoming second edition of ***"Algorithms for Optimization"*** by Mykel Kochenderfer and Tim Wheeler.

The DIRECT algorithm incrementally refines a rectangular partition of the design space, using a heuristic inspired by potential Lipschitz constants to guide the refinement. This partitioning allows for global optimization by balancing exploration and exploitation of the design space.

Normalization:

To simplify the mathematics and avoid oversensitivity to dimensions with larger domains, DIRECT first normalizes the search space to be the unit hypercube.

If we are minimizing $f(\mathbf{x})$ in the interval between lower and upper ranges ($\mathbf{a}$) and ($\mathbf{b}$) DIRECT will instead minimize:

\[g(\mathbf{x}) = f(\mathbf{x} \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a})\]

After finding the minimum $f(\mathbf{x})$ of ($g$), the minimum of ($f$) is:

\[\mathbf{x}^* \odot (\mathbf{b} - \mathbf{a}) + \mathbf{a}\]

Partitioning the Search Space:

DIRECT maintains a partition of this unit hypercube into hyperrectangular intervals. Each interval has:

  • A center ($c^{(i)}$)
  • An associated objective function value $f(\mathbf{c}^{(i)})$
  • A radius ($r^{(i)}$), which is the distance from the center to a vertex.

DIRECT begins every iteration by identifying intervals to be split with additional function evaluations. It splits the intervals for which a Lipschitz constant ($\ell$) exists such that that interval contains the Lipschitz lowerbound.

Splitting Intervals

DIRECT splits the identified intervals along their longest dimensions, dividing each interval into thirds. These splits refine the resolution in areas of interest, increasing accuracy near the minimum while avoiding excessive evaluations in less promising regions. The heuristic ensures that intervals are split in a way that balances the need for exploring the global space and refining local areas of interest.

Lipschitz Lower Bound

The Lipschitz lower bound for an interval is a circular cone extending downwards from its center ($\mathbf{c}^{(i)}$):

\[f(\mathbf{x}) \geq f(\mathbf{c}^{(i)}) - \ell \|\mathbf{x} - \mathbf{c}^{(i)}\|_2\]

This lower bound helps guide the decision of which intervals to split. The intervals for which a Lipschitz constant ($\ell$) exists such that the interval contains the Lipschitz lower bound are selected for further subdivision.

Visulizations:

To better understand the optimization process, here are some visualizations that represent different stages and aspects of the DIRECT algorithm's progress:

Image 13

The Lipschitz lower bound for different Lipschitz constants. Not only does the estimated minimum change locally as the Lipschitz constant is varied, the region in which the minimum lies can vary as well.

Image 14

The DIRECT lower bound for different Lipschitz constants `. The lower bound is not continuous. The minimum does not change locally but can change regionally as the Lipschitz constant changes

Image 12

The Lipschitz lower bound is an intersection of cones, which creates complicated surfaces in multidimensional space. The divided rectangle lower bound isolates each lower-bound cone toits own hyper-rectangular region, making it trivial to compute the minimum value in each region given a Lipschitz constant.

The left contour plot shows such a lower bound using five function evaluations. The right contour plot shows the approximation made by DIRECT, which divides the region into hyper-rectangles—one centered about each design point. Making this assumption allows for the rapid calculation of the minimum of the lower bound.

Image 21

The potentially optimal intervals for the DIRECT method form a piecewise boundary that encloses all intervals along the lower-right. Each dot corresponds to a hyper-rectangle.

Image 20

Potentially-optimal hyper-rectangle identification for a particular Lipschitz constant. Black dots represent DIRECT hyper-rectangles and their location in $(f(\mathbf{c}), r)$ space. A black line of slope is drawn through the dot belonging to the best interval. The dots for all other hyper-rectangles must lie on or above this line.

Image 18

Interval splitting in multiple dimensions (using DIRECT) requires choosing an ordering for the split dimensions.

Image 17

Consider the function $f(x) = sin(x) + sin(2x) + sin(4x) + sin(8x)$ on the interval [−2, 2] with a global minimizer near −0.272. Optimization is difficult because of multiple local minima. The figure above shows the progression of the univariate DIRECT method on this objective function, with intervals chosen for splitting rendered in blue.

Important Note: The content above is borrowed from Chapter 7, ***"Direct Methods,"*** of the forthcoming second edition of ***"Algorithms for Optimization"*** by Mykel Kochenderfer and Tim Wheeler.

diff --git a/dev/index.html b/dev/index.html index 5bfe5f2..05f4f3f 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Home · DividedRectangles.jl
GitHub

DividedRectangles.jl

CI Documentation Status codecov –-

DividedRectangles.jl provides an implementation of the DIRECT (DIvided RECTangles) algorithm for global optimization. The DIRECT algorithm is particularly useful for optimizing functions where the Lipschitz constant is unknown. This package allows users to perform both univariate and multivariate optimization efficiently.

Key Equation:

The algorithm is guided by the following fundamental equation:

\[f(x) = \sum_{i=1}^{n} c_i x_i\]

where:

  • ($x_i$) represents the variables.
  • ($c_i$) represents the coefficients corresponding to each variable.

This equation forms the basis for dividing the search space into smaller rectangles, optimizing the function by evaluating it at specific points.

This documentation provides detailed usage examples, theoretical background, and advanced customization options to help you get the most out of DividedRectangles.jl.

+Home · DividedRectangles.jl
GitHub

DividedRectangles.jl

CI Documentation Status codecov –-

DividedRectangles.jl provides an implementation of the DIRECT (DIvided RECTangles) algorithm for global optimization. The DIRECT algorithm is particularly useful for optimizing functions where the Lipschitz constant is unknown. This package allows users to perform both univariate and multivariate optimization efficiently.

Key Equation:

The algorithm is guided by the following fundamental equation:

\[f(x) = \sum_{i=1}^{n} c_i x_i\]

where:

  • ($x_i$) represents the variables.
  • ($c_i$) represents the coefficients corresponding to each variable.

This equation forms the basis for dividing the search space into smaller rectangles, optimizing the function by evaluating it at specific points.

This documentation provides detailed usage examples, theoretical background, and advanced customization options to help you get the most out of DividedRectangles.jl.

diff --git a/dev/installation/index.html b/dev/installation/index.html index 9103e1b..cce466c 100644 --- a/dev/installation/index.html +++ b/dev/installation/index.html @@ -1,4 +1,4 @@ Installation · DividedRectangles.jl
GitHub

Installation

To install the package, start Julia and run the following command:

using Pkg
 Pkg.add(url="https://github.com/sisl/DividedRectangles.jl")
-
+ diff --git a/dev/theory/index.html b/dev/theory/index.html index 8e82c73..a351111 100644 --- a/dev/theory/index.html +++ b/dev/theory/index.html @@ -1,2 +1,2 @@ -Theory · DividedRectangles.jl
GitHub

Theory

The DIRECT (DIvided RECTangles) algorithm is a global optimization method that does not require a known Lipschitz constant. This characteristic makes it particularly robust and versatile, applicable to a wide range of optimization problems. The algorithm operates by dividing the search space into smaller hyper-rectangles and evaluating the function at the center of each rectangle.

Key Concepts of the DIRECT Algorithm

  1. Division of Search Space:

    • The algorithm begins by treating the entire feasible region as a single hyper-rectangle.
    • This hyper-rectangle is then divided into smaller rectangles by splitting the dimensions.

  2. Function Evaluation:

    • The function is evaluated at the center of each hyper-rectangle.
    • This evaluation helps in identifying the most promising regions for further exploration.

  3. Selection of Potentially Optimal Rectangles:

    • After evaluation, the algorithm identifies potentially optimal rectangles. A rectangle is considered potentially optimal if it could contain the global minimum based on the evaluations performed so far.

  4. Recursive Division:

    • The selected rectangles are further divided, and the process repeats.
    • The algorithm continues to refine the search by focusing more on regions that are likely to contain the global minimum.

Mathematical Formulation

The algorithm relies on the following core mathematical principles:

Evaluation Function: The objective function $f(x)$ is evaluated at the center of each hyper-rectangle:

\[f(x) = \sum_{i=1}^{n} c_i x_i\]

where ($x_i$) are the variables, and ($c_i$) are the corresponding coefficients.

  • Rectangle Selection Criterion: A rectangle ($R$) is considered potentially optimal if:

\[ f(x_R) - L \cdot r_R \leq f(x) - L \cdot r_x \quad \text{for all } x \in R\]

where:

  • \[f(x_R)\]

    is the function value at the center of the rectangle.
  • ($r_R$) is the radius of the rectangle.
  • ($L$) is the Lipschitz constant.

Recursive Division: The hyper-rectangles are recursively divided along their longest dimension:

\[x_R = \frac{a_i + b_i}{2}\]

where ($a_i$) and ($b_i$) are the bounds of the rectangle along the ($i$)-th dimension.

Strengths of the DIRECT Algorithm

The strength of the DIRECT algorithm lies in its ability to systematically explore the entire search space while focusing on the most promising areas. This systematic coverage helps the algorithm escape local minima, making it particularly effective for objective functions with multiple local minima.

By not requiring the Lipschitz constant, the DIRECT algorithm is adaptable to various optimization problems, including those where the smoothness of the objective function is not well understood.

+Theory · DividedRectangles.jl
GitHub

Theory

The DIRECT (DIvided RECTangles) algorithm is a global optimization method that does not require a known Lipschitz constant. This characteristic makes it particularly robust and versatile, applicable to a wide range of optimization problems. The algorithm operates by dividing the search space into smaller hyper-rectangles and evaluating the function at the center of each rectangle.

Key Concepts of the DIRECT Algorithm

  1. Division of Search Space:

    • The algorithm begins by treating the entire feasible region as a single hyper-rectangle.
    • This hyper-rectangle is then divided into smaller rectangles by splitting the dimensions.

  2. Function Evaluation:

    • The function is evaluated at the center of each hyper-rectangle.
    • This evaluation helps in identifying the most promising regions for further exploration.

  3. Selection of Potentially Optimal Rectangles:

    • After evaluation, the algorithm identifies potentially optimal rectangles. A rectangle is considered potentially optimal if it could contain the global minimum based on the evaluations performed so far.

  4. Recursive Division:

    • The selected rectangles are further divided, and the process repeats.
    • The algorithm continues to refine the search by focusing more on regions that are likely to contain the global minimum.

Mathematical Formulation

The algorithm relies on the following core mathematical principles:

Evaluation Function: The objective function $f(x)$ is evaluated at the center of each hyper-rectangle:

\[f(x) = \sum_{i=1}^{n} c_i x_i\]

where ($x_i$) are the variables, and ($c_i$) are the corresponding coefficients.

  • Rectangle Selection Criterion: A rectangle ($R$) is considered potentially optimal if:

\[ f(x_R) - L \cdot r_R \leq f(x) - L \cdot r_x \quad \text{for all } x \in R\]

where:

  • \[f(x_R)\]

    is the function value at the center of the rectangle.
  • ($r_R$) is the radius of the rectangle.
  • ($L$) is the Lipschitz constant.

Recursive Division: The hyper-rectangles are recursively divided along their longest dimension:

\[x_R = \frac{a_i + b_i}{2}\]

where ($a_i$) and ($b_i$) are the bounds of the rectangle along the ($i$)-th dimension.

Strengths of the DIRECT Algorithm

The strength of the DIRECT algorithm lies in its ability to systematically explore the entire search space while focusing on the most promising areas. This systematic coverage helps the algorithm escape local minima, making it particularly effective for objective functions with multiple local minima.

By not requiring the Lipschitz constant, the DIRECT algorithm is adaptable to various optimization problems, including those where the smoothness of the objective function is not well understood.

diff --git a/dev/usage/index.html b/dev/usage/index.html index 4d8155c..14b41ae 100644 --- a/dev/usage/index.html +++ b/dev/usage/index.html @@ -31,4 +31,4 @@ # Optimize result = DividedRectangles.optimize(f, a, b) -println("Optimal value found at: ", result)

Parameters

Advanced Usage

Fine-Tuning Optimization:

The optimize function offers several parameters for fine-tuning the optimization process:

Example with custom parameters:

result = DividedRectangles.optimize(f, a, b, max_iterations=500, min_radius=1e-6)
+println("Optimal value found at: ", result)

Parameters

Advanced Usage

Fine-Tuning Optimization:

The optimize function offers several parameters for fine-tuning the optimization process:

Example with custom parameters:

result = DividedRectangles.optimize(f, a, b, max_iterations=500, min_radius=1e-6)