Performance Indicators

PerformanceIndicators

This module includes performance indicators to assess evolutionary multi-objective optimization algorithms.

• gd Generational Distance.
• igd Inverted Generational Distance.
• gd_plus Generational Distance plus.
• igd_plus Inverted Generational Distance plus.
• covering Covering indicator (C-metric).
• hypervolume Hypervolume indicator.

Example

julia> import Metaheuristics: PerformanceIndicators, TestProblems

julia> A = [ collect(1:3) collect(1:3) ]
3×2 Array{Int64,2}:
 1  1
 2  2
 3  3

julia> B = A .- 1
3×2 Array{Int64,2}:
 0  0
 1  1
 2  2

julia> PerformanceIndicators.gd(A, B)
0.47140452079103173

julia> f, bounds, front = TestProblems.get_problem(:ZDT1);

julia> front
                          F space
         ┌────────────────────────────────────────┐ 
       1 │⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠈⠄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠈⢆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠢⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠈⠢⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠀⠀⠉⠢⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
   f_2   │⠀⠀⠀⠀⠀⠀⠀⠀⠈⠑⢤⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠲⢄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠒⢤⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠙⠢⢄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠑⠢⢄⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠉⠢⠤⣀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ 
         │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠑⠢⢤⣀⠀⠀⠀⠀⠀│ 
       0 │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠒⠢⢄⣀│ 
         └────────────────────────────────────────┘ 
         0                                        1
                            f_1

julia> PerformanceIndicators.igd_plus(front, front)
0.0

pareto_front(state::State)

Returns the non-dominated solutions in state.population.

pareto_front(population::Array)

Returns non-dominated solutions.

gd(front, true_pareto_front; p = 1)

Returns the Generational Distance.

Parameters

front and true_pareto_front can be:

• N×m matrix where N is the number of points and m is the number of objectives.
• State
• Array{xFgh_indiv} (usually State.population)

gd_plus(front, true_pareto_front; p = 1)

Returns the Generational Distance Plus.

Parameters

front and true_pareto_front can be:

• N×m matrix where N is the number of points and m is the number of objectives.
• State
• Array{xFgh_indiv} (usually State.population)

igd(front, true_pareto_front; p = 1)

Returns the Inverted Generational Distance.

Parameters

front and true_pareto_front can be:

• N×m matrix where N is the number of points and m is the number of objectives.
• State
• Array{xFgh_indiv} (usually State.population)

igd_plus(front, true_pareto_front; p = 1)

Returns the Inverted Generational Distance Plus.

Parameters

front and true_pareto_front can be:

• N×m matrix where N is the number of points and m is the number of objectives.
• State
• Array{xFgh_indiv} (usually State.population)

spacing(A)

Computes the Schott spacing indicator. spacing(A) == 0 means that vectors in A are uniformly distributed.

covering(A, B)

Computes the covering indicator (percentage of vectors in B that are dominated by vectors in A) from two sets with non-dominated solutions.

A and B with size (n, m) where n is number of samples and m is the vector dimension.

Note that covering(A, B) == 1 means that all solutions in B are dominated by those in A. Moreover, covering(A, B) != covering(B, A) in general.

If A::State and B::State, then computes covering(A.population, B.population) after ignoring dominated solutions in each set.

hypervolume(front, reference_point)

Computes the hypervolume indicator, i.e., volume between points in front and reference_point.

Note that each point in front must (weakly) dominates to reference_point. Also, front is a non-dominated set.

If front::State and reference_point::Vector, then computes hypervolume(front.population, reference_point) after ignoring solutions in front that do not dominate reference_point.

deltap(front, true_pareto_front; p = 1)
Δₚ(front, true_pareto_front; p = 1)

Returns the averaged Hausdorff distance indicator aka Δₚ (Delta p).

"Δₚ" can be typed as \Delta<tab>\_p<tab>.

Parameters

front and true_pareto_front can be:

• N×m matrix where N is the number of points and m is the number of objectives.
• Array{xFgh_indiv} (usually State.population)

epsilon_indicator(A, B)

Computes the ε-indicator for non-dominated sets A and B. It is assumed that all values in A and B are positive. If negative, the sets are translated to positive values.

Interpretation

• epsilon_indicator(A, PF) is unary if PF is the Pareto-optimal front.
• epsilon_indicator(A, B) == 1 none is better than the other.
• epsilon_indicator(A, B) < 1 means that A is better than B.
• epsilon_indicator(A, B) > 1 means that B is better than A.
• Values closer to 1 are preferable.

Examples

julia> A1 = [4 7;5 6;7 5; 8 4.0; 9 2];

julia> A2 = [4 7;5 6;7 5; 8 4.0];

julia> A3 = [6 8; 7 7;8 6; 9 5;10 4.0 ];

julia> PerformanceIndicators.epsilon_indicator(A1, A2)
1.0

julia> PerformanceIndicators.epsilon_indicator(A1, A3)
0.9

julia> f, bounds, pf = Metaheuristics.TestProblems.ZDT3();

julia> res = optimize(f, bounds, NSGA2());

julia> PerformanceIndicators.epsilon_indicator(res, pf)
1.00497701620997