Initial refactor of cost functions #1071

jd-lara · 2024-03-08T01:10:30Z

@GabrielKS this is a first pass at a rewrite of all the costs associated with different unit types and the market costs.

src/models/cost_functions/ThermalGenerationCost.jl

src/parsers/power_system_table_data.jl

…ableCost`

It was never fully supported, we can add it back at some point if we intend to fully support it.

GabrielKS · 2024-03-27T18:51:16Z

Companion branches:

src/data_format_conversions.jl

src/models/cost_functions/ValueCurves.jl

test/test_cost_functions.jl

rodrigomha · 2024-04-19T18:14:54Z

Do we have methods to unpack the points of PieceWisePointCurves? Or any other InputOutput curve?

We should have a method that wraps:

power_cost_curve = PiecewisePointCurve([(1.0, 1.0), (2.0, 3.0), (3.0, 6.0), (4.0, 10.0), (5.0, 15.0)])
input_points, output_points = (zip(get_points(power_cost_curve)...) .|> collect)

Also, are we expecting users to use the get_points method? I like the presentation of the get_points when you print the curve, so I think it would be good to use this one to print in the REPL

julia> get_points(power_cost_curve)
5-element Vector{@NamedTuple{x::Float64, y::Float64}}:
 (x = 1.0, y = 1.0)
 (x = 2.0, y = 3.0)
 (x = 3.0, y = 6.0)
 (x = 4.0, y = 10.0)
 (x = 5.0, y = 15.0)

instead of:

julia> power_cost_curve
PiecewisePointCurve(PiecewiseLinearData(@NamedTuple{x::Float64, y::Float64}[(x = 1.0, y = 1.0), (x = 2.0, y = 3.0), (x = 3.0, y = 6.0), (x = 4.0, y = 10.0), (x = 5.0, y = 15.0)]))

GabrielKS · 2024-04-19T18:20:56Z

@rodrigomha I'm still ironing out the public interface (have taken a few stabs at this but I keep getting the domain-specific terminology wrong) but yes there should ultimately be a way to get all the x-coordinates and a way to get all the y-coordinates. In fact these functions exist at the underlying FunctionData level, I just need to figure out how best to wrap them. Definitely planning a more friendly println/etc. presentation as well.

rodrigomha · 2024-04-19T18:28:43Z

@rodrigomha I'm still ironing out the public interface (have taken a few stabs at this but I keep getting the domain-specific terminology wrong) but yes there should ultimately be a way to get all the x-coordinates and a way to get all the y-coordinates. In fact these functions exist at the underlying FunctionData level, I just need to figure out how best to wrap them. Definitely planning a more friendly println/etc. presentation as well.

This is good, we can just make a wrapper of get_x_coords, in some form (like get_input_points) such that we can call it directly to the InputOutputCurve, rather than the function_data itself.

Regarding the presentation, we should discuss if using pretty_tables or other form makes sense here or not.

src/models/cost_function_timeseries.jl

test/test_cost_functions.jl

src/models/cost_functions/variable_cost.jl

test/test_cost_functions.jl

jd-lara · 2024-04-22T15:40:46Z

Do we have methods to unpack the points of PieceWisePointCurves? Or any other InputOutput curve?

We should have a method that wraps:
power_cost_curve = PiecewisePointCurve([(1.0, 1.0), (2.0, 3.0), (3.0, 6.0), (4.0, 10.0), (5.0, 15.0)])
input_points, output_points = (zip(get_points(power_cost_curve)...) .|> collect)
Also, are we expecting users to use the get_points method? I like the presentation of the get_points when you print the curve, so I think it would be good to use this one to print in the REPL
julia> get_points(power_cost_curve)
5-element Vector{@NamedTuple{x::Float64, y::Float64}}:
 (x = 1.0, y = 1.0)
 (x = 2.0, y = 3.0)
 (x = 3.0, y = 6.0)
 (x = 4.0, y = 10.0)
 (x = 5.0, y = 15.0)
instead of:
julia> power_cost_curve
PiecewisePointCurve(PiecewiseLinearData(@NamedTuple{x::Float64, y::Float64}[(x = 1.0, y = 1.0), (x = 2.0, y = 3.0), (x = 3.0, y = 6.0), (x = 4.0, y = 10.0), (x = 5.0, y = 15.0)]))

There is an open issue for the printing. We should merge this to continue with the implementation in PSI.

src/models/cost_functions/variable_cost.jl

GabrielKS · 2024-04-22T22:11:51Z

Tracking issue is #1092, I'd love some substantive review of the added source files before I merge this.

daniel-thom · 2024-04-23T14:13:08Z

docs/src/modeler_guide/cost_functions.md

+
+PowerSystems.jl provides an extensive type hierarchy to explicitly express relationships between power production and cost. This lets the modeler represent cost functions as linear, quadratic, or piecewise input-output curves, potentially piecewise marginal heat rates, average heat rates, and more, as best fits the input data.
+
+To represent a cost for a particular [`Component`](@ref), the modeler first chooses one of the variable cost representations in the table below. Then, they wrap this [`ProductionVariableCost`](@ref) in either a [`CostCurve`](@ref) to indicate a cost in currency or in a [`FuelCurve`](@ref) to indicate a cost per unit of fuel plus a fuel cost. Finally, the user creates a domain-specific [`OperationalCost`](@ref) that contains this variable cost as well as other costs that may exist in that domain, such as a fixed cost that is always incurred when the unit is on. For instance, we may have `RenewableGenerationCost(CostCurve(TODO), 0.0)` to represent the cost of a renewable unit that produces at TODO, or `ThermalGenerationCost(; variable = FuelCurve(TODO), fixed = TODO, start_up = TODO, shut_down = TODO)` to represent the cost of a thermal unit that produces at TODO. Below, we give the options for `ProductionVariableCost`s. Information on what domain-specific cost must be provided for a given component type can be found in that component type's documentation.


Can the TODOs get completed?

daniel-thom · 2024-04-23T14:15:06Z

src/data_format_conversions.jl

+        op_cost["shut_down"],
+    )
+
+# TODO implement remaining _convert_op_cost methods 


Does this need to happen?

daniel-thom · 2024-04-23T14:17:39Z

src/models/cost_function_timeseries.jl

@@ -1,3 +1,24 @@
+# MarketBidCost has two variable costs, here we mean the incremental one
+get_generation_variable_cost(cost::MarketBidCost) = get_incremental_offer_curves(cost)
+# get_generation_variable_cost(cost::OperationalCost) = get_variable_cost(cost)


Does this need to stay?

daniel-thom · 2024-04-23T14:26:58Z

src/models/cost_functions/MarketBidCost.jl

+    "Variable Cost TimeSeriesKey"
+    incremental_offer_curves::Union{
+        Nothing,
+        IS.TimeSeriesKey,


What does this become with the new time series features (variable number of key/value pairs)?

daniel-thom · 2024-04-23T14:31:27Z

src/models/cost_function_timeseries.jl

+get_generation_variable_cost(cost::MarketBidCost) = get_incremental_offer_curves(cost)
+# get_generation_variable_cost(cost::OperationalCost) = get_variable_cost(cost)
+
+function _validate_time_series_variable_cost(


I don't think we are consistent, but we have often followed a convention where checker functions that throw start with check or _check. validate or _validate functions usually return true/false. I'm sure that isn't documented, but we should do it. It helps the reader to know when an exception can get thrown.

daniel-thom · 2024-04-23T15:06:27Z

src/models/cost_functions/variable_cost.jl

+    power_units::UnitSystem = UnitSystem.NATURAL_UNITS
+end
+
+CostCurve(value_curve) = CostCurve(; value_curve)


Is it painful to implement all of these positional argument constructors? You could require that all cost structs be constructed with kwargs. It might help with consistency.

daniel-thom · 2024-04-23T15:11:17Z

src/models/cost_functions/HydroGenerationCost.jl

@@ -0,0 +1,34 @@
+"""


Would it help users to make cost_functions a submodule? Maybe it helps searchability.

daniel-thom · 2024-04-23T15:13:39Z

src/parsers/power_models_data.jl

-            cost = PolynomialFunctionData(Dict((i, c / sys_mbase^i) for (i, c) in coeffs))
+            coeffs = Dict((i, c / sys_mbase^i) for (i, c) in coeffs)
+            quadratic_degrees = [2, 1, 0]
+            (keys(coeffs) <= Set(quadratic_degrees)) || throw(


Is this idiomatic Julia? It also seems more natural to me to say if error-condition throw or error-condition && throw. Asserts use this form.

daniel-thom · 2024-04-23T15:20:32Z

src/parsers/power_system_table_data.jl

@@ -1267,7 +1280,8 @@ function make_renewable_generator(
    base_power = gen.base_mva
    var_cost, fixed, fuel_cost =
        calculate_variable_cost(data, gen, cost_colnames, base_power)
-    operation_cost = TwoPartCost(var_cost, fixed)
+    @assert fixed == 0 "RenewableGenerationCost cannot have a fixed cost, got $fixed with variable cost $varcost"


Suggested change

@assert fixed == 0 "RenewableGenerationCost cannot have a fixed cost, got $fixed with variable cost $varcost"

@assert fixed == 0 "RenewableGenerationCost cannot have a fixed cost, got $fixed with variable cost $var_cost"

daniel-thom · 2024-04-23T15:21:45Z

test/test_constructors.jl

+    for T in subtypes(PSY.OperationalCost)
+        isabstracttype(T) || (@test T(nothing) isa IS.InfrastructureSystemsType)
+    end
+    # TODO add concrete subtypes of ProductionVariableCost?


Does this need to get done?

src/models/cost_functions/MarketBidCost.jl

jd-lara added 7 commits March 7, 2024 16:59

delete old cost data

5818ed1

add ProductionVariableCosts

23c6e0d

add ThermalGenerationCost

f7c1aab

delete old exports

4f1a34a

delete outdated setter

1fc21fb

partially update the structs

a39cff7

add new cost functions

771b98b

github-actions bot reviewed Mar 8, 2024

View reviewed changes

src/models/cost_functions/ThermalGenerationCost.jl Outdated Show resolved Hide resolved

src/parsers/power_system_table_data.jl Outdated Show resolved Hide resolved

format run

779f537

jd-lara requested a review from claytonpbarrows March 8, 2024 16:07

jd-lara and others added 10 commits March 8, 2024 14:34

some updates

f8c28fe

Fix typos; minor refactors for legibility

6a267a6

Add LoadCost

6d19398

Add cost function includes

6ed554b

Eliminate TwoPartCost in generated structs

ddd5d08

Refine usage of LoadCost

2d434dd

Fix serialization, typos

3032e2d

Fix OperationalCost constructors, test

cdfd673

Make OperationalCost structs take the right kind of `ProductionVari…

0156734

…ableCost`

Remove PolynomialFunctionData

7af370b

It was never fully supported, we can add it back at some point if we intend to fully support it.

GabrielKS added 9 commits March 29, 2024 12:04

Accommodate FunctionData refactors

d7ff445

Add ValueCurve for input-output, incremental, average rate curves

44e5f1a

ValueCurves: add documentation, support serialization

d9f1e1d

Redefine CostCurve, FuelCurve in terms of ValueCurve

652d09c

Add some zero methods

d63dae6

Standardize OperationalCost naming

31b848a

Make test_cost_functions pass

f0fdd47

Export new cost structs

d055875

Refine cost struct constructors

2d82421

github-actions bot reviewed Apr 17, 2024

View reviewed changes

src/data_format_conversions.jl Outdated Show resolved Hide resolved

src/models/cost_functions/ValueCurves.jl Outdated Show resolved Hide resolved

test/test_cost_functions.jl Outdated Show resolved Hide resolved

Rename PiecewiseSlopeCurve, revert ill-advised constructor refactor

259b6cb

github-actions bot reviewed Apr 17, 2024

View reviewed changes

test/test_cost_functions.jl Outdated Show resolved Hide resolved

Save MarketBidCost refactor progress before HDF5 transformations work

0afc7bc

github-actions bot reviewed Apr 19, 2024

View reviewed changes

src/models/cost_function_timeseries.jl Outdated Show resolved Hide resolved

test/test_cost_functions.jl Outdated Show resolved Hide resolved

GabrielKS added 3 commits April 19, 2024 19:52

Implement HDF serialization properly for ValueCurves

bd1cfeb

Wrap up high-level cost structs in time series for MarketBidCost

d758391

Run formatter

b26d639

github-actions bot reviewed Apr 20, 2024

View reviewed changes

jd-lara assigned GabrielKS and unassigned jd-lara Apr 22, 2024

GabrielKS added 3 commits April 22, 2024 10:30

Update test for new cost structs

e8193dd

Rename MarketBidCost.ancillary_services -> `ancillary_service_offers

f524e21

Use cost function aliases

234936e

GabrielKS mentioned this pull request Apr 22, 2024

PSY Remaining TODOs for Second Cost Refactor #1092

Closed

10 tasks

GabrielKS reviewed Apr 22, 2024

View reviewed changes

src/models/cost_functions/variable_cost.jl Show resolved Hide resolved

This was referenced Apr 22, 2024

Cost Functions Refactor 2.1 NREL-Sienna/PowerSystemsTestData#52

Merged

Cost Functions Refactor 2.1 NREL-Sienna/PowerSystemCaseBuilder.jl#81

Merged

GabrielKS self-requested a review April 22, 2024 22:09

daniel-thom reviewed Apr 23, 2024

View reviewed changes

jd-lara commented Apr 25, 2024

View reviewed changes

src/models/cost_functions/MarketBidCost.jl Show resolved Hide resolved

GabrielKS added 2 commits April 27, 2024 01:44

Fix typo

b0f4f60

Merge branch 'psy4' into jdgk/cost_functions

39c87fb

jd-lara merged commit 48472d2 into psy4 Apr 30, 2024
1 of 8 checks passed

jd-lara deleted the jdgk/cost_functions branch May 21, 2024 19:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial refactor of cost functions #1071

Initial refactor of cost functions #1071

jd-lara commented Mar 8, 2024

GabrielKS commented Mar 27, 2024 •

edited

Loading

rodrigomha commented Apr 19, 2024

GabrielKS commented Apr 19, 2024

rodrigomha commented Apr 19, 2024

jd-lara commented Apr 22, 2024

GabrielKS commented Apr 22, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024

daniel-thom Apr 23, 2024


		PowerSystems.jl provides an extensive type hierarchy to explicitly express relationships between power production and cost. This lets the modeler represent cost functions as linear, quadratic, or piecewise input-output curves, potentially piecewise marginal heat rates, average heat rates, and more, as best fits the input data.

		To represent a cost for a particular [`Component`](@ref), the modeler first chooses one of the variable cost representations in the table below. Then, they wrap this [`ProductionVariableCost`](@ref) in either a [`CostCurve`](@ref) to indicate a cost in currency or in a [`FuelCurve`](@ref) to indicate a cost per unit of fuel plus a fuel cost. Finally, the user creates a domain-specific [`OperationalCost`](@ref) that contains this variable cost as well as other costs that may exist in that domain, such as a fixed cost that is always incurred when the unit is on. For instance, we may have `RenewableGenerationCost(CostCurve(TODO), 0.0)` to represent the cost of a renewable unit that produces at TODO, or `ThermalGenerationCost(; variable = FuelCurve(TODO), fixed = TODO, start_up = TODO, shut_down = TODO)` to represent the cost of a thermal unit that produces at TODO. Below, we give the options for `ProductionVariableCost`s. Information on what domain-specific cost must be provided for a given component type can be found in that component type's documentation.

	@assert fixed == 0 "RenewableGenerationCost cannot have a fixed cost, got $fixed with variable cost $varcost"
	@assert fixed == 0 "RenewableGenerationCost cannot have a fixed cost, got $fixed with variable cost $var_cost"

Initial refactor of cost functions #1071

Initial refactor of cost functions #1071

Conversation

jd-lara commented Mar 8, 2024

GabrielKS commented Mar 27, 2024 • edited Loading

rodrigomha commented Apr 19, 2024

GabrielKS commented Apr 19, 2024

rodrigomha commented Apr 19, 2024

jd-lara commented Apr 22, 2024

GabrielKS commented Apr 22, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

GabrielKS commented Mar 27, 2024 •

edited

Loading