e2niee / pandapipes Goto Github PK

When I run the tests in pandapipes locally, files are created in pandapipes.results. This should never happen! Please make sure to use temporary files where to save results when building tests. Can anyone tell me where they come from? I would like to remove them then...

"What tests do we have to check if the outcome of the properties is what we want?
If we calculate a property with a float, a float should be returned. If we insert a numpy array, we need to return a numpy array of same length and dtype. Is there a way to do it with np.ones_like?"
[transfered from #62]

If you create just a one node system without any pipes, the solver can't find a solution and won't converge. This shouldn't be the case!

Adaptions have to be made in line with e2nIEE/pandapower#1177

Hello all,

today I wrote the code for a heating network and was very confused because the signs of the heat in the model for the heat exchanger are defined in exactly the opposite way from what I intuitively assumed and there is no corresponding reference in the documentation. I would suggest reversing the definition. This would mean that positive heat flows go into the fluid in the network and negative flows out.

Best regards
Dennis

In create_pump_from_parameters, the name parameter name 'regression parameter' is misleading.
Actually, it is the polynomial degree of the fit - the name should be changed accordingly.
Also, there should be a warning if a non-integer is passed to the variable (internally, the np.polyfit function is called, which silenty applies int(x) on the polynomial degree variable).

networks_init_.py

from pandapipes.networks.simple_gas_networks import *
from pandapipes.networks.simple_water_networks import *
from pandapipes.networks.simple_heat_transfer_networks import * #is missing in 0.40

Hi,

I think that saying that a par has a default value is confusing, if it is actually required

create_heat_exchanger returns indeed
TypeError: create_heat_exchanger() missing 1 required positional argument: qext_w
if not provided

To generalize calculate derivatives and extract results on branch_models.py level might lead to problems in near future, as a most abstract class defines really real and concrete functions/classmethods. Therefore, there should be a discussion on how to make the code as abstract as possible, prevent duplicated code and seperate it from really concrete functions which should only be part of the real component models.

I would like to start a discussion on how the values of pressure, velocity and potentially temperature should be initialized in case of time series simulations. In pandapower, there are many different options available. Looking at the documentation, I find the following text:

init (str, “auto”) - initialization method of the loadflow pandapower supports four methods for initializing the loadflow:
    - “auto” - init defaults to “dc” if calculate_voltage_angles is True or “flat” otherwise
    - “flat”- flat start with voltage of 1.0pu and angle of 0° at all PQ-buses and 0° for PV buses as initial solution
    - “dc” - initial DC loadflow before the AC loadflow. The results of the DC loadflow are used as initial solution for the AC loadflow.
    - “results” - voltage vector of last loadflow from net.res_bus is used as initial solution. This can be useful to accelerate convergence in iterative loadflows like time series calculations.

init_vm_pu (string/float/array/Series, None) - Allows to define initialization specifically for voltage magnitudes. Only works with init == “auto”!
    - “auto”: all buses are initialized with the mean value of all voltage controlled elements in the grid
    - “flat” for flat start from 1.0
    - “results”: voltage magnitude vector is taken from result table
    - a float with which all voltage magnitudes are initialized
    - an iterable with a voltage magnitude value for each bus (length and order has to match with the buses in net.bus)
    - a pandas Series with a voltage magnitude value for each bus (indexes have to match the indexes in net.bus)

In general, I think that it makes sense to implement a similar strategy in pandapipes. However, we have a less rigid and in some places more implicit approach to building our internal structures. I see the following obstacles:

• Extracting results from results tables means that they have to be taken from all branch result tables for velocity and all node result tables for pressure and temperature.
• If we consider internal nodes and edges, the results are not even included in any tables.

Therefore I think that it makes sense to only extract results from the internal structure (node_pit, branch_pit). This raises the following questions:

• Do we want to make sure that the previous pipeflow has led to valid results, or should this be left to the user?
• If we want to allow handing over an initialization vector (a float would be no problem), how can we support the user to hand over the correct values in the correct positions? The pit is created on the basis of the component list, i.e. in a specific order, but the user should be able to understand the order in which e.g. branch elements are inserted into the branch_pit and whether there are internal branches or nodes. Do we want to support this process? Could we for instance provide a function that inserts initial values from user input in the correct positions of the pit or creates a correctly sorted array from it? And if this is possible by going through all the components, will this still save any time?

I would like to implement a first suggestion on how to extract results from the pit and reuse the values as initial values for a new pipeflow. Then at least we would have the two options of a flat init and extracting from results, which should be a good first step.

Hi,

using simple_plot on the net in circular_flow_in_a_district_heating_grid tutorial the heat ex is missing
(something like in the png figure would be good)

thanks

Currently, 5 of the cited books are in German. It would be helpful to provide equivalents in English, where possible.
So far, I found two translations:

BS10: H.D. Baehr and K. Stephan. Wärme- und Stoffübertragung. Springer, 7th edition, 2010.:
• Heat and Mass Transfer | Baehr and Stephan | Springer
JHP02: Ferziger J.H. and M. Peric. Numerische Strömungstechnik. Springer, third edition, 2002:
• Computational Methods for Fluid Dynamics | Ferziger and Peric | SpringerLink

Still only in German:
Cer08: Günter Cerbe, editor. Grundlagen der Gastechnik: Gasbeschaffung - Gasverteilung - Gasverwendung. Hanser, München and Wien, 7th edition, 2008.
DR08: W. Dahmen and A. Reusken. Numerik für Ingenieure und Naturwissenschaftler. Springer, second edition, 2008.
EH90: R. Eberhard and R. Hüning. Handbuch der Gasversorgungstechnik. Oldenbourg, second edition, 1990.

If you know any suitable references, please add them in the comments. They will be added to the documentation soon.

I would like to a start a discussion on how to establish a correlation between pipe and power networks?
In pandapipe pumps and sources are power consumption elements, but this feature didn't include. This issue is also present in pandapower, in which heat pump and pipe grid sources should consider as power loads. These should be taken into account on pipeflow and powerflow. Also, the managing standard type section is still in process and it is not possible in this way either.

This issue should be a short overview of possible measures to decrease the calculation time of the pipeflow in general or in the case of a timeseries calculation.

We should address the following aspects for timeseries calculations:

• Initialization of state variables based on user input or previous results (see issue #86 ).
• Recycle options to take over internal variables from previous calculations, in analogy to pandapower. So, it should be possible to just take over the pit, to just update the jacobi matrix (cf. PR #87 ) and to reuse lookups.

Just concerning the internals of pipeflow, I see the following possible improvements:

• Vectorize the derivative calculation of components that are based on the same code. This is rather urgent, as currently a pipeflow calculation for a grid with pipes and valves takes much longer than for a grid with only pipes, although the derivative calculation does exactly the same and could be summarized for both components.
• The result extraction of different components should be checked to make sure that it is as efficient as possible. It should as much as possible be based on numpy code, pandas calls like loc should be avoided where possible.
• Eliminate equations from the system matrix that can be simplified (e.g. at a node with just two branch elements the mass flow balance might be left out and calculated afterwards).

The list should be extended and I think it would make sense to create an action item list with milestones from it.

I think we should avoid using the type keyword for any variable, since it is a built-in name. Anyways, for the StdType class I would suggest to replace it with component, since that would be clearer.

When running a pipeflow, I think that most users are not very much interested about the displayed output. For me, it is a little annoying when running a number of simulations and having to search the whole output for some special information of interest that is completely going down in the flood of information from the pipeflow. If there is opposition to removing the output (about iterations, residual etc.), we might add an output mode to the options, that can be set to "verbose" if needed.

Further TODOs:

• create non-numba versions (vectorized) of derivatives calculation for hydraulics or a way of wrapping the jit decorator only in case that numba is installed
• create numba and non-numba versions of derivatives calculations for heat transfer
• change the structure, so that functions for the whole internal process (i.e. between finishing pit creation and extraction of results) can be found in one module or package
• check if all terms are correct inside derivative calculation -> somehow the df_dp and df_dp1 seems to be odd wrt the signs

Sometimes it would be handy to have an export option from pandapipes to Excel.
A brief test showed that pandapower's to_excel works to a large exentend also for pandpipes networks:

import os
from pandapipes import networks as nw
from pandapower.file_io import to_excel

net = nw.schutterwald()
path = os.path.join(os.getcwd(), "Schutterwald.xlsx")
to_excel(net, path)

However, some pandapipes-particularities can not be processed:

hp.pandapower.io_utils - WARNING: Could not serialize net.fluid
hp.pandapower.io_utils - WARNING: Could not serialize net.component_list
hp.pandapower.io_utils - WARNING: Could not serialize net.std_type

This is because they are not pandas dataframes.

As far as I can tell, ext_grid ist the only element that is named the same in pandapipes and pandapower. All other names seem to be unique (bus/junction, line/pipe etc.). Is that correct? And if so, wouldn't it make sense to rename ext_grid in pandapipes to something else, so that all names are unique?

Seems to me this would make it easier to write applications that combine pandapower and pandapipes.

That's the basic example on the website:
import pandapipes as pp
net = pp.create_empty_network(fluid="lgas")
j1 = pp.create_junction(net, pn_bar=1.05, tfluid_k=293.15, name="Junction 1")
j2 = pp.create_junction(net, pn_bar=1.05, tfluid_k=293.15, name="Junction 2")
j3 = pp.create_junction(net, pn_bar=1.05, tfluid_k=293.15, name="Junction 3")
ext_grid = pp.create_ext_grid(net, junction=j1, p_bar=1.1, t_k=293.15, name="Grid Connection")
sink = pp.create_sink(net, junction=j3, mdot_kg_per_s=0.045, name="Sink")
pipe = pp.create_pipe(net, from_junction=j1, to_junction=j2, length_km=0.1, diameter_m=0.05, name="Pipe 1")
valve = pp.create_valve(net, from_junction=j2, to_junction=j3, diameter_m=0.05, opened=True, name="Valve 1")

When i execute the code i get the following error message:
pipe = pp.create_pipe(net, from_junction=j1, to_junction=j2, length_km=0.1, diameter_m=0.05, name="Pipe 1") TypeError: create_pipe() missing 1 required positional argument: 'std_type'

the error persists, when i add "add_stdtypes=False (if i am not mistaken this option disables the "Standardtypes" for Pipelines, which is by default set to "True")
net = pp.create_empty_network(fluid="lgas",add_stdtypes=False)

I had to search for a standardpipeline in the library, delete the diameter and replace it with a std_type
pipe = pp.create_pipe(net, from_junction=j1, to_junction=j2, length_km=0.1, std_type="200_GGG", name="Pipe 1")

At the pandapipes installation via pip install there come some errors during the wheel building. The installation ends successfull, so a fix is not very important but it looks better.

The diameter unit is currently given in meters.

For normal pipe diameters it seems more user friendly to me to allow the input in cm instead of meters. "This pipe has a diameter of 86 cm" seems much more intuitive to me than "This pipe has a diameter of 0.86 m".

Is there a good reason to use meters instead of centimeters here?

I think it would be worthwhile to implement a similar behaviour as in pandapower with pre-defined _empty_res_xy tables which can be copied. This would increase performance significantly. In my test case the result extraction with df initialization made up around 30% of the run time of the pipeflow. In pandapower the result extraction was only 10% of the runpp call, even though the pandapower net contained more tables.

I think it is a good idea to update the tutorials section on our homepage. I created this issue mainly to remind myself to:

• Have a look at the first to heating tutorials. Do they make sense?
• Move any tutorials, which are currently only available in the documentation to the tutorials section on the homepage

@dlohmeier , @SimonRubenDrauz , @jkisse : In december, we created also some german tutorials for a workshop. Shall we we also create links for these german versions on the homepage?

It seems that the heat transfer solver in pandapipes has problems with one-directional flows inside a network when the slack node for temperature values is downstream "at the end" of the flow and there is more than one branch. The following example can illustrate and reproduce the case where convergence fails, regardless of iteration steps specified:

import pandapipes as ppi
import pandapipes.plotting as ppplot

net=ppi.create_empty_network(name='net',fluid='water')

ppi.create_junctions(net,4,pn_bar=3,tfluid_k=330,geodata=[(0,0),(1,0),(2,1),(2,-1)])
ppi.create_pipes_from_parameters(net,from_junctions=[0,1,1],to_junctions=[1,2,3],length_km=[0.1,0.3,0.1],diameter_m=[0.05]*3)#,k_mm=[0.049]*3,loss_coefficient=[0.15]*3,alpha_w_per_m2k=[0]*3,sections=[5]*3)
ppi.create_sources(net,junctions=[2,3],mdot_kg_per_s=[0.02]*2)
ppi.create_ext_grid(net,junction=0,p_bar=2.5,t_k=323,type='pt')

ppplot.simple_plot(net)

ppi.pipeflow(net,mode='all')

This topology is, however, useful in many cases where e.g. district heating networks are separated for calculation purposes in feed line and reflux line (this case above representing the reflux line) and it shall be solved for a specific reflux temperature back at the heating plant (@ junction 0). Consumers wouldb e represented by sources in this case.

Whenn additional external grids as slack nodes are addes to the junctions connected to the sources and the external grid at junction 0 (heating plant) is changed to type='t' the desired result can be attained.

net=ppi.create_empty_network(name='net',fluid='water')

ppi.create_junctions(net,4,pn_bar=3,tfluid_k=330,geodata=[(0,0),(1,0),(2,1),(2,-1)])
ppi.create_pipes_from_parameters(net,from_junctions=[0,1,1],to_junctions=[1,2,3],length_km=[0.1,0.3,0.1],diameter_m=[0.05]*3)#,k_mm=[0.049]*3,loss_coefficient=[0.15]*3,alpha_w_per_m2k=[0]*3,sections=[5]*3)
ppi.create_sources(net,junctions=[2,3],mdot_kg_per_s=[0.02]*2)
ppi.create_ext_grid(net,junction=2,p_bar=3,t_k=330,type='t')
ppi.create_ext_grid(net,junction=3,p_bar=3,t_k=330,type='t')
ppi.create_ext_grid(net,junction=0,p_bar=2.5,t_k=323,type='p')

ppplot.simple_plot(net)

ppi.pipeflow(net,iter=100,mode='all')

In pipeflow_setup in line 599ff are those lines really required

I would like to discuss the default-compressibility factors that are stored in properties/[fluid name]/compressibility.txt in the format slope, offset.
For H and L natural gas, the slope (=der_comp) is currently -0.0022, i.e. Z=0.78 @ 100bar. According to this figure on ResearchGate this means that we are on the 0° C isothermal curve. At 25 °C, Z would be 0.85, according to the chart.

We could also choose the 10 °C curve (-0.0019) as default since it seems closer to real operational conditions. In the long term, we could also implement different factors for a range of temperatures. What do you think?

This is in particular relevant for the calculation of high pressure networks, as the error is proportional to the pressure.

Hi,

it seems that in
https://github.com/e2nIEE/pandapipes/blob/develop/tutorials/circular_flow_in_a_district_heating_grid.ipynb
in the figure junction 1 and 3 have been swapped, w.r.t how the network is build

pp.create_pipe_from_parameters(net, from_junction=j0, to_junction=j1, ...
and
pp.create_pipe_from_parameters(net, from_junction=j2, to_junction=j3,...

not sure if it's prettier to change the figure or the code

In test_release_control.py, some tests are not run in the various master versions. This is partly because the json file cannot be read with controllers in the network, and also because identifiers of the time series calculation have changed. In the case of the "from_json" problem, the io_utils.py and file_io.py in pandapipes and the io_utils.py in pandapower must be adjusted. In the following, the tests of test_release_control.py that pass and fail are listed:

develop

test_release_control_pipeflow_json passed
test_release_control_pipeflow_without_json passed
test_release_control_ts_json failed -> from_json and run_timeseries_ppipe
test_release_control_ts_without_json failed -> run_timeseries_ppipe

master version 0.1.2 (latest)

test_release_control_pipeflow_json passed
test_release_control_pipeflow_without_json passed
test_release_control_ts_json failed -> from_json and run_timeseries_ppipe
test_release_control_ts_without_json failed -> run_timeseries_ppipe

master version 0.1.1

test_release_control_pipeflow_json passed
test_release_control_pipeflow_without_json passed
test_release_control_ts_json failed -> from_json
test_release_control_ts_without_json passed

master version 0.1.0

test_release_control_pipeflow_json passed
test_release_control_pipeflow_without_json passed
test_release_control_ts_json failed -> from_json
test_release_control_ts_without_json passed

I don't really understand some lines of code in the pipeflow procedure. If we look at the pipeflow function, we find the following:

create_lookups(net, NodeComponent, BranchComponent, BranchWInternalsComponent)
node_pit, branch_pit = initialize_pit(net, Junction.table_name(),
                                      NodeComponent, NodeElementComponent,
                                      BranchComponent, BranchWInternalsComponent)

and in the pipeflow_setup these lines of code:

in create_lookups:

for comp in net['component_list']:
    if issubclass(comp, BranchComponent):
        branch_from, branch_table_nr = comp.create_branch_lookups(
            net, branch_ft_lookups, branch_table_lookups, branch_idx_lookups, branch_table_nr,
            branch_from)
    if issubclass(comp, NodeComponent) | issubclass(comp, BranchWInternalsComponent):
        node_from, node_table_nr = comp.create_node_lookups(
            net, node_ft_lookups, node_table_lookups, node_idx_lookups, node_from,
            node_table_nr, internal_nodes_lookup)

and in initialize_pit:

for comp in net['component_list']:
    if issubclass(comp, NodeComponent) | \
            issubclass(comp, BranchWInternalsComponent) | \
            issubclass(comp, NodeElementComponent):
        comp.create_pit_node_entries(net, pit["node"], node_name)
    if issubclass(comp, BranchComponent):
        comp.create_pit_branch_entries(net, pit["branch"], node_name)

As far as I can recall, the component classes should handle this kind of behaviour, as we want to be able to include new components without adapting the whole pipeflow code. So, if a component is a branch component and does not contain internal nodes, the create_node_lookups and create_pit_node_entries functions should simply do nothing. But that should be up to the component.

This code with global imports of different components also raises another issue that if inside a component we import something else that again references into pipeflow (in my case to the PipeFlowNotConvergedException that must be moved otherwise), we get a circular import. Of course that should be avoided.

Hi,

it would be useful for quick understanding of the network, to implement labels in simple_plot

(with few parameters like:
show_labels=False, labels_font_size=3 , etc...)

thanks

Similar to pandapower it could be possible to create different valve types that allow to connect valves between a junction and a branch element (e.g. a pipe). It would require a few changes in the valve component:

• add a column like "et" in pandapower switch table
• add internal nodes for node-to-branch valves in create_node_lookups of valve component
• consider these internal nodes in all methods of the valve component

Some suggestions to improve the evaluation of pressure and velocity values:

• If the results table is plotted, the p_bar value shows the relative pressure. At the moment, the function to retrieve internal pipe results returns absolute pressure values. I can change the behaviour of this function, but it might help to note that all displayed pressure values are relative by naming them, for example, "p_rel_bar".

• The values in the pipe results for v_to and v_from are misleading, because if a pipe has several sections, the mean value of all v_to velocities of the pipe is computed. Of course, v_to should be the velocity of the last pipe section positioned before the next junction.

Not all folders in the "doc" directory are included in the pypi package. Thus, if a user wants to build the documentation locally, it will likely be truncated. We very much recommend the online documentation on readthedocs.io (for master or develop).

If users want to build the documentation locally, it is recommended to download/checkout the repository directly from GitHub instead of the pypi package.

Hi,

the eq. in https://pandapipes.readthedocs.io/en/latest/components/pipe/pipe_component.html#heat-transfer-mode
seems to be missing the surface factor ($\pi d l $ ) -- also, that would be rather a power transfer, right? ($\dot{Q}$)

here

it seems well implemented -- if that's what you use

I think there is still an error in the extract_results method of the ext_grid component. If there is an ext_grid table with junction entries [4, 8, 2], then _sum_by_group should return the values for [2, 4, 8], i.e. the order is changed. This doesn't seem to be reflected in the remaining code.

check, if pandapipesNet.new() can be called and the net.update(self.obj) leads to the same results, instead of creating an empty network in io-utils. Missing tables and columns from earlier versions should be added in the convert_format script, instead of just overwriting everthing in io-utils.

Currently the external and internal representation of the circulation pumps do not match. We should definately rethink this concept. If doing so we should also think about #88.

Hi,

i am having the attached error with run_timeseries in pandapipes (I am trying to adapt the tutorial on time-series)

any idea?

thanks

ps:
pandapower 2.6.0
pandapipes 0.4.0

pps: it would be nice to have some doc in https://pandapower.readthedocs.io/en/latest/timeseries/output_writer.html :) -- thanks

the logs are in e2nIEE/pandapower#1272

I just realized that when a pipeflow does not converge, two diverging error messages appear:

pandapipes.pipeflow - WARNING: Maximum number of iterations reached but hydraulics solver did not converge.
UserWarning: Pipeflow converged, however, the results are phyisically incorrect as pressure is at the nodes [ 237. 280. 301. ... 85568. 85569. 85593.] are negative

The first is just a logger warning, the second one is a UserWarning without raising an exception. It occurs in the junction component model upon result extraction. This issue raises the question how to deal with non-convergence. How should the results be written into the tables? Should the result tables be deleted, filled with NaN-values or just filled with whatever values calculated?

Specify dtypes in pd.Series to silence warnings pd.Series(dtype=float64)

e..g.:

openmodelica_comparison/test_water_openmodelica.py: 26 warnings
  /opt/hostedtoolcache/Python/3.8.6/x64/lib/python3.8/site-packages/pandapipes/test/openmodelica_comparison/pipeflow_openmodelica_comparison.py:106: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
    v_diff_mean_valve = pd.Series()

I am currently running some calculations of different network topologies for a large gas network and found that the Colebrook-White algorithm does not use a limit for its iterations. However, in some cases the algorithm obviously does not converge, as I had to kill my script after nearly half a million iterations. I think it would be wise to implement a maximum number of iterations and use a normal pipeflow exception, if this number is reached. But endless loops should really be avoided.

@dcronbach do you have a suggestion on a suitable default number of maximum iterations?

There should be a robust way to open old pandapipes nets that were stored with an earlier release, also when something changed, e.g. the fluid-property class or column names in the component tables.
As a first step, it might be a good idea to store the version number in the json file.
Also, some try-except clauses could help to restore as much as possible, even when particular components can not be loaded successfully.

pandapower faced a similar issue when 2.0 was released, we can have look how they solved it.

I am working with some constant properties for my current simulation example and am getting many warnings during the pipeflow, which state 'WARNING: One constant property has several input variables even though it is independent of these' . In the calculations, this is quite annoying, and there is nothing I could do about them except changing the logging level. Here's what happens internally:

• During the pipeflow, some properties like density or viscosity are requested in dependence of the temperature. This is intended behavior, as most fluids show a dependency, but in my case this behavior is simplified.
• Since the constant property does not depend on any variable, it raises this warning. However, it is impossible to change the pipeflow code to respect this aspect.

I would suggest to add a flag to the ConstantProperty class whether a warning should be displayed or not. And I would set it to False by default, as this is normal behavior.

Using the current implementation of circulation pumps, there are some configurations in combination with external grids and sinks which might not work.

We can

a) either change the concept of the circulation pump. Probably related issue: vgl. auch Issue 214 auf GitHub #214

b) identify these configurations and implement exceptions for them.

Both alternatives are time-consuming.

Documentation on Time Series has to be updated to new version.

Check if it makes sense to run calculations in pu instead of absolute values. Also check if this makes sense for hydraulic and thermal calculcations.

In the SimBench grids with valves, these valves are not correctly connected between junctions and pipes.

Regarding the coupling between hydraulic and temperature calculations, it makes sense to compare a fully coupled approach, where all equations are solved in one matrix, with an approach where the matrices for hydraulics and temperature calculations are solved iteratively. What do we need and why?

In the circular_flow_in_a_district_heating_grid tutorial
I am tring to create a heating station, with create_pump (instead of create_circ_pump_const_mass_flow or create_circ_pump_const_pressure) and I get

RuntimeWarning: invalid value encountered in double_scalars
  residual_norm = (linalg.norm(epsilon) / (len(epsilon)))
Maximum number of iterations reached but hydraulics solver did not converge.

which I also get with type='p' (for create_circ_pump_const_mass_flow or create_circ_pump_const_pressure)

I am also adding a create_heat_exchanger with negative qext_w

So, I am wondering if it is possible at all in PP to create a heating source, without fixing pressure or temperature at the heating source

Thanks