apsiminitiative / apsimx Goto Github PK

So that Arbitrator tries to arbitrate before sowing for crops that don't have phenology

From Val (2014-08-19):

• if, for example, rain is not in the body (i.e. the columns of data) in the weather file but is in the header (which should provide a constant value every day) then APSIM X thinks that rain cannot be found and crashes
• need a better way of handling the case when the dates in Clock in a simulation are not within the dates in the weather file. At present this is an unhandled exception.

1. Can we allow tabs in met files? People keep making that mistake.
2. Can we just allow xlsx met files? Save an extra step

From Bangyou (2015-03-20):

I am testing WheatPMFtest.apsimx under Tests\WheatPMFPrototype. Some variables cannot report in Sturcture.cs, e.g. MainStemNodeAppearanceRate. It seems to relate with deep copy variables in Structure.cs

This error is caused by both MainStemPrimordiaNo and MaximumNodeNumber equal to zero for WheatPMFPrototype (May be not real problem for this error). However, I believe there is a logic error in MainStemFinalNodeNumberFunction.

MainStemFinalNodeNumberFunction sets the MaximumMainStemNodeNumber to 0 if FinalLeafNumber is null and Structure.MainStemPrimordiaNo equals to 0.

MainStemFinalNodeNumberFunction is called before sowing when MainStemPrimordiaNo equals to 0 and don't call again. So the MaximumMainStemNodeNumber always equals to 0 if MainStemFinalNodeNumber is used in model. Then, MainStemPrimordiaNo is the minimum of daily increase of MainStemPrimordiaNo and MaximumNodeNumber. As the MaximumNodeNumber always equals to 0, there is no increase of MainStemPrimordiaNo (=0).

May change the default values of MaximumMainStemNodeNumber to a big value to fix it (L25 in MainStemFinalNodeNumberFunction.cs), but need to daily update the MaximumMainStemNodeNumber.

The error message is:

n exception of type 'System.Exception' occurred in Model.exe but was not handled in user code

Additional information: ERROR in file: C:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Tests\WheatPMFPrototype\WheatPMFtest.apsimx

Simulation name: PMFWheatTestBase

Unable to find assembly 'bamk2cod, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'.

at System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()

at System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo assemblyInfo, String name)

at System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA, Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo, SizedArray assemIdToAssemblyTable)

at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryObjectWithMapTyped record)

at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryHeaderEnum binaryHeaderEnum)

at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()

at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)

at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)

at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream)

at Utility.Reflection.BinaryDeserialise(Stream stream) in c:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Utility\Reflection.cs:line 389

at Utility.Reflection.Clone(Object sourceObj) in c:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Utility\Reflection.cs:line 462

at Models.Report.ReportColumn.StoreValue() in c:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Models\Report\ReportColumn.cs:line 381

at Models.Report.ReportColumn.OnReportFrequency(Object sender, EventArgs e) in c:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Models\Report\ReportColumn.cs:line 366

at System.EventHandler.Invoke(Object sender, EventArgs e)

at Models.Clock.OnDoCommence(Object sender, EventArgs e) in c:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Models\Clock.cs:line 212

at Models.Core.Simulation.DoRun(Object sender) in c:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Models\Core\Simulation.cs:line 170

at Models.Core.Simulation.Run(Object sender, DoWorkEventArgs e) in c:\Users\zhe00a\Documents\Working\04-Software\032-ApsimX\Models\Core\Simulation.cs:line 95

If there is a handler for this exception, the program may be safely continued.

As unknown reason when merging in the APSIMX repos, some of my changes don't merge into master branch. I will manually go through all my changed files, then redo all changes.

An agroforestry model is required. Implementation will be by Neil Huth and Justin Fainges. The implementation will involve the creation of a multi-Zone aware tree model and associated GUI.

From Hamish (2014-05-04)

With the existing test object you chose a simulation that you want to test, choose the table where data will come from to test, choose the variable to test on and provide some parameters.

But with stats auto generated for each simulation I want to test against all simulations and had to do a lot of box setting to get this working (See attached image). This is only a single test and it will worse for validations that have hundrids of simulations in them.

Solution. I think the order of selection should be switched so the first thing we select is the table from which data is to come and then select the simulation to test with the option in here of testing all simulations so where there are lots they don't need to be specified seperately. This way we can still test on any simulation from any table but can also chose broader tests which is what we need for movel validation statistics.

In addition, when using the TimeSeriesStats table there can be multiple instances of a simulation name, one for each variable that the PredictedObserved object is able to match on. Take a look at the attached image

From Justin (2014-07-17):
Tested with OilPalmValidation.apsim with no existing .db file. Simulations are all called base; they reside in a factorial node named after the experiment number. Each Experiment node is in a seperate folder.

Run the Base simulation for Expt 504 which generates a report. Clicking on the MonthlyOutput report shows the results of the run. If you now click on the MonthlyOutput report for Expt324, you will get the results for 504 show (This sim has not yet been run). Run Expt 324 and the MonthlyOutput will show the expected results. However, if you now click the MonthlyOutput node for 504, you will get the results for 324. Running 504 again gives results for 504 regardless of which experiment you click.

This could be happening because all simulations are named 'Base'. Might need to either expand the uniqueness search to the whole XPath, or disallow identical simulation names even though they're in different places.

If you have French windows you cannot drag heavy clay soil from
training toolbox into your example. Nothing happens when you drop it.
Columbian windows doesn't work with apsimx

Enable values from external files to be read and used as state variables in PMF-Slurp

The PMF needs some code to deal with multi-crop and multi-zone simulations please :-)

Apsim.FindAll works fine with models available when the simulation was loaded. I have code that gets a list of all Zones in a simulation. It finds all the Zones that were loaded from the file, but if I create a new Zone and rerun the code, the new Zone is not picked up.

Saving and reloading does pick up the new Zone correctly.

It seems renaming the component also picks it up.

From Hamish (2014-05-04):

Now we are ready to use the testing to keep models on track we need some documentation describing what each does and how to set the parameters.

A couple of additional test types might also be useful that allow a pass if the models stats get better but fails if they get worse (by a specified tollerence)

1. A test that allows variables to decrease but fails if they increase (for stats like MSD)
2. A test that allows variables to increase but fails if they decrease (for stats like R^2)
3. A test that allows variables to move closer to zero but fails if they move further away from zero (for stats like residule)

4 A test that does not allow a variable to change at all (I think this can ge done with current tests and will be necessary for tne "n" variable in the stats table as if this changes it shows simulations have dropped out)

Development of a simulation with fields, with multiple soil zones, sown, fertilised and irrigated from god level manager scripts.

This will allow the utility classes to be shared across multiple projects.

Add new modules for Aquaculture such as Pond, Supplement (Food), Prawn, Pond Nutrients, Dissolved Oxygen etc.

nb. I think we may be using GrazPlan supplement module for food for prawns, and other aquaculture animals.

We are working on a tool that generates APSIM soil descriptions from the New Zealand soils database and are hoping for two new information fields on the APSIM soils front view:

1. A text field that contains northings and eastings of the area that the soil represents.
2. A double field that contains the area of the polygon.

if there are two operation actions for the same module on the same date, only the last one triggers.

Some people kept having a problem with the sample UI. If they change
the depth to 0-180 and then run or something it doesn't save. You need
to click somewhere else and go back or something. The change doesn't
seem to be changed automatically

From OilPalm workshop:

People forget to put a space before "as" in the reporting variables UI - do we need a space between a bracket and the "as"?

From Neil H: Not sure this one can be done because it may change in the future or with configuration changes.

The treeview control in the left panel of the ExplorerView presents a popup menu when a node has been right-clicked. However, the contents of the menu do not reflect the node which has been right-clicked, but rather the last node to have been left-clicked. That is because the default behaviour of the WinForms Treeview control is not to have a right-click alter the currently selected node.
This behaviour is confusing, and it would seem better if the right-click would select the node which was clicked, and display a popup appropriate for that node.

Output variables are case sensitive but people don't get an error to
say they have don't something wrong - they just get a blank column at
run time

In APSIM we could put a layer thickness node onto a soil and the model would use this to set up the layering of the soil models and use the layer thickness specified in the Water node of the layer thickness node was absent.

APSIMX only sets soil layers from the layering specified on the water node and needs to have the layer thickness option implemented.

See attached example. Factorials cannot include a factor that is a c# script. It throws at various points depending on what you have done.

From Bangyou (2014-11-04):
The interface in the right side panel will disappear when clicking Save button. It will be better to refresh the interface in the right side panel after saving.

The comment style (/*! */) will cause a warning message:

 Missing XML comment for publicly visible type or member. 
 But the alternative style (i.e. ///) will not cause any warning messages.

The xml special character "<, >" will cause a warning message too

warning CS1570: XML comment on 'myclass' has badly formed XML -- 
'A name was started with an invalid character.". A 

Need to escape them using < and >

Currently we use the ModelEditControl which is unstable. Change the memo control to use the inbuilt RTF editor.

If a run fails due to an error, the GUI correctly displays an error message. However, if the user corrects the error and re-runs the simulation, the old error message is displayed at the end of the run, even though the simulation was successful.

Can we get the alignment of things in manager scripts sorting its self out nicely like it does in visual studio?

In future the plan is for pull requests to be not be accepted until all warnings are removed.

From NeilH (2015-02-16):

the following, if you select the Fert node and use ctrl-up arrow move the fert node above the popn node, the popn node and the fert node change places but the children don't. So the Fert node has 5 populations and the popn node has 2 n rates.

See attachment

From Justin (2014-07-17):
error is "OxyPlot paint exception: Value to add was out of range". Clicking away from the graph resets it to normal.

Doesn't seem to happen on predicted/observed plots.

There is a working example of how we want to do this on feature branch https://github.com/PlantandFoodResearch/ApsimX/tree/WaterExtractionTest located in C:\ApsimX\Tests\ReadExternalFiles\ReadFileTest.apsimx

This will only work on machines where APSIMX is installed in C:\ApsimX so won't pass on Jenkins. However I can't work out how to make this a relative file path.

I am trying to setup some cultivar specified parameter values in the WheatPMFPrototype.xml, e.g.

<Cultivar>
    <Name>hartog</Name>
    <Alias>hartog</Alias>
            <Command>[Phenology].Vernalisation.VernSens = 0.9</Command>
            <Command>[Phenology].Vernalisation.PhotopSens = 0.9</Command>
            <Command>[Phenology].FloralInitiationFloweringPhase.Target = 630</Command>
</Cultivar>

It seems [Phenology].Vernalisation.VernSens and [Phenology].Vernalisation.PhotopSens are overwrote during simulation, but [Phenology].FloralInitiationFloweringPhase.Target still uses the default value.

Do I use the correct way to specify the [Phenology].FloralInitiationFloweringPhase.Target?

Not sure how to debug and fix this problem.

From Bangyou (2015-03-12):

Function "CoverGreen" in the SimpleLeaf.cs using CoverFunction or ExtinctionCoefficientFunction to calculate green cover. Both "CoverFunction" and "ExtinctionCoefficientFunction" are optional. "CoverGreen" only checks whether "CoverFunction" exists.

Program will fail if both "CoverFunction" and "ExtinctionCoefficientFunction" don't exist in the model.

   public double CoverGreen
    {
        get
        {
            if (CoverFunction == null)
                return 1.0 - Math.Exp((-1 * ExtinctionCoefficientFunction.Value) * LAI);
            return Math.Min(Math.Max(CoverFunction.Value, 0), 1);
        }
    }


    [Link(IsOptional = true)]
    IFunction CoverFunction = null;

    [Link(IsOptional = true)]
    IFunction ExtinctionCoefficientFunction = null;

I try to run a simulation (*.apsimx) file in the command line. But it seems Model.exe only supports the absolute file path. This command will give an error message:

..\..\Bin\Model.exe "WheatPMFtest.apsimx"

The path is not of a legal form.
Finished running simulations. Duration .40 sec.

Model.exe will run the simulation if I specify the absolute path which started from drive letter, e.g.

..\..\Bin\Model.exe "C:\ApsimX\Tests\WheatPMFtest\WheatPMFtest.apsimx"

It will be better if Model.exe can support the relative path as we would like to run APSIM using Condor. In Condor environment, we only can specify the relative path.

From Justin (2014-09-10):
If you try using a built in function (say Utility.Math.Sum) and pass it a value that doesn't exist, you get the following exception:

ERROR in file: C:\ApsimX\Tests\SoilArbitrator\SoilArbitrator.apsimx
Simulation name: 1Tree1Field
Object reference not set to an instance of an object.
at Utility.Math.Sum(IEnumerable Values) in C:\ApsimX\Utility\Math.cs:line 221
at Models.Script.OnDoManagement(Object sender, EventArgs e)
at Models.Clock.OnDoCommence(Object sender, EventArgs e) in C:\ApsimX\Models\Clock.cs:line 145
at Models.Core.Simulation.DoRun(Object sender) in C:\ApsimX\Models\Core\Simulation.cs:line 154
at Models.Core.Simulation.Run(Object sender, DoWorkEventArgs e) in C:\ApsimX\Models\Core\Simulation.cs:line 82

In the VS debugger it simply points to external code since it's in the manager script. The only real pointer to the issue is Models.Script.OnDoManagement.

Not sure we can do much here since there's no way of knowing the value is null before it's run and after that you can't isolate the code since it's in a precompiled block.

Logging it here because I know this will cause grief for users in the future.

Occasionally, Model.exe will fail to delete old contents in SQLite table (L100 in Report.cs) as this.simulation equals to null.

 dataStore.DeleteOldContentInTable(this.simulation.Name, this.Name); 

I cannot reproduce this error every time, but it will be better to check this.simulation firstly. Will submit a pull request to fix it.

Steps to reproduce my problem:

1. Run my test phenology.apsimx in command line Model.exe phenology.apsimx.
2. Edit WheatPMFPrototype.xml file to change parameter value Target of `FloralInitiationFloweringPhase'.
3. Run phenology.apsimx again.
4. Check results of two runs, and I get the same answer.
5. I get different answer in I repeat the steps in the debug mode using VS.

It looks the WheatPMFPrototype.xml in the Model\Resources is compiled into Model.exe.

Is it true?

The current "Export to HTML" option for a model component is generating HTML with an entry for "r>" in the Model Documentation section.
Also, it might be nice if exporting the HTML could include additional information depending on the module type. For example, SummaryFile could include the actual summary results.

If you have a multiple zone simulations, with a report model in each zone, when graphing all the results are included in a single series, rather than providing a graph with a series for each zone.

I am find a way to create a minimum simulation to predict phenology, i.e. only using the necessary models. It will be useful to create a new simulation from other script language, e.g. R. The Optional argument in Link provides a very useful mechanism for this target.

GeminatingPhase links with Soils model, which is used to determine whether crop can germinate if ESW more than 0.

I suggest to set Soils link to Optional, which assumes crop can always germinate if Soils model doesn't exist.

Any suggestions about this point?