See this run. A fi is missing.

There are many issues in the OPTIMADEStructureAttributes datamodel.

nelements is used both as a dimension and as a property. It can only be one of them, in this nelements.

Same with nsites.

elements_rations is not well-defined. The description should describe how this array constructed?

With the update of the parser strategy, session handling has also been updated with backwards-incompatible consequences. This new way of working against OTEAPI should be implemented.

The container image is available at optimade-python-tools.

Version 2 has been released for SINTEF/ci-cd and should be used.

Currently, only structures are supported, but OPTIMADE has other resources.

Furthermore, relationships and more are also not supported.

An OPTIMADE parse strategy would parse a raw OPTIMADE response using the pydantic data models from OPTIMADE Python tools.

We need to add usage example(s).

This has been done in OTEAPI Core, as well as other OTEAPI repositories and is not a bad move.

Please update the below figure. The filter strategy should come after the data resource strategy in the pipeline:

This is conceptually important. We have the partial pipeline documenting the data source (typically DataResource >> Parse >> Mapping). That should be treated as one unit and always come first in the pipeline. The filter is not a part of the documentation of the data resource, but is a part of our application that selects the subset of all the data in the data resource we are interested in. It should use the initialize() method to specify the query such that only the information we need is fetched from the data resource.

I did this originally, but unfortunately the session object is not returned correctly from the "filter strategy", so I had to switch them around. I believe this issue is being resolved in OTElib, but this is an issue.
Furthermore, retrieving the data from the session object is plain wrong and should not be done like this. But that's out-of-scope of this tutorial to fix.

Originally posted by @CasperWA in #125 (comment)

Semantic mapping to the CIF Ontology should be added.
Furthermore, a new example should be added with the mapping strategy being part of a partial pipeline.

These are not handled properly when creating the final request URL.

See SINTEF/dlite#629

See EMMC-ASBL/oteapi-core#133

This should use the oteapi container image from ghcr.io/EMMC-ASBL/oteapi as well as the otelib package to test the plugin.

Extend the strategies with support for utilizing DLite to handle the retrieved OPTIMADE resources.
A SOFT/DLite model should be added and a workflow path to create DLite instances should be enabled - especially in the resource strategy.

There are repetitive connection issues with docker images in CI.

I think the issue may be the use of docker compose. Especially since it's using a custom network. This may interfere in unknown ways with the GitHub Actions' own internal docker setup.
A workaround might be to either run the docker compose using the host network - or instead not running the services using docker compose, but rather using the services field or starting each one using regular docker, which should add in the correct network automagically.

Use two space indentation.

Due to the added version part of the URL path (from mike), the reference to the pipeline figure under /img/ is not shown.

A first draft (simple) OPTIMADE resource strategy for downloading OPTIMADE structures should be implemented.

The pre-commit hooks for updating the Python API reference and documentation landing page have been implemented in CasperWA/ci-cd and should be reimplemented here.

Until #163 has been fixed, we need to pin to a specific docker image version of oteapi in the relevant CI test.

The docker image version to pin to: 1.20231108.329

See this job.

Fix the inputs for the external workflows called from SINTEF/ci-cd.

Update the repository to support pydantic v2.

Only then can OTEAPI Core >= v0.6 be used.

The main dependency hindering a pydantic v2 upgrade is OPTIMADE Python tools (optimade).

Libraries should not call setLogger and setHandler from the logging module - that should be done by the main application.

See also PR #171

Add more tests for:

• Parse strategy
• Utility functionality
• Model validators
• Custom type
• Resource strategy (more)
• Filter strategy

See this run for reference.

This is the same issue as defined in EMMC-ASBL/oteapi-dlite#126.

See this run.

Repeating the same example twice it quite boring. Can you split it up into two sub-pages, one for each otelib backend.

Yes and no. As this was a quick'n'dirty example, I didn't have capacity to come up with a more exciting example. Furthermore, it shows that there is no (or should be no) difference between the two methods, other than added information.

Originally posted by @CasperWA in #125 (comment)

See the specification.

This is to determine the version part of the OPTIMADE versioned base URL.

The species_at_sites property of OPTIMADEStructureAttribute is currently a list of strings, referring to the name of the individual species.

Having species_at_sites being a ref-type to OPTIMADEStructureSpecies would be more semantic and less entangled. This would also be a more efficient way to store the data.

This should work as intended for the DLite example, as the data should be stored in the DLite Collection (not retrieved from the session).

This is related to #143

An OPTIMADE filter strategy would construct an OPTIMADE query to be used with the resource strategy.

The new workflow should be auto-merged if all checks pass.

The example should be run at a part of the CI testing.

This repository already has an end-2-end test that does exactly this. No need to also run the tutorial.
That said, the mkdocs-jupyter plugin allows for executing the notebooks upon building the documentation, hence this could act as a CI test for whether the notebook works or not.
However, since a separate server is required and the notebook references localhost:80 it might be quite tricky to make it work properly. Hence, I'd consider it out-of-scope for this PR and writing this example, but definitely worth further investigation down the road.

Originally posted by @CasperWA in #125 (comment)

Notebooks:

• otelib.ipynb
• dlite.ipynb

Update: PR #127 which introduced the dlite.ipynb also added execution test for it when building the documentation.

The SOFT data models can effectively be minimized to a single file, effectively combining attributes and the top data model, while doing away with the species and assemblies data model in favor of simple dict types.

The dedicated parser would need to change accordingly.

See for example this job.
This can be fixed by simply updating to the latest release of Black.

It seems dependabot does not work with pyproject.toml :(