hewlettpackard / cmf Goto Github PK

CMF library helps to collect and store information associated with ML pipelines. It tracks the lineages for artifacts and executions of distributed AI pipelines. It provides API's to record and query the metadata associated with ML pipelines. The framework adopts a data first approach and all artifacts recorded in the framework are versioned and identified by the content hash.

Home Page: https://hewlettpackard.github.io/cmf/

License: Apache License 2.0

Python 76.44% Dockerfile 0.75% Jupyter Notebook 7.51% Shell 0.12% HTML 0.35% CSS 2.08% JavaScript 12.76%

lineage metadata pipeline

cmf's People

Contributors

Stargazers

Watchers

Forkers

annmary-roy abhinavchobey sergey-serebryakov revathyramanan varkha-d-sharma justcherie gyanaranjan1302 iffmainak bikeshinro rishabhsharma22 be-secure

cmf's Issues

cmf- visualization with comparision charts for each hyper parameter

Issues with installing on Apple Mac M3

I am trying to install cmf according to the instructions on this page: https://hewlettpackard.github.io/cmf/ using conda but I am facing issues with peer dependency ml-metadata.

I am getting the following error while trying to install using conda and github version:

I also tried to install using conda and pypi but got simillar message:

Tried the virtual environment route as well but faced similar issue.

cmf prechecks - dvc remote , git etc

cmf logger should capture the GPU & CPU metrics

Execution id should be a guid and unique across different sites

Verification - provide steps or tools to verify that cmf is installed properly

Error messages - Review all the error messages and its handling

Documentation using Mkdocs

Documentation - Document about what's happening under the hood (Git + DVC, and how CMF uses it).

Documentation - Improve documentation on metadata query

Simplify logging the artifacts

Feedback - Can one common meta writer for all artifacts like model, metric, input, and output? The artifact type should be a user argument.

Fix documentation

Documentation today lacks the context of the project , eg: The distributed approach and the git like simplicity is missing ,we can pull in that context from our internal GitHub

Support for newer versions of ML Metadata (MLMD)

Hi!
CMF currently depends on ml-metadata version 1.11.0. I remember we discussed incrementing this version, but do not remember why this has not been done. I would like to understand if it is possible to upgrade and what it will take.

The reason I am asking this is the following. While working on developer-friendly graph-like API for MLMD, I am experimenting with MLMD's capability to filter nodes by providing an instance of ListOptions class with filter_query field set. The proto file defines usage examples. I was experimenting with it, and found out that I can't filter contexts by parent/child IDs. After googling a bit, I found that this happens to be a bug on MLMD side and has already been reported in this thread. This is fixed in version 1.14.0. This happens to be the latest version released on Aug 9, 2023.

Supportability - Support for python > 3.9 .

we have a dependency on mlmd. Mlmd supports only python 3.7,3.8 and 3.9.
Current mlmd version we have is 1.11.0.
Latest mlmd version is 1.12.0..but no major changes between the two versions

Enhancement - It will be good if we can add some validations, for example we are logging inputs and outputs , but if user missed to specify input/output this may lead to broken lineages .

This would need enhancements to do auto logging. Can be taken care by integration with looking into code with tools like mlinspect or similar
Can we have an optional debug mode ?
Can we check simple things like if input is logged or anything is logged at all ?

CMF logger running in its separate thread/process. This will enable logging in parallel with execution of the main process

Something similar to python logging (or MLCommons logger). Just a crazy idea - can we reuse functionality of a python logging library for this? In other words: CMF frontend logging API -> python logging -> CMF backend API running in separate process and being taken care and managed by python logging library.

Enhancement - It would be good if were are able to see the changes in the content before publishing/pushing, so that if user has some control on whether to commit the changes or not