LFX Workspace: Integrate MLX as a new WASI-NN backend about wasmedge HOT 11 OPEN

@guptaaryan16
Please open a working PR related to your work ASAP for the midterm evaluation.

from wasmedge.

Well received. Thanks @guptaaryan16
Happy coding :-)

from wasmedge.

Week 1:

• Reading and brushing up on C++ basics and standards from Modern C++ programming(Federico Busato)
• Reading the plugin docs on the WasmEdge website
• Initial Ideas from the ml-explore/mlx-c project from the Apple ecosystem
• Completing the initial workspace issue in the WasmEdge repo

Week 2

• Completed all lectures and some programs in C++ and cmake to understand the behaviour of things within the ecosystem, Also use Valgrind and leaks to understand some debugging
• Discussed with hydai and awni about usage of either safetensors or ggml load in mlx/io
• Wrote small programs and researched to understand C++ behaviour
• Will start with the plugin this week(probably soon) after understanding a bit about what designs of previous plugins are

from wasmedge.

Week 3-4

• Since on inspection and discussion with mentors(hydai and awni), I did not find any implementation for C++ based inference on MLX/C++ directly for LLAMA and other llms, so I started working on my own NN class and definitions and will try to complete a working example by sunday(end of 4th week)
• I have also worked on the plugin files and completed the initial loading of model weights part (using API from MLX and WasiNN plugin) -> load function
• Working on the CPP frontend for the MLX library to load weights and use them in the layers API
• Research and complete the working example of CPP based on the MLX C++ API

Week 5-6

• Completed Initial implementation of NN Classes and working on loading weights problem(still WIP) with the custom NN classes

from wasmedge.

Please ensure that you post the progress every week.

from wasmedge.

Hi @hydai sorry for not updating my progress over here:

Week 7-8

• Currently working on implementing the classes and functions related to LLAMA models with thorough testing of inputs and outpus
• Was able to load weights from a GGUF type and safetensor type files and tensors correctly(finally)
• Need to still work on register_layer function for adding layers to the model and thus create a workable implementation of TransformerBlock
• Aiming to complete a workable implementation by the end of the week
• Also trying to understand some parts of LLAMA.cpp for usage in our project if possible

Also I would like to communicate that for the end of this week and next week, I may work for less time due to my university exams but would cover the work after the exams are over (in the following weeks).

from wasmedge.

Week 9- 10

• Completed work on register_layer, load_from_GGUF and load_from_safetensors functions
• Completing two model files phi2.cpp and llama.cpp using the classes and implemented necessary parts like Embedding, PhiAttention etc. as necessary, still working on some bugs related to embeddings and tokenisation
• After this, I am gonna work on the mlx plugin files and complete them before the deadline, adding all these functionalities and creating a working demo
• Also will start work on documentation from today so that it gets completed within deadline

from wasmedge.

It's very close to the deadline, please push your commits to the PR.

from wasmedge.

This is a final notification. We still haven't received your PR or updates for the final evaluation.

from wasmedge.

Hi @hydai I am completing some work on the PR and hope to submit it as soon as possible, sorry for the delay.

from wasmedge.

Please push working commits, even if they're dirty, to update the current PR so we can at least do the final evaluation based on that.

from wasmedge.