marker-inc-korea / autorag Goto Github PK

Now, we save which module and param is best to the best filename directly.
But, it will occur some unintended errors when param or module name has underscore (_) or hyphen (-).
So, I have to find a way how to notify which module and module params are selected in node run function.

Generator Node designed to return log probs and tokens as default, but as we can't get these informations at llama index, we had to make log probs llm module for this feature.

When merge release branch to main, automatically deploy package to pypi.

make standard logging method for AutoRAG project

The fast we change, it is better.

It is so confusing and difficult to make deploy function at retrieval_node for hybrid retrieval.
So, I have to make deploy function for hybrid retrieval.
Plus, I have to make hybrid retrieval yaml file for deployment.

Use deploy method, load it to fastAPI server automatically.

add autorag to docs requirements

From PR #40

I think these n-gram based metrics can open some options.

• N : int minimum unit for n-gram
• tokenizer : select tokenizer. (according to language, method etc include in tokenizer, tokenizer can influence value of each metric)

• etc

• #222
• #223

• Add some default modules, evaluation process and parser from config yaml file.
• Make evaluation result of each node_lines and nodes.

I had run python -m pytest in terminal

and I got this Error messege

======================================================================================================= short test summary info =======================================================================================================
FAILED tests/autorag/test_evaluator.py::test_start_trial - OSError: [Errno 22] Invalid argument: 'C:\\Users\\hanpa\\PycharmProjects\\AutoRAG\\0\\retrieve_node_line\\retrieval\\bm25=>top_k_50.parquet'
FAILED tests/autorag/nodes/retrieval/test_run_retrieval_node.py::test_run_retrieval_node - OSError: [Errno 22] Invalid argument: 'C:\\Users\\hanpa\\PycharmProjects\\AutoRAG\\tests\\resources\\test_project\\test_trial\\test_node_line\\retrieval\\bm25=>top_k_4.parquet'
============================================================================================= 2 failed, 26 passed, 23 warnings in 24.79s ==============================================================================================
(venv) PS C:\Users\hanpa\PycharmProjects\AutoRAG>

can't get multiple module params at get_param_combination method.

We support summary.csv feature for easy inspection, but not implemented yet.

There will be possible many things to edit at retrieval run.py

First, I will make use function for using found optimal pipeline for single query.

Implement Query Expansion base.py and run.py

This is for branch protection, too.

For easy-to-use, add CLI execution.

Type hyphothetical README with warning sign.

It override previous result, so delete all columns except result column.
So, it will can't preserve older results, it can be cause malfunction at certain RAG workflows.

Add some metrics for evaluating context relevancy.
It will calculate how retrieved passage contents are similar with retrieval gt contents.

It will have crucial part for evaluating passage compressor node.

Plus, run function for that.
It looks easy to make haha.

• Add first passage compressor module.
• Add passage compressor node decorator.
• Add passage compressor run functions.

from PR #41

sometimes, generation evaluation need to provide evaluation result in progress with token scale(like log_prob etc,)
for more advanced generation.

It will not functioning well, but we can fix this.