This is related to #465 task.
All the writers are fetch columns from the get_columns_list_with_types.
Which is a SampleColumnsProvider at the moment and a default provider.
The latter does not include frames.

For the following scenario:

python3 -m arekit_ss.sample --writer csv --source ruattitudes --sampler nn --src_lang ru --dest_lang ru --docs_limit 10 --text_parser nn --output_dir ./_out/a/ --no-vec

We have an output without annotated frames

sample-train-0.csv

• subject -> object as a classic of attitude extraction task
• target as a targeted sentiment analysis

Main goal of this release: #34
At this stage, we have a text parser presets that could be splitted into smaller parts and utilized in future.

• 🔧 remove update_arekit.sh script
• 📓 provide logo.png for the project
• 📓 rename reference section with Powered by AREkit.
• prompting techniques could be mentioned not in reference, but for a prompt-related comment
• #37
• 🔧 #43
• 📓 : Explain how to read the project name ("arekit double s")
• 📓 Mention image of the results in the README.md (Google colab 🪄 results)
• Move examples from the notebook into the test folder
• Clarify in filename that nn is only for sentiment

  arekit-ss/arekit_ss/framework/arekit/rows_nn.py

  Line 14 in 7785455

  def create_nn_rows_provider(labels_scaler):

• quick fix ident

  arekit-ss/arekit_ss/text_parser/text_nn_frames.py

  Line 35 in 7785455

  LemmasBasedFrameVariantsParser(

Post updates

• ❌ #57
• 📂 #59
• 🔧 #60
• 🔧 #61

https://github.com/nicolay-r/arekit-ss/blob/d972b26cb7c279fdc56bb8f76fa46f9a6e25b0ef/arekit_ss/sample.py#L44C51-L44C69

Here we may pass None to vectorizers argument:
https://github.com/nicolay-r/arekit-ss/blob/7785455be33c6be77cdf2ec967c8eb35c6bfa10f/arekit_ss/framework/arekit/rows_nn.py#L30C25-L30C25

• add --src_lang key
• limit key value to ru for some particular sources.

Once it was done
nicolay-r/AREkit#435

Treat as framework for polishing datasets.

Application is as follows:

1. Sampling with prompting.
2. Application of LLM
3. Gathering results and manual analysing.

Use NEREL for experiments

• #56
• #63

We use the following instruction:

python3 -m arekit_ss.sample --writer csv --source nerel --sampler prompt \
--prompt "For text: '{text}', is the relation of type {label_val} from '{s_val}' towards '{t_val}'? Answer yes or no, and explain why if no." \
--src_lang ru --dest_lang en --text_parser lm --output_dir ./_out/nerel-prompting-fact-checking/ --splits train:test

 python3 -m arekit_ss.sample --writer csv --source nerel-bio --sampler prompt \
 --prompt "For the text part of the PubMed abstract: '{text}', is the relation of type {label_val} from '{s_val}' towards '{t_val}'? Answer yes or no, and explain why if no." \
 --dest_lang en --text_parser lm --output_dir ./_out/nerel-bio-prompting-fact-checking/ --splits train:test

User experice and feedback

• #70
• #71

#38 related fix

• Collect every spruce in a separated folder (future movements and ParlAI experience)
• source list make a single Json with the rest setups within it

• support reading of any text, with sentence splitter
• support NER

#56 related

• Implement the latter as a mode prompt

Reason: Boost sampling up to x2 time. 🔥
nicolay-r/AREkit#462
Limitation: AREkit now supports sampling only for CSV

The impelementation of the text-processing pipeline for nn which supports frame annotation is as folllows:

Now, we mindfuly consider to avoid this support, because GoogleTranslate parser item was designed only for entiies as an another objects mentioned in text:

To add the related support, it is better to first generalize object representation in framework, with Value parameter as the common

Reason: sample file now includes the related information. it would be great to move everything in a separated file/folder.

• add src_list.py file

• update on colab
• update on README section

Perform the following updates:

• __init__.py
• setup.py
• README updates in project
• rewrite installation section
• move sampler into the root folder

https://github.com/nicolay-r/arekit-prompt-sampler/blob/f9e2188d9e72ce1428780c4a1496b03d86bc525f/sources/s_sentinerel.py#L96-L113

Main feature is support of the custom documents

Backend Updates and Extended Schemata

• Switch to the AREkit==0.24.0 (be14ffc)

Quality of the sampled data

According to the #52 experiments:

• #70
• #71

Dynamic prompting support

When analysing NEREL-bio outputs, it was found that some labels and relations are better to describe more precisely with prompts

• #67
• #66

Simplify new sources registration

There were few feedbacks on bad API with labels and that whole project might be crashed because of the incompletely registered source

• #62
• #58
• #59

Other / Minor updates

• #68
• #69
• #42
• doc_ids argument support which allows to select the specific doc_ids for processing (90488c2)
• [output_dir] customization removed (234ae21)
• #72
• dest_lang is optional parameter (by default is src_lang) (234ae21)
• #74
• 🔧 #75
• #76

This is the related issue in AREkit project:
nicolay-r/AREkit#496

• label returns uint label, while it is expected to be a formatted textual value. (fixed in AREkit)
• s_ind and t_ind should not be used without a value; we need to add support for s_val and t_val (fixed in AREkit)
  

#40 related

#34 related

continues #23

• update google colab script

• Use pure pandas API in cell for resuls reading.

• fix bugs at colab (#39, nicolay-r/AREkit#500)
• enhance prompting #40
• propose AD in which display logo + script for launching prompt sampling

#61 related.

Reason: some parts are designed for the particular cases of application in ml. For example:

• training data is expected to have a labels
• test data without labels.

is to provide the specific part of the dataset that supposed to be taken for sampling; by default we may consider no-folding, which means keep all the documents.

https://github.com/nicolay-r/arekit-googletrans-sampler/blob/65cb6f9a90ab8e7e982924111fd891e227969516/sources/s_rusentrel.py#L16

https://github.com/nicolay-r/arekit-googletrans-sampler/blob/65cb6f9a90ab8e7e982924111fd891e227969516/sources/s_ruattitudes.py#L12

In 0.23.1 it is not applicable, because we have a set of fixed sources, where grouping is based on the predefined entity parsers.

We can follow the way it works in ARElight
https://github.com/nicolay-r/ARElight/blob/c3d388cc7bcc5cea5be1ed1f0c19419c6157d309/examples/serialize_bert.py#L68-L69

Then it is followed by grouping method for entities:
https://github.com/nicolay-r/ARElight/blob/c3d388cc7bcc5cea5be1ed1f0c19419c6157d309/examples/serialize_bert.py#L76-L82