database to markdown file about marathi-shabd HOT 21 CLOSED

w.r.t to the existing scripts there are some optimisations that can be done.
lets try to write programs using the unix philosophy. basically these
2 points for now -

• Write programs that do one thing and do it well.
• Write programs to work together.

applying this to your sort and gen-md scripts, we can do the following -

1. sort -

• make the sorting function universal/generic instead of specific to the input
  csv format.
• like, determine the number of columns in the csv from the number of elements
  in the top row instead of hardcoded values.
• when sorting we can pass both the column index to be used for sorting and
  the order of sort as arguments.
• let the sort function return the output csv file (or its instance) instead
  of saving the file in some location.
• so that the calling function can decide what to do with the retured file.

1. generate-markdown

• lets make one function which outputs exactly one block (the struture in the
  word template.md file)
• this function will work only on one 1 row of csv text stream and output the
  text stream for 1 block of the output.
• once again let the caller function pass the csv stream as input and catch
  the output stream as a return value.
• the idea here is that we can reuse this function in multiple places, where
  we need to generate md file for entire library or for topic based words, or
  for md files split as per word initials etc.

and then write a parent function which calls these and does the needed thing as
per the type of output md file needed.
I am still working on this, that is the types of files we need to create. But
they will be something like -

• entire library in 1 file
• 1 file each for 1 alphabet (like A.md will contain all words starting with the
  letter 'a', B.md for 'b' and so on..)
• 1 file for each topic

Thanks! I can make the optimizations for the generate-markdown script fairly quickly so I'll do those first.

from marathi-shabd.

from marathi-shabd.

I can start working on those in the afternoon.

from marathi-shabd.

• input file will be the db.csv
• output will be a markdown file which will be used on the github pages website. for now it will the be the home page of the site.
• for now, a user will have to manual search for a word of interest (or can also use the browser's search function.)

from marathi-shabd.

prerequisites -

• have a csv file with content in en and mr columns, at least.
• have a template markdown file for the output

steps -

1. read the csv file
2. create a new markdown file from the template
3. extract the en and mr words from a row of the csv
4. fill the extracted words in the markdown file
5. repeat 3-4 till all rows of csv are done

from marathi-shabd.

Do you have the markdown file template ready?

from marathi-shabd.

Do you have the markdown file template ready?

Yes. It's there in the template folder. Not in a template shape right now but more like an example.

If you want the exact template with placeholders, then I will prepare it later today. But it won't be much different for the example file that is present there currently. It's it suits you, you can begin with that and later update your script once the final template is ready.

The example file shows 3 different ways to arrange the output. Please use the 1st option for now.

from marathi-shabd.

template is added. pls check the explanation in the readme file in the template folder.

from marathi-shabd.

I merged part of the PR #13 into main branch. tested ok at my end.
I will close the PR.

There are some enhancements that can be done. I will think and let you know.

from marathi-shabd.

Hey, do you have anything for me? I have time to work on the project.

from marathi-shabd.

from marathi-shabd.

w.r.t to the existing scripts there are some optimisations that can be done.
lets try to write programs using the unix philosophy. basically these
2 points for now -

• Write programs that do one thing and do it well.
• Write programs to work together.

applying this to your sort and gen-md scripts, we can do the following -

1. sort -

• make the sorting function universal/generic instead of specific to the input
  csv format.
• like, determine the number of columns in the csv from the number of elements
  in the top row instead of hardcoded values.
• when sorting we can pass both the column index to be used for sorting and
  the order of sort as arguments.
• let the sort function return the output csv file (or its instance) instead
  of saving the file in some location.
• so that the calling function can decide what to do with the retured file.

2. generate-markdown

• lets make one function which outputs exactly one block (the struture in the
  word template.md file)
• this function will work only on one 1 row of csv text stream and output the
  text stream for 1 block of the output.
• once again let the caller function pass the csv stream as input and catch
  the output stream as a return value.
• the idea here is that we can reuse this function in multiple places, where
  we need to generate md file for entire library or for topic based words, or
  for md files split as per word initials etc.

and then write a parent function which calls these and does the needed thing as
per the type of output md file needed.
I am still working on this, that is the types of files we need to create. But
they will be something like -

• entire library in 1 file
• 1 file each for 1 alphabet (like A.md will contain all words starting with the
  letter 'a', B.md for 'b' and so on..)
• 1 file for each topic

from marathi-shabd.

i have hosted the website with some dummy links under the "browse" section. pls have a look at it. you'll get an idea of the type of outputs we need to generate.

from marathi-shabd.

I have merged pr #18. Thanks!

I will now create a .py file with pseudo code for the parent function to make output md file for entire library and other types of output files (topic, alphabetical etc.). You can then use that to write your code.

from marathi-shabd.

@zarbod hi, did you see the .py files I added in the src folder and the pseudo code added in those? I have updated part of the db.csv file and would like to create atleast the md output file for all words (the entire library link on the website). Pls let me know when you are planning to implement those scripts. In case any part is not understood, let me know.

from marathi-shabd.

thanks. please make sure to pull the latest repo first since I made some updates.
also on the website as of now 3 links are having dummy files (entire lib, topics and "a" initial words).
Once your scripts are ready, we can run those on the db.csv file and put the md files containing the actual words from the database onto these links :)

from marathi-shabd.

@zarbod now that the filter script is done, we could continue with the gen-out and gen-block files so that we can use them together to generate the specific MD files.

Let me know if you can start on these.

from marathi-shabd.

@zarbod
तू यावर काम चालू केलं आहेस का? तुला जर वेळ लागणार असेल तर सांग. तसं असेल तर त्या दरम्यान मी पण माझ्याबाजूने program लिहायला प्रयत्न करून बघतो. मला browse site ची पानं शक्य तितकी लवकर अपलोड करायची आहेत म्हणून.

from marathi-shabd.

from marathi-shabd.

चालेल. 👍🏼

from marathi-shabd.

Closing this since the basic operation is working fine. Will open separate issues for specific enhancements.

from marathi-shabd.