Hello,

Interesting tool! I'm trying it out to create a blog post from a Jupyter notebook. I'm getting a File Not Found error when exporting a notebook:

  1 $ blogger convert binary_img_classification_pipeline-pytorch.ipynb -b pytorch_image_classification
  2 
  3 CONVERTING 1 FILES
  4 :: Meta tags: <!-- --> Not found
  5 :: Got topic, 
  6 :: Found 8 images
  7 :: Detected DATA URI img. Writing to pytorch_image_classification/images/binary_img_classification_pipeline-pytorch/image_1.png
  8 :: Detected DATA URI img. Writing to pytorch_image_classification/images/binary_img_classification_pipeline-pytorch/image_2.png
  9 Traceback (most recent call last):
 10   File "/Users/ker/miniconda2/envs/sci/bin/blogger", line 11, in <module>
 11     sys.exit(cli())
 12   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/click/core.py", line 764, in __call__
 13     return self.main(*args, **kwargs)
 14   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/click/core.py", line 717, in main 
 15     rv = self.invoke(ctx)
 16   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
 17     return _process_result(sub_ctx.command.invoke(sub_ctx))
 18   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/click/core.py", line 956, in invoke
 19     return ctx.invoke(self.callback, **ctx.params)
 20   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/click/core.py", line 555, in invoke
 21     return callback(*args, **kwargs)
 22   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/click/decorators.py", line 64, in new_func
 23     return ctx.invoke(f, obj, *args, **kwargs)
 24   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/click/core.py", line 555, in invoke
 25     return callback(*args, **kwargs)
 26   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/blogger_cli/commands/cmd_convert.py", line 81, in cli
 27     filenames_meta = convert_and_copyfiles(ctx)
 28   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/blogger_cli/commands/convert_utils/classifier.py", line 22, in convert_and_copyfiles
 29     html_filename_meta = converter(ctx, file)
 30   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/blogger_cli/converter/ipynb_to_html.py", line 22, in convert_and_copy_to_blog
 31     html_body, meta)
 32   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/blogger_cli/converter/ipynb_to_html.py", line 141, in write_html_and_ipynb
 33     ipynb_topic_filename)
 34   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/blogger_cli/converter/extractor.py", line 30, in extract_and_write_static
 35     extract_images(ctx, images, static_path, filename, blog_post_dir)
 36   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/blogger_cli/converter/extractor.py", line 110, in extract_images
 37     blog_post_dir)
 38   File "/Users/ker/miniconda2/envs/sci/lib/python3.6/site-packages/blogger_cli/converter/extractor.py", line 128, in extract_and_write_uri
 39     with open(static_path, 'wb') as wf:
 40 FileNotFoundError: [Errno 2] No such file or directory: 'pytorch_image_classification/images/binary_img_classification_pipeline-pytorch/image_2.png'
 41 (

The contents of the images/ dir after executing the above command is:

$ ls -l pytorch_image_classification/images/binary_img_classification_pipeline-pytorch/
total 24
-rw-r--r--@ 1 azalcman  staff  9235 Aug  2 15:55 image_1.png

The notebook is: https://github.com/commandlinegirl/machine-learning/blob/master/histopathologic-cancer-detection/binary_img_classification_pipeline-pytorch.ipynb

Have I missed anything?

This feature would allow the extraction of images from urls to local directory for permanent accesibility.
For ipynb it would reduce the junk in the main html as images in ipynb are represented by data uri.
Similarly the images from urls will be downloaded and written to images dir.

Extraction Process

The extraction is done from original files. If the file is ipynb data uri is extracted from src using beautifulsoup and it is then encoded in base64 to bytes and written as a png files. The html_tree is updated with new src of the dir of the images.

If the file contains urls then urllib is used to download and write the files and src in html tree is again updated using beautifulsoup.

Storing Images

The directory where images are stored can be specified by user in;
- Respective blog's config
- Using --img-dir option during conversion
- If neither of above is given. User is prompted to use the folder to store blog posts to store images also.

After the image folder is obtained a folder of same name as posts file is created and the images name is given as image_1.png, image_2.png etc.

Getting src path

The relative path between images and blog_posts is obtained by changing the python dir to the dir of blog_posts as using os.relpath by giving full path of image as parameter.

Testing should be added for rapid feature development and avoid breaking functionalities.

Since click supports colors, blogger should try to support it. Especially for better highlighting warnings and errors.

Overview

There is currently no easy way to subscribe to posts. We have to use some third party tools. However we can setup a flask server in heroku to manage the subscription system ourselves. This tool will be in separate repository and will have /blogname route with same designs.

Working

There will be a html form with subscribe button sending post requests to server. Now the server will send confirmation email about the information to user. After being clicked it will redirect to server's route for that blog and added into database column of that blogname. The notification will be sent everytime a blogpost is ready via blogger which invokes the server to send mails to users.

A new notify or publish command may be added with blogger cli for this feature.

Although converting ipynb files to and from blog folder works, it does not seem to be the case for md files.

When inside the blog dir, copying of md files to destination dir should be skipped.

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

A templating feature would allow users to write metadata on top of files which will be available as variables in the blog_post_layout. Users can create their own template and use jinja templating to place these variables as such.
Example Metadata:

*/
title : some title
date : Tues 9 May 2019
tags: ['reviews','programming','books']
author: my name
*/

Then this could be parsed and passed as variables to the template selected by user.

Example Template:

<div id='post-heading'>
      <h1 id='post-title'> {{ title }} </h1>
     <span>{{ meta.author }}</span><small>{{ meta.date }}</small>
     {% for tag in meta.tags %}
         <span>tag<span>
     {% endfor %}
</div>

<div id='post-body'>
    <p> {{ body }} </p>
</div>

There are prebuilt variable as title and body which can be accesed easily whereas the user defined metadata variables have to be accessed by the meta.var syntax.

Without affecting the required dependencies git support can be added. User can install dulwich to through pip to access the new git command. It is useful for some constrained environments.

Although it may be distributed ready made if installation happens through custom installation.

Overview

Topic based indexing can be enabled by the user by modifying his index file. This issue is linked to #2
Same metadata variables will be passed through the index template.

Example of post metadata
*/
topic: Review
tags: ['programming', 'review']
author: name
data: date
*/

Example of index template

{{ html_head }}
{{ navbar }}
 <div class="container">
        <div class='blog_list'>

            {% if meta.topic %}
                  <h1 class='meta topic'> {{ meta.topic }} </h1>
            {% endif %}

            {% for link, title in post_info.items() %}
            
            <ul>
                <li><a href="{{ link }}">{{ title }}</a> <p> </li>
            </ul>
            {% endfor %}

        </div>
    </div>
{{ google_analytics }}

Caveats
There are some difficulties in doing this as it is very difficult to extract index for meta infos and placing them back so any meta information should be in class 'meta varname' to be extracted properly.

This should be the default feature so no index template should be created by user to achieve this feature.

This is very desirable because of many dependencies. Will look into some projects to learn from.

A global or per blog config file to cover things like div names of blog index, parser and other such stuffs.

The basic idea is to load and override default attrs from a config.

I think basic configparser from stdlib will do the job.

According to documentation relative folders can be shown by

blog_posts_dir : /posts

But / represents root dir and wont work it should mention posts/ instead.