posativ / acrylamid Goto Github PK

Here http://sebix.github.com/ are some changes that make the template scalable to almost every device/screen size.

Sometimes it is useful to break summarization after a given keyword, e.g. when you have big images following.

The summarize filter always inserts a dot after the summarize_identifier.

I would remove it, but we should maybe discuss it first :D

File acrylamid/filters/summarize.py line 101.

Yes, Acrylamid is very fast due it's caching and incremental updates. But currently it has a lot of overhead. First every filter, no matter wether actually used, is initialized, leading into various interesting workarounds (see lazy import of docutils in rst.py) that are definitely not needed. Secondly Acrylamid stores every intermediate as extra file in .cache/ which produces approximately 5,7 files per entry. I have a diff-based pickle in mind, that contains every intermediate as diff.

Another thing is threading (or due CPython's GIL "multiprocessing") to use multiple cores for rendering but this is not that trivial when using multiprocessing because of their fork-model circumventing the GIL. In my initial test, lazy evaluating filters like hyphenation make the performance very bad because they do not cache the pattern (not mentioning multiple language patterns). Thus Acrylamid should offer a way to process those expensive functions in a shared (=> single core) environment.

d0f8616 introduced a regression that needs to be fixed as fast as possible: this commit changes all id/guid from tag:domain.tld,$date$path to tag:domain.tld/path,$date$path, note the /path! And we all know permalinks should never ever change!1

The old approach is inspired by Mark Pilgrim's blog post where only the domain itself is included as host identifier, but now conflicts with the "kind" of the feed (blog or tag). I am reverting this but this won't be the final solution.

Sorry for all of you running trunk and updated four days ago :P

This bug does happen randomly when you compile the second (!) time with acrylamid aco and have a german LANG (so it must be month März). This is also mentioned on bugs.python.org as invalid bug: http://bugs.python.org/issue5398

Traceback (most recent call last):
  File "/Users/ich/dev/acrylamid/acrylamid/__init__.py", line 271, in __init__
    commands.autocompile(conf, env, **options.__dict__)
  File "/Users/ich/dev/acrylamid/acrylamid/commands.py", line 223, in autocompile
    compile(conf, env, **options)
  File "/Users/ich/dev/acrylamid/acrylamid/commands.py", line 186, in compile
    for html, path in v.generate(request):
  File "/Users/ich/dev/acrylamid/acrylamid/views/articles.py", line 41, in generate
    env=union(self.env, num_entries=len(entrylist)))
  File "/Users/ich/dev/acrylamid/lib/python2.7/site-packages/Jinja2-2.6-py2.7.egg/jinja2/environment.py", line 894, in render
    return self.environment.handle_exception(exc_info, True)
  File "layouts/articles.html", line 1, in top-level template code
    {% extends "base.html" %}
  File "layouts/base.html", line 39, in top-level template code
    {%- block content -%}
  File "layouts/articles.html", line 15, in block "content"
    <th>{{ months[0].date.strftime('%B')}}</th>
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 1: invalid continuation byte

and later from wordpress dump maybe.

and I should remove the dead code for filter exclusion. This is no longer required.

I would like to have a --help on every subcommand.

Without converting the document. Pelican [1] has done this but its GPL unfortunately.

[1] https://github.com/ametaireau/pelican/blob/master/pelican/readers.py

http://misaka.61924.nl/

This also involves #25 – asset management.

Currently the "theming" support is just a default directory where you can put some templates and you have to copy your CSS/JS to output/ and exclude it from the acrylamid clean. Acrylamid should feature at least basic asset support like so:

• assets and layouts in theme/
• during compilation
  • get template out of theme/ and continue as usual
  • move assets (== everything that is not a template) to output/ while preserving the directory structure
  • assets should be tracked by helpers.event so they can properly be ereased
  • only copy assets when the timestamp as changed but remove them
• don't include them into the sitemap ;-)

With this, a theme can be bundled as folder and can be used as a drop in.
Any thoughts on this?

2.1 is the first major version update of Markdown since three years, but it seem's the AsciiMathML plugin is not compatible with it:

File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/acrylamid-0.3.0-py2.7.egg/acrylamid/utils.py", line 375, in content
      res = f.transform(res, self, *f.args)
    File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/acrylamid-0.3.0-py2.7.egg/acrylamid/filters/md.py", line 62, in transform
      return md.convert(text)
    File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/__init__.py", line 287, in convert
      newRoot = treeprocessor.run(root)
    File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/treeprocessors.py", line 289, in run
      text), child)
    File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/treeprocessors.py", line 110, in __handleInline
      data, patternIndex, startIndex)
    File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/treeprocessors.py", line 237, in __applyPattern
      node = pattern.handleMatch(match)
    File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/acrylamid-0.3.0-py2.7.egg/acrylamid/filters/mdx_asciimathml.py", line 43, in handleMatch
      math = asciimathml.parse(m.group(2).strip(), markdown.etree.Element, markdown.AtomicString)
  AttributeError: 'module' object has no attribute 'etree'

We should add the filter's version to hash, so we can increment the version and Acrylamid will automatically re-render content with a bugfixed filter.

When you do a rsync which takes some time it looks like Acrylamid crashed.

They are using a stem algorithm and have stemming rules for english and japanese, but using this particular search algorithm does not offer a search as the user might expect.

It would be nice if acrylamid supports ftp upload.

I suggest some changes:
commands.py:

def upload(conf, env):
    """Subcommand: upload -- Upload the compiled output on the ftp-Server
    specified in conf.py"""
    if ('ftp_server' not in conf) or ('ftp_user' not in conf) or ('ftp_password' not in conf):
        raise AcrylamidException('FTP_SERVER, FTP_USER and FTP_PASSWORD not specified in conf.py')
    if 'ftp_root' not in conf:
        ftp_root = '/'
    else: 
        ftp_root = conf['ftp_root']
    ftp = FTP(conf['ftp_server'], conf['ftp_user'], conf['ftp_password'])
    try:
        for root, dirs, files in os.walk('output/'):
            ftp.cwd(os.path.join(ftp_root, root[len('output/'):]))
            remote_dirs = ftp.nlst()
            for ldir in dirs:
                if ldir not in remote_dirs:
                    ftp.mkd(os.path.join(ftp_root, root[len('output/'):], ldir))
            for infile in files:
                fd = open(os.path.join(root, infile), "r+")
                fd.readline()
                raw_uid = fd.readline()
                raw_uid = raw_uid[4:-4]
                name, uid = raw_uid.split('=')
                if name == 'uid':
                    uid = uid.replace('"','')
                ret = ftp.sendcmd("mlst " + infile)
                ret_uid = ''
                for elem in ret.split(';')[:-1]:
                    if elem.startswith('unique'):
                        ret_uid = elem[7:] 
                fd.seek(0)
                if uid != ret_uid or ret_uid == '':
                    ftp.storbinary("STOR " + infile, fd)
                    fd.seek(0)
                    ret = ftp.sendcmd("mlst " + infile)
                    newret_uid = ''
                    for elem in ret.split(';')[:-1]:
                        if elem.startswith('unique'):
                            newret_uid = elem[7:]
                    buf = [fd.readline()]
                    fd.readline()
                    buf += ['<!--uid="' + newret_uid + '"-->\n']
                    buf += fd.readlines()
                    fd.seek(0)
                    fd.truncate()
                    fd.writelines(buf)
                    if(ret_uid == ''):
                        log.info("create " + os.path.join(root, infile))
                    else:
                        log.info("update " + os.path.join(root, infile))
                else:
                    log.info("identical " + os.path.join(root, infile))
                fd.close()


    except Exception as e:
        traceback.print_exc()

    finally:
        ftp.quit()
    log.info('All files uploaded')

In the second line of each html, xml file must be the string "". The uid must be "" if the file is new or updated. The correct uid will be inserted by the uploay function. I have just edited base.html, rss.xm, atom.xml. But then there are problems with updating, because the default files are just used for creating a new blog. The user must edit the layout files manually

init.py

elif sys.argv[1] in ('upload',):
            usage = "%prog " + sys.argv[1] + ""
            options = []

elif args[0] in ('upload'):
            try:
                commands.upload(conf, env)
            except AcrylamidException as e:
                log.critical(e.message)
                sys.exit(1)

Just add this to the right places. First one before parser creating, Last one before the last else

There are 4 new conf.py Variables, FTP_SERVER, FTP_USER, FTP_PASSWORD and FTP_ROOT. FTP_ROOT can be ommitted and defaults to '/'
I have tested it with only one server. But there shouldn't be problems with other servers, because the output of the mlst command is specified in an RFC.

When I change a variable in conf.py, compile and autocompile don't think they had anything to change!

The summarizing is done using a HTMLParser and several handlers when a given tag starts, end, data comes in or an entity is there. The latter will cause this wrong behaviour. Hy-phe-na-ti-on is count as five words instead of one.

see the embedding sections https://en.wikipedia.org/wiki/MathML

Google has some uber python script: https://github.com/ralsina/nikola/blob/master/nikola/sitemap_gen.py but we may short it up to 100 LoC...

Specific for mako and jinja2 filters: when referenced entries change (e.g. the title), these changes are not updated.

write a BSD touch replacement for Linux.

The current has_changed algorithm only detects content and layout changes. If a file vanished through various reasons (drafting, moving) it can't be traced, because it's outside the observed array.

workaround: remove physical output file or force regeneration.

http://pelican.notmyidea.org/en/latest/settings.html#tag-cloud

I would appreciate it, when acrylamid used relative paths for linking files in renderd html-output.
Using ./ instead of / would support all possibilites, those blogs which are in a root directoy and those which are located in subfolders.

Another possibilty is to add a config-option for the path, because www_root is only used in feeds.

A few filters (metalogo and pygments) use some CSS rules and we have currently no way to add filter-specific CSS (inline or as stylesheet). We should add a method called "inject" which takes no arguments and returns a dictionary with {type: text}. Acrylamid itself should decide (based on user-configuration) wether to add this file as condensed file into the header, as inline or, for HTML5 using scoped CSS.

Stylesheet/js/html injection is a per-filter thing and it should not be injected to a view where no such filter is applied.

There's a fork with some enhancements to smarty pants (typography filter) for the german language. When we're already work on this filter, we should move each sub-function to a big single sub regex, because smarty pants is aweful slow (15% of the whole compilation process).

Here a quick gprof graph: http://rw.posativ.org/xq9

Jinja supports kinda this: http://jinja.pocoo.org/docs/extensions/#i18n-extension
Maybe we should just write i18n values into templates instead of gettexting everything. At least Acrylamid should be able to create english and german templates.

As we support only Python ≥ 2.6 we port our codebase more easier to Python 3... status of our dependencies:

• jinja2 (experimental)
• Markdown (compatible)
• translitcodec (not started yet)

Ressources for porting to Python 3:

• http://docs.python.org/py3k/howto/pyporting.html
• http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/
• http://docs.pythonsprints.com/python3_porting/py-porting.html

Currently, for each different filter(s) chain the whole content is re-compiled, but it is only necessary to compile each (recursively of course) different "branch" solely if it has no common anchestor/node.

pages -> Markdown, summarize, typo
entry   -> Markdown, typo
feed    -> Markdown, h1

Should compile this way with much performance improvements:

Markdown --> h1 -> done!
         \-> typo --> done!
                  \-> summarize -> done!

instead of:

Markdown --> h1 -> done!
Markdown --> typo --> done!
Markdown --> typo --> summarize --> done!

• https://github.com/ddfreyne/nanoc/tree/master/lib/nanoc/extra/validators

Just a minor. Insert a rst or Markdown to this entry if they use their native formats.

I have currently this in a branch, using jinja2 as additional filter before Markdown:

{% for tr in env.translations(entry.identifier) -%}
- [{{ tr.lang }}: {{ tr.title }}]({{ tr.permalink }})
{% endfor %}

Any feedback to this syntax?

A suggestion from @sebix to add the popular pingback and/or trackback support to acrylamid. While this is a great idea, it almost impossible to implement it in Acrylamid (at least bi-directional communication):

Pingback

• http://www.hixie.ch/specs/pingback/pingback
• requires modification of the HTTP header and/or add a custom link attribute
• requires a executable that reacts on XML-RPC calls

Traceback

• http://www.movabletype.org/documentation/trackback_manual.html#trackback%20ping%20format
• requires a fixed URL where a executable responds to a GET/POST request

Refback

• https://en.wikipedia.org/wiki/Refback
• requires access to access log and valid (non-spamming) referers

plus drafted entries must not show up in generated sitemap.

With bbf5a08 we can use alternate template engines. On reddit someone requested Mako.

In readers.py there is a line in the distinguish function that strips quotes from metadata values:

return unicode(value.strip('"').strip("'"))

This means that an entry cannot have a title starting or ending with a quote. So when I want to migrate my current website to an Acrylamid based site, the titles of e.g. these pages need to be changed:

• http://www.vlent.nl/weblog/2011/02/08/change-workflow-content-type-default/ (title ending with a quote)
• http://www.vlent.nl/weblog/2012/08/09/attributeerror-querymethodid-when-creating-object/ (title starting with a quote)

It's not a huge deal, but I do wonder why Acrylamid has this 'restriction'.

items_per_page: 10
len(entries): 20

will render an empty third page.

For example I can't make a rule that only shows one specific tag on one site. I can use

"condition": lambda e: 'mytag' in e.tags

but this will also route each additional tag in those entries and that's not my intention. Maybe we can add some context, so

"condition": lamda c, e: c.tag == 'mytag'

is possible (in the knowledge, that the used view is the tag view of course).