posativ / acrylamid Goto Github PK
View Code? Open in Web Editor NEW(unmaintained) static blog generator in python with incremental rendering ⛺
Home Page: http://posativ.org/acrylamid/
License: Other
(unmaintained) static blog generator in python with incremental rendering ⛺
Home Page: http://posativ.org/acrylamid/
License: Other
Here http://sebix.github.com/ are some changes that make the template scalable to almost every device/screen size.
Sometimes it is useful to break summarization after a given keyword, e.g. when you have big images following.
The summarize filter always inserts a dot after the summarize_identifier
.
I would remove it, but we should maybe discuss it first :D
File acrylamid/filters/summarize.py
line 101.
Yes, Acrylamid is very fast due it's caching and incremental updates. But currently it has a lot of overhead. First every filter, no matter wether actually used, is initialized, leading into various interesting workarounds (see lazy import of docutils
in rst.py) that are definitely not needed. Secondly Acrylamid stores every intermediate as extra file in .cache/ which produces approximately 5,7 files per entry. I have a diff-based pickle in mind, that contains every intermediate as diff.
Another thing is threading (or due CPython's GIL "multiprocessing") to use multiple cores for rendering but this is not that trivial when using multiprocessing
because of their fork-model circumventing the GIL. In my initial test, lazy evaluating filters like hyphenation
make the performance very bad because they do not cache the pattern (not mentioning multiple language patterns). Thus Acrylamid should offer a way to process those expensive functions in a shared (=> single core) environment.
d0f8616 introduced a regression that needs to be fixed as fast as possible: this commit changes all id/guid from tag:domain.tld,$date$path
to tag:domain.tld/path,$date$path
, note the /path
! And we all know permalinks should never ever change!1
The old approach is inspired by Mark Pilgrim's blog post where only the domain itself is included as host identifier, but now conflicts with the "kind" of the feed (blog or tag). I am reverting this but this won't be the final solution.
Sorry for all of you running trunk and updated four days ago :P
This bug does happen randomly when you compile the second (!) time with acrylamid aco
and have a german LANG (so it must be month März). This is also mentioned on bugs.python.org as invalid bug: http://bugs.python.org/issue5398
Traceback (most recent call last):
File "/Users/ich/dev/acrylamid/acrylamid/__init__.py", line 271, in __init__
commands.autocompile(conf, env, **options.__dict__)
File "/Users/ich/dev/acrylamid/acrylamid/commands.py", line 223, in autocompile
compile(conf, env, **options)
File "/Users/ich/dev/acrylamid/acrylamid/commands.py", line 186, in compile
for html, path in v.generate(request):
File "/Users/ich/dev/acrylamid/acrylamid/views/articles.py", line 41, in generate
env=union(self.env, num_entries=len(entrylist)))
File "/Users/ich/dev/acrylamid/lib/python2.7/site-packages/Jinja2-2.6-py2.7.egg/jinja2/environment.py", line 894, in render
return self.environment.handle_exception(exc_info, True)
File "layouts/articles.html", line 1, in top-level template code
{% extends "base.html" %}
File "layouts/base.html", line 39, in top-level template code
{%- block content -%}
File "layouts/articles.html", line 15, in block "content"
<th>{{ months[0].date.strftime('%B')}}</th>
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 1: invalid continuation byte
and later from wordpress dump maybe.
and I should remove the dead code for filter exclusion. This is no longer required.
I would like to have a --help on every subcommand.
Without converting the document. Pelican [1] has done this but its GPL unfortunately.
[1] https://github.com/ametaireau/pelican/blob/master/pelican/readers.py
This also involves #25 – asset management.
Currently the "theming" support is just a default directory where you can put some templates and you have to copy your CSS/JS to output/
and exclude it from the acrylamid clean
. Acrylamid should feature at least basic asset support like so:
theme/
theme/
and continue as usualoutput/
while preserving the directory structurehelpers.event
so they can properly be ereasedWith this, a theme can be bundled as folder and can be used as a drop in.
Any thoughts on this?
2.1 is the first major version update of Markdown since three years, but it seem's the AsciiMathML plugin is not compatible with it:
File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/acrylamid-0.3.0-py2.7.egg/acrylamid/utils.py", line 375, in content
res = f.transform(res, self, *f.args)
File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/acrylamid-0.3.0-py2.7.egg/acrylamid/filters/md.py", line 62, in transform
return md.convert(text)
File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/__init__.py", line 287, in convert
newRoot = treeprocessor.run(root)
File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/treeprocessors.py", line 289, in run
text), child)
File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/treeprocessors.py", line 110, in __handleInline
data, patternIndex, startIndex)
File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/Markdown-2.1.1-py2.7.egg/markdown/treeprocessors.py", line 237, in __applyPattern
node = pattern.handleMatch(match)
File "/usr/local/Cellar/python/2.7.2/lib/python2.7/site-packages/acrylamid-0.3.0-py2.7.egg/acrylamid/filters/mdx_asciimathml.py", line 43, in handleMatch
math = asciimathml.parse(m.group(2).strip(), markdown.etree.Element, markdown.AtomicString)
AttributeError: 'module' object has no attribute 'etree'
We should add the filter's version to hash, so we can increment the version and Acrylamid will automatically re-render content with a bugfixed filter.
When you do a rsync
which takes some time it looks like Acrylamid crashed.
They are using a stem algorithm and have stemming rules for english and japanese, but using this particular search algorithm does not offer a search as the user might expect.
It would be nice if acrylamid supports ftp upload.
I suggest some changes:
commands.py:
def upload(conf, env):
"""Subcommand: upload -- Upload the compiled output on the ftp-Server
specified in conf.py"""
if ('ftp_server' not in conf) or ('ftp_user' not in conf) or ('ftp_password' not in conf):
raise AcrylamidException('FTP_SERVER, FTP_USER and FTP_PASSWORD not specified in conf.py')
if 'ftp_root' not in conf:
ftp_root = '/'
else:
ftp_root = conf['ftp_root']
ftp = FTP(conf['ftp_server'], conf['ftp_user'], conf['ftp_password'])
try:
for root, dirs, files in os.walk('output/'):
ftp.cwd(os.path.join(ftp_root, root[len('output/'):]))
remote_dirs = ftp.nlst()
for ldir in dirs:
if ldir not in remote_dirs:
ftp.mkd(os.path.join(ftp_root, root[len('output/'):], ldir))
for infile in files:
fd = open(os.path.join(root, infile), "r+")
fd.readline()
raw_uid = fd.readline()
raw_uid = raw_uid[4:-4]
name, uid = raw_uid.split('=')
if name == 'uid':
uid = uid.replace('"','')
ret = ftp.sendcmd("mlst " + infile)
ret_uid = ''
for elem in ret.split(';')[:-1]:
if elem.startswith('unique'):
ret_uid = elem[7:]
fd.seek(0)
if uid != ret_uid or ret_uid == '':
ftp.storbinary("STOR " + infile, fd)
fd.seek(0)
ret = ftp.sendcmd("mlst " + infile)
newret_uid = ''
for elem in ret.split(';')[:-1]:
if elem.startswith('unique'):
newret_uid = elem[7:]
buf = [fd.readline()]
fd.readline()
buf += ['<!--uid="' + newret_uid + '"-->\n']
buf += fd.readlines()
fd.seek(0)
fd.truncate()
fd.writelines(buf)
if(ret_uid == ''):
log.info("create " + os.path.join(root, infile))
else:
log.info("update " + os.path.join(root, infile))
else:
log.info("identical " + os.path.join(root, infile))
fd.close()
except Exception as e:
traceback.print_exc()
finally:
ftp.quit()
log.info('All files uploaded')
In the second line of each html, xml file must be the string "". The uid must be "" if the file is new or updated. The correct uid will be inserted by the uploay function. I have just edited base.html, rss.xm, atom.xml. But then there are problems with updating, because the default files are just used for creating a new blog. The user must edit the layout files manually
init.py
elif sys.argv[1] in ('upload',):
usage = "%prog " + sys.argv[1] + ""
options = []
elif args[0] in ('upload'):
try:
commands.upload(conf, env)
except AcrylamidException as e:
log.critical(e.message)
sys.exit(1)
Just add this to the right places. First one before parser creating, Last one before the last else
There are 4 new conf.py Variables, FTP_SERVER, FTP_USER, FTP_PASSWORD and FTP_ROOT. FTP_ROOT can be ommitted and defaults to '/'
I have tested it with only one server. But there shouldn't be problems with other servers, because the output of the mlst command is specified in an RFC.
When I change a variable in conf.py
, compile and autocompile don't think they had anything to change!
The summarizing is done using a HTMLParser and several handlers when a given tag starts, end, data comes in or an entity is there. The latter will cause this wrong behaviour. Hy-phe-na-ti-on is count as five words instead of one.
see the embedding sections https://en.wikipedia.org/wiki/MathML
Blog compiled in 0.19s
update [0.52s] output/2012/polipo-a-caching-web-proxy/index.html
update [0.08s] output/rss/index.html
update [0.01s] output/atom/index.xml
update [0.07s] output/atom/full/index.xml
update [0.35s] output/index.html
update [0.01s] output/tag/os-x/index.html
update [0.00s] output/tag/polipo/index.html
update [0.00s] output/tag/proxy/index.html
Blog compiled in 1.52s
update [0.31s] output/2012/polipo-a-caching-web-proxy/index.html
update [0.05s] output/rss/index.html
update [0.01s] output/atom/index.xml
update [0.06s] output/atom/full/index.xml
Blog compiled in 1.69s
update [0.34s] output/2012/polipo-a-caching-web-proxy/index.html
update [0.05s] output/rss/index.html
update [0.01s] output/atom/index.xml
update [0.06s] output/atom/full/index.xml
Blog compiled in 2.06s
update [0.31s] output/2012/polipo-a-caching-web-proxy/index.html
update [0.05s] output/rss/index.html
update [0.01s] output/atom/index.xml
update [0.06s] output/atom/full/index.xml
Blog compiled in 2.04s
update [0.31s] output/2012/polipo-a-caching-web-proxy/index.html
update [0.05s] output/rss/index.html
update [0.01s] output/atom/index.xml
update [0.06s] output/atom/full/index.xml
Blog compiled in 2.41s
Google has some uber python script: https://github.com/ralsina/nikola/blob/master/nikola/sitemap_gen.py but we may short it up to 100 LoC...
Specific for mako
and jinja2
filters: when referenced entries change (e.g. the title), these changes are not updated.
write a BSD touch
replacement for Linux.
The current has_changed algorithm only detects content and layout changes. If a file vanished through various reasons (drafting, moving) it can't be traced, because it's outside the observed array.
workaround: remove physical output file or force regeneration.
I would appreciate it, when acrylamid used relative paths for linking files in renderd html-output.
Using ./ instead of / would support all possibilites, those blogs which are in a root directoy and those which are located in subfolders.
Another possibilty is to add a config-option for the path, because www_root is only used in feeds.
A few filters (metalogo and pygments) use some CSS rules and we have currently no way to add filter-specific CSS (inline or as stylesheet). We should add a method called "inject" which takes no arguments and returns a dictionary with {type: text}
. Acrylamid itself should decide (based on user-configuration) wether to add this file as condensed file into the header, as inline or, for HTML5 using scoped CSS.
Stylesheet/js/html injection is a per-filter thing and it should not be injected to a view where no such filter is applied.
There's a fork with some enhancements to smarty pants (typography filter) for the german language. When we're already work on this filter, we should move each sub-function to a big single sub regex, because smarty pants is aweful slow (15% of the whole compilation process).
Here a quick gprof graph: http://rw.posativ.org/xq9
Jinja supports kinda this: http://jinja.pocoo.org/docs/extensions/#i18n-extension
Maybe we should just write i18n values into templates instead of gettexting everything. At least Acrylamid should be able to create english and german templates.
As we support only Python ≥ 2.6 we port our codebase more easier to Python 3... status of our dependencies:
Ressources for porting to Python 3:
Currently, for each different filter(s) chain the whole content is re-compiled, but it is only necessary to compile each (recursively of course) different "branch" solely if it has no common anchestor/node.
pages -> Markdown, summarize, typo
entry -> Markdown, typo
feed -> Markdown, h1
Should compile this way with much performance improvements:
Markdown --> h1 -> done!
\-> typo --> done!
\-> summarize -> done!
instead of:
Markdown --> h1 -> done!
Markdown --> typo --> done!
Markdown --> typo --> summarize --> done!
Just a minor. Insert a rst
or Markdown
to this entry if they use their native formats.
I have currently this in a branch, using jinja2 as additional filter before Markdown:
{% for tr in env.translations(entry.identifier) -%}
- [{{ tr.lang }}: {{ tr.title }}]({{ tr.permalink }})
{% endfor %}
Any feedback to this syntax?
A suggestion from @sebix to add the popular pingback and/or trackback support to acrylamid. While this is a great idea, it almost impossible to implement it in Acrylamid (at least bi-directional communication):
plus drafted entries must not show up in generated sitemap.
With bbf5a08 we can use alternate template engines. On reddit someone requested Mako.
In readers.py
there is a line in the distinguish
function that strips quotes from metadata values:
return unicode(value.strip('"').strip("'"))
This means that an entry cannot have a title starting or ending with a quote. So when I want to migrate my current website to an Acrylamid based site, the titles of e.g. these pages need to be changed:
It's not a huge deal, but I do wonder why Acrylamid has this 'restriction'.
items_per_page: 10
len(entries): 20
will render an empty third page.
For example I can't make a rule that only shows one specific tag on one site. I can use
"condition": lambda e: 'mytag' in e.tags
but this will also route each additional tag in those entries and that's not my intention. Maybe we can add some context, so
"condition": lamda c, e: c.tag == 'mytag'
is possible (in the knowledge, that the used view is the tag view of course).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.