ebook-format=kf8 doesn't have/use monospace font? about asciidoctor-epub3 HOT 36 CLOSED

Can I ask how you are verifying that the .mobi does not use monospace font? Are you looking at it in a particular viewer or examining the source?

from asciidoctor-epub3.

I used two asciidoc documents, one the gitmagic manual from github, the other a private example text; converted them using asciidoctor-epub3 to mobi files; then transferred the latter via sneakernet (USB cord) to a Kindle Paperwhite.

from asciidoctor-epub3.

from asciidoctor-epub3.

A mobi file is actually two files in one, a mobi file (aka mobi7) and a kf8 (aka mobi8). (You can extract the contents to inspect what these files contain using https://github.com/kevinhendricks/KindleUnpack).

When kindlegen generates the mobi7 file, which is loaded by the Kindle Paperwhite, it drops custom fonts. However, it still attempts to reference these fonts in the markup. Here's sample markup for monospace text in the mobi7 file:

<code><tt><font face="M+ 1mn" color="#191918">epub:type</font></tt></code>

The M+ 1mn font is being requested, but it is not present in the mobi7 file. This causes the viewer to fallback to default font, sans-serif, which explains why you are not seeing monospace text.

You would expect the face attribute value to be M+ 1mn, monospace as it in the CSS. However, it seems that kindlegen stops parsing after it encounters a font family with a space in the name (perhaps a bug). We can use make use of fallback fonts in the CSS as long as we avoid fonts with spaces in the name.

The fix for this is to use built-in font names for the mobi7 file. This is possible using a media query:

@media amzn-mobi {
  code, kbd, pre {
    font-family: monospace;
  }
}

With this rule added, the HTML generated by kindlegen in the mobi7 file ends up as:

<code><tt><font face="monospace" color="#191918">epub:type</font></tt></code>

Exactly what we want.

I'll switch to built-in font names for all elements for mobi7.

Keep in mind, however, that optimizing for mobi7 is not a design goal of Asciidoctor EPUB3. We will make changes where reasonable, as in this case, but don't expect it to look particularly sharp. mobi7 is far to limited to achieve much more than rudimentary look & feel.

from asciidoctor-epub3.

PR sent. Feel free to review.

from asciidoctor-epub3.

After checking your PR and master again, I think I may have misfiled my report. Both your PR and the original asciidoctor-epub3 do seem to respect the monospace if I selected the Publisher Font, i.e., the body text is a proportional typeface and the code text is a monospaced one. The space in "M+ 1mn" does not appear to be the culprit.

However I typically switch to Futura (strange, I know) and there is no longer a distinction between the typefaces used by body and code -- both are set in Futura. I concede that a case could be made that this is "expected behavior" (I did ask for Futura).

I assume the solution is to hard-wire a typeface other than Noto Serif into data/styles/epub3-font.css (e.g., by invoking a google webfont that's closest to Futura). If I'm not mistaken, there is no other way to shadow the default, e.g., on the command-line. While it would be nice to have the KPW's typeface selection affect only the body, I realize this may be strictly outside the scope of asciidoctor-epub3, and I can live with it.

from asciidoctor-epub3.

I think you pointed to a real issue. The mobi7 was pointing to a font which is not available in the mobi7, which is invalid. Though, I'll dig a bit deeper by doing some device testing.

The space in "M+ 1mn" does not appear to be the culprit.

The space isn't a problem by itself (it is still a valid font name). The problem is that kindlegen doesn't pass on the fonts listed after a font name with a space in it. So it was still a problem.

If I'm not mistaken, there is no other way to shadow the default, e.g., on the command-line.

To use your own font, you should provide your own CSS file.

...there is a slight problem here because we are always loading the epub3-fonts.css file, which isn't correct. We should allow all the stylesheets to be replaced.

While it would be nice to have the KPW's typeface selection affect only the body, I realize this may be strictly outside the scope of asciidoctor-epub3, and I can live with it.

Welcome to the strange world of EPUB3 readers :)

from asciidoctor-epub3.

Could you send a screenshot (or photo) of the Kindle Paperwhite rendering (preferably the sample epub in this repo on a page that has monospace text). When I see how it renders, it tells me a lot about what CSS it is applying.

from asciidoctor-epub3.

For the screenshots, please see https://github.com/ds26gte/scratchpad. (The jpegs seemed a bit too large to put here for posterity.)

from asciidoctor-epub3.

To use your own font, you should provide your own CSS file.

I'm OK with that, and I am able to do that successfully, but the Paperwhite holds the promise of allowing a a document to be typeface-agnostic to a limited extent (it's a safe bet that very few people share my enthusiasm for Futura for extended text). It gives the reader the freedom to change the typeface to one of a handful of suggested choices (Baskerville, Bookerly, etc) instead of sticking to the Publisher Font.

It stands to reason that the Paperwhite is using a CSS cascade to let the reader intersperse their own preferred typeface into the mix.

I wonder if asciidoctor-epub3 can exploit the CSS property annotation !important to ensure that the Paperwhite's CSS cascade chooses the reader-specified typeface for the body text while retaining the asciidoctor-epub3-specified typeface for the code snippets. I can experiment, although I'm hampered by not knowing what the Paperwhite's own CSS file is.

from asciidoctor-epub3.

Thank you for providing the screenshots! They correct my initial assumption. The Kindle Paperwhite is using the KF8 part of the MOBI file, which is excellent news. That brings us back to the problem that selecting a custom font changes all the fonts and not just the paragraph fonts. This seems to be a major error in the design of Kindle because it just doesn't make any sense to try to reduce the world down to a single font family. (Come on Kindle engineers, what are you thinking?) They should have font options for serif, sans-serif and monospace instead of a global font selection.

Now to find a workaround :)

from asciidoctor-epub3.

I wonder if asciidoctor-epub3 can exploit the CSS property annotation !important...

Quite possible. That's a good place to start.

I can experiment, although I'm hampered by not knowing what the Paperwhite's own CSS file is.

That's been my frustration all alone trying to style EPUB3 documents. Complete lack of transparency. It has taken a lot of trial and error.

from asciidoctor-epub3.

This might not be a cascading issue though (from what I understand about how Kindle works). It's very likely that it is looking at the font-family and if it doesn't see a value it likes, it just switches to sans-serif. The keyword to satisfy Kindle Paperwhite seems to be "Courier, Courier New". It must be hard-coded to look for that family and not change the face. But I still need to test that theory.

from asciidoctor-epub3.

I have good news! I did a bunch of experimentation in the device lab this afternoon and I discovered that this is a cascading issue after all. Your theory about using !important worked!

I don't know how strong the rule has to be, so I couldn't find a way other than !important, but I think this is a pretty clear cut case for when !important is warranted.

I've updated the pull request to include this change. (In the process, I also discovered that the text-rendering setting was crashing certain Kindle eInk devices).

It gives the reader the freedom to change the typeface to one of a handful of suggested choices (Baskerville, Bookerly, etc) instead of sticking to the Publisher Font.

We can now achieve this goal for all text other than preformatted text. \o/

from asciidoctor-epub3.

It turns out we had to completely unrelated bugs in this one issue, but to the reader they appeared to be the same bug.

from asciidoctor-epub3.

I'm going to do a bit more testing before I merge.

from asciidoctor-epub3.

Excellent! I tried out your modified issue-56 and it Just Works. A huge step toward making software manuals on the Kindle intelligently responsive to reader customization; thanks!

from asciidoctor-epub3.

That's great news!!! Thanks for testing!

from asciidoctor-epub3.

I did some more testing today and made some more discoveries. Update coming shortly.

from asciidoctor-epub3.

Here's what I learned after many hours a reserve and a few beers.

It appears that Kindle Voyage / Paperwhite is adding a declaration when the user selects a personal font, which is roughly equivalent to:

* {
  font-family: <selected font> !important;
}

We end up having to add !important pretty much everywhere that isn't prose, even on nested elements. I found a whole bunch more cases that were slipping through and got them patched up. For instance, syntax highlighted code was reverting to the user's personal font instead of remaining monospace.

Fortunately, we can do all of this inside the @media amzn-kf8 media query without blowing up the stylesheet :)

While debugging, I also discovered that iBooks was not recognizing our publisher font for prose (Noto Serif). The problem was that iBooks requires a declaration in a proprietary file inside the EPUB archive that enables what is the equivalent of the Publisher Font setting in Kindle (called Original in iBooks).

META-INF/com.apple.ibooks.display-options.xml

<?xml version="1.0" encoding="UTF-8"?>
<display_options>
<platform name="*">
<option name="specified-fonts">true</option>
</platform>
</display_options>

Now iBooks shows the book exactly the way we want it out of the box.

Unlike Kindle, when the user selects a custom font in iBooks, it applies CSS which is roughly equivalent to:

div, p, :not(pre) span {
  font-family: <selected font> !!!!!!!!!!!!!!!important;
}

You cannot override this setting.

While is is good for the reader, unfortunately, it doesn't catch all the prose atm. We're going to need to look into wrapping any "naked" prose in a span so that iBooks can control it like the Kindle Paperwhite / Voyage. I fixed one case (simple content inside a <dd>).

With these updates, I think we're really solid now!!

from asciidoctor-epub3.

I was also able to get custom font selection to work for other EPUB3 readers such as Gitden and Namo Pubtree.

Gitden seems to have a similar override as the Kindle Paperwhite / Voyage, but the overrides to have a parent selector (e.g., body h1). Namo Pubtree only overrides div, p and span (like iBooks), though it seems possible to be able to override these settings using !important.

I think we've got everything reasonable covered now.

from asciidoctor-epub3.

Namo Pubtree seems to apply the custom font to span elements where ever they appear, where is iBooks is more discerning and does not apply it to span elements inside a pre element.

from asciidoctor-epub3.

It appears that while Kindle for Android gives you a font selection, it has no affect on the text. Well, when it does start working, we'll be ready for it :)

from asciidoctor-epub3.

There's one minor bug which wasn't in your original fix, although it doesn't matter for someone willing to choose a Kindle-offered font:

Publisher Font (Noto Serif in this case) is no longer among the choices on the Kindle Paperwhite for asciidoctor-epub3-generated docs.

from asciidoctor-epub3.

from asciidoctor-epub3.

This is truly curious. The Publisher Font shows up when I convey the .mobi via the USB cord ("sneakernet") to the Paperwhite. It does not show up when I email the .mobi to it. In my previous bug reports I was exclusively using the sneakernet method. So I suspect the email method was stripping the Publisher Font all along.

I made sure the Paperwhite didn't have previous versions of the document when trying out both methods of transferring the .mobi to it.

from asciidoctor-epub3.

Emailing a .mobi to the Paperwhite does seem to be lossy compared to copying it via USB cord.

I have a gitmagic.mobi file of size 564028. When I email it to the Paperwhite, it lands as an .azw3 file of size 160004: practically a decimation. Googling suggests the lossiness may be a known phenomenon, and uncharitable commentary further suggests this is meant to be a disincentive to emailing "foreign" .mobis onto the Kindle. However, the USB method is also known as a reliable workaround.

So, this may not be an asciidoctor-epub3 problem after all.

from asciidoctor-epub3.

First, that's crazy. Second, I discovered how to work around it!

When you e-mail a file to Kindle, it seems to get routed through Kindle Direct Publishing (KDP), which ultimately invokes kindlegen on Linux (apparently in some special way). KDP is known to clobber font-related CSS rules. The same font-clobbering behavior can be observed when using the web-based interface for KDP.

After a doing a bunch of digging, I discovered this post on the KDP forums that provides a simple workaround. The workaround is to introduce a layer of indirection by moving the real CSS to a separate file and using @import to include it. As it turns out, the font-clobbering only resolves one level down while the viewer (i.e., web-browser) happily follows all references.

Fortunately, we can handle this in the packager so the change is completely transparent. I submitted #60 with this fix.

With that change, fonts are now preserved even when using KDP. That means we can get custom fonts (as well as the font override) for books published to the Kindle store!! \o/

from asciidoctor-epub3.

I still need to verify the e-mail .mobi works. It just seems like it should now work in theory.

from asciidoctor-epub3.

It would appear that this fix doesn't make it though the "Send to Kindle" filter. However, it seems to be generally recommended not to use that method to move files around. At least we know you can upload to KDP now, so some progress was made.

Amazon seems to be about as difficult as a vendor of technology can be about all this. Someone needs to knock them off their throne.

from asciidoctor-epub3.

uncharitable commentary further suggests this is meant to be a disincentive to emailing "foreign" .mobis onto the Kindle

Sounds like a typical Amazon response.

from asciidoctor-epub3.

When I email it to the Paperwhite, it lands as an .azw3 file of size 160004:

How did you find the file? I can't seem to locate it when it arrives on Kindle for Android.

from asciidoctor-epub3.

Okay, I can verify that our font CSS rule hack works when using "Send to Kindle", but we hit a secondary problem. All the font files have been removed from the archive. This appears intentional since the references to the font url in the @font-face declaration is xxxxxxxxxxxxxx. Thanks Amazon.

So there's not much we can do about the "Send to Kindle" problem.

from asciidoctor-epub3.

How did you find the file? I can't seem to locate it when it arrives on Kindle for Android.

For the Kindle Paperwhite at least (I have no experience with Kindle for Android), I connect it to my Ubuntu computer via USB, then cd /media/ds26gte/Kindle/documents (replace ds26gte with your username), and look for an .azw3 whose basename has some connection with the document's inner title. It is usually not the same as the basename of the .mobi file that got emailed to the device.

from asciidoctor-epub3.

I discovered that on the Android app, the file has the extension .prc, but it is still an .azw3 file.

I'm afraid to even think what these codebases at Amazon look like because I see absolutely no consistency from one app or tool to the next.

from asciidoctor-epub3.

What's important is that we have a pretty good understanding of what's going on now and that will enable us to make decisions as we move forward. In the meantime, keep that USB cable handy :)

from asciidoctor-epub3.