bmaupin / go-epub Goto Github PK

I think I'm asking for a feature?

My conclusion is based on behaviour, not on debugging: go-epub will leave the src attribute for images and other media untouched, right? It will not go and call addImage() for them.

Maybe it should? It would be really convenient. I'm assuming for the huge majority of epub use cases you'd want to keep images embedded in the epub.

Thanks!

Hi everyone,

As I mentioned in #28, I don't actually use this project any more. I've tried to maintain it over the last couple of years but at this point it doesn't make much sense for me to keep working on it with the limited time that I have.

I'd like to handle the transition as smoothly as possible so that it doesn't break for anyone. I've created a new go-epub organisation as my first thought is to transfer this project there. My understanding is that GitHub will automatically create a Git redirect so that it won't break for anyone using this library with the old Git URL.

It's due for a new major release anyway due to the new Go version requirements as part of #66, so I think this would be a good time to make the change.

@owulveryck You're the only other maintainer at the moment so I don't want to just dump this on you without consulting you first. Some options I see:

• Transfer the project to the new go-epub organisation as-is, leaving its future in your hands. I can stick around as a maintainer long enough to help transfer the project, update the README, etc.
• Transfer the project and add some more maintainers. @fmartingr has expressed some interest.
• Archive the project and allow it to evolve naturally. I guess whoever creates a fork and maintains it will become the de-facto maintainer, and if a particular fork stands out I can update the readme and point to it

What do you think? It's been 2 years since I created #28 so there's no urgency to make a decision right away, but I wanted to start the conversation.

After migrating to travis-ci.com (#29) , it seems that Travis CI no longer offers free builds for open-source projects: https://news.ycombinator.com/item?id=25338983

GitHub Actions seems the next logical choice...

This might help get us started: https://github.com/actions/starter-workflows/blob/main/ci/go.yml

And then maybe use this for goveralls? https://github.com/shogo82148/actions-goveralls

Hi!

This library is using extensively the panic function. I guess that, in most case, we should raise an error instead of panicing.

For example, I am using this library in a tool that fetches article on the web, apply a reading mode and create an epub.
Depending on the HTML structure, some stuffs are failing (bad src tags for images for examples, cf #45 ).
We would benefit from a proper error handling. The drawback is that it would change the API.

for example

func (e *Epub) SetCover(internalImagePath string, internalCSSPath string) {
...
}

could return an error if it cannot set the cover:

func (e *Epub) SetCover(internalImagePath string, internalCSSPath string) error {
...
}

I can take the action to post a PR for discussion if you want.

go.uuid has recently introduced breaking changes in their API:

$ go get github.com/bmaupin/go-epub
# github.com/bmaupin/go-epub
../../../go/src/github.com/bmaupin/go-epub/epub.go:134:44: multiple-value uuid.NewV4() in single-value context

Possible solutions:

• Lock dependencies to a specific tag/commit of go.uuid (e.g. 0633591)
  • I believe this will require checking in the vendor folder so that it will still work for users who are using go get instead of golang/dep
• Update to the new go.uuid API (missdeer/go-epub@ed07f4e)
  • There's lots of pushback regarding the new changes, probably best to wait
• Switch to another UUID library (e.g. google/uuid)
  • google/uuid might be a more official UUID library (breaking changes handled better), but doesn't yet have a stable API

Right now Travis only runs the Go tests via go test which is great but may not catch if the resulting EPUB is invalid.

The tests are already configured to run EPUBCheck if it's present. Can we configure the CI/CD to run EPUBCheck? This would make it much easier for contributors to submit PRs and see right away if there are EPUB validation issues.

As per #21, an empty title is invalid.

However, that's currently used as the mechanism to add a section to an EPUB without adding it to the table of contents:

The title is optional; if no title is provided, the section will not be added to the table of contents.

https://pkg.go.dev/github.com/bmaupin/go-epub?utm_source=godoc#Epub.AddSection

Two problems need to be solved:

1. What should the appropriate behaviour be when an empty title is added?

   This seems to be related to the philosophy of how this library should handle errors... Should we

   1. Throw an error if an empty title is added?
   2. Generate a title if an empty title is added?
   3. Keep the current behaviour but make a note in the documentation that an empty title is invalid?

2. How should we handle the ability to add a section to the EPUB without adding it to the table of contents? Some possibilities:

   • Keep the current behaviour (conflicts with #17)
   • Remove this functionality
   • Add this functionality in a different way

Coveralls doesn't seem to have any builds since switching from Travis CI (#32)

This is a thread to open the discussion about the need of a mutex in the main structure.

Wouldn't it be easier to declare the methods unsafe for concurrent use?

Hi @bmaupin

I got a issue report gonejack/html-to-epub#1 about section file naming.

Would you like to resolve it or I would like to make a PR later.

One of the tests failed, likely because it was running in Windows (which is a bit slower) and the got timestamp in the <package> didn't match the timestamp in Expected:

=== RUN   TestEpubWrite
    epub_test.go:137: Package file contents don't match
        Got: <?xml version="1.0" encoding="UTF-8"?>
        <package xmlns="http://www.idpf.org/2007/opf" unique-identifier="pub-id" version="3.0">
          <metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
            <dc:identifier id="pub-id">urn:uuid:fb92f2ca-2778-4577-8016-8fb5bf177da8</dc:identifier>
            <dc:title>My title</dc:title>
            <dc:language>en</dc:language>
            <meta property="dcterms:modified">2021-06-15T17:34:48Z</meta>
          </metadata>
          <manifest>
            <item id="nav" href="nav.xhtml" media-type="application/xhtml+xml" properties="nav"></item>
            <item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"></item>
          </manifest>
          <spine toc="ncx"></spine>
        </package>
        
        Expected: <?xml version="1.0" encoding="UTF-8"?>
        <package xmlns="http://www.idpf.org/2007/opf" unique-identifier="pub-id" version="3.0">
          <metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
            <dc:identifier id="pub-id">urn:uuid:fb92f2ca-2778-4577-8016-8fb5bf177da8</dc:identifier>
            <dc:title>My title</dc:title>
            <dc:language>en</dc:language>
            <meta property="dcterms:modified">2021-06-15T17:34:49Z</meta>
          </metadata>
          <manifest>
            <item id="nav" href="nav.xhtml" media-type="application/xhtml+xml" properties="nav"></item>
            <item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"></item>
          </manifest>
          <spine toc="ncx"></spine>
        </package>

https://github.com/bmaupin/go-epub/pull/38/checks?check_run_id=2832041267

Right now if doCleanup = false in epub_test.go, the files in /tmp aren't deleted. It seems like it would be desirable to always clean those up and instead leave just the test epub in the project directory.

I add an image with AddImage(filename, destpath), but then the image name starts with an digit, I get an error from epubcheck.

For example when I add 07-savepages.png, the resulting line in package.opf in the manifest section is:

<item id="07-savepages.png" href="images/07-savepages.png" media-type="image/png"></item>

The 07-savepages.png is not a valid XML id.

@andypillip @gonejack @pgundlach @ystyle @AmandaCameron @1l0 @propan @missdeer @onelio @lucasew

First of all, my apologies for the spam. Secondly, I'm not abandoning this project :)

I created this project because I wanted to learn Go and at the time there weren't any decent EPUB libraries for it so it seemed like something fun to do. I had big plans to use Go, but in spite of all the amazing things about Go I didn't find it very fun to use, and moved on to other languages.

I don't actually use this library any more and I didn't expect it to get so popular, but it's gotten over 100 stars and I thought it might be good to actually have the people using it be more involved, e.g.:

• Helping review and merge pull requests
  • Mostly I've been making sure that the API stays as stable as possible (or the version number is bumped as needed) and that new changes are covered by tests as best as reasonably possible
• Making decisions about the direction of the project
  • I've tried to make sure that at the very least this library generates valid EPUBs. Beyond that, is it okay if it allows users to generate EPUBs that aren't valid? Or should every effort possible be made to make sure that doesn't happen? (for example, see #24)
  • At some point this project should probably hit v1, so it would be nice to maybe figure some of this out beforehand

I'm not completely sure how this works, but I think there are a few options:

• I can add collaborators directly to this repository
• I can create an organization just for this repository and transfer it there
• I can transfer this repository to an existing organization
  (I did reach out to gofrs a while back about this last option but never heard anything: gofrs/help-requests#40)

At any rate, anyone who is interested feel free to follow up here. I am going to try to make an effort in the meantime to go through the issue/PR backlog and see if I can make some progress there 🤞

Thanks!

if you have file name with special character in name like 14-05-1402-11-22-52-ق-ظ.jpg go-epub generate invalid id if that file in opf file while use AddImage

The documentation needs to be updated for #45 before we can make a new release, e.g. this only mentions URLs and local files: https://pkg.go.dev/github.com/bmaupin/go-epub?utm_source=godoc#Epub.AddCSS

Not sure if that's acceptible in the epub spec, but I'm working on a program to convert some various web sources into .epubs for downlod, and one of the sources I'm using has a .bmp on it which is causing the system to fail.

Another thought might be, when a url is passed to AddImage it could use the returned Content-Type instead of trying to divine it from the file name, at least for images.

Edit: #5 and then #6 should be resolved first as both will affect this.

The vendor directory was added to the repository to work around issues with go.uuid (see #2). This shouldn't cause any problems, but once golang/dep is officially absorbed into the go toolchain (expected around Go v1.10), the vendor directory should no longer be necessary.

• Investigate whether uuid dependency is even needed
• If it is needed, investigate alternative libraries (satori/go.uuid appears to be unmaintained)
  • gofrs/uuid
    • Fork of satori/go.uuid
  • google/uuid
    • It's currently marked as unstable but stability should come with v1

I include an otf font into an epub. Running epubcheck gives me a warning that the epub references the font by application/x-font-otf. I believe it should be application/font-sfnt (see again https://www.w3.org/publishing/epub3/epub-spec.html#sec-cmt-supported)

I get the same warning with ttf.

BTW: thanks for the great project and the quick fix yesterday!

how to add subSection ?

Now that versioned Go modules (formerly called vgo) have been accepted as the package versioning tool for go, once it's been officially added (anticipated for Go 1.12) it would make sense to switch to it (pending resolution of #5).

Technically we always test with the latest version (1.x), so we'll need to add 1.16.

Coming from lapwat/papeer#12
Is it possible to just log a warning instead of returning an error?
In my particular case, there seems to be a broken img link that is halting the whole process, I really don't care if a single image is not downloaded so I would still like for the process to continue.

No thumbnail for the original one.

By dragging origin.epub into Apple Books then drag it out, the export.epub, it got thumbnail.

epubs.zip

The last line should use path instead of filepath.

Because filepath returning ..\images\.... on Windows.

Which is invalid for html img src.

https://github.com/bmaupin/go-epub/blob/master/epub.go#L439-L443

func addMedia(source string, internalFilename string, mediaFileFormat string, mediaFolderName string, mediaMap map[string]string) (string, error) {
	err := validateFileSource(source)
	if err != nil {
		return "", &FileRetrievalError{
			Source: source,
			Err:    err,
		}
	}

	if internalFilename == "" {
		// If a filename isn't provided, use the filename from the source
		internalFilename = filepath.Base(source)
		// If that's already used, try to generate a unique filename
		if _, ok := mediaMap[internalFilename]; ok {
			internalFilename = fmt.Sprintf(
				mediaFileFormat,
				len(mediaMap)+1,
				strings.ToLower(filepath.Ext(source)),
			)
		}
	}

	if _, ok := mediaMap[internalFilename]; ok {
		return "", &FilenameAlreadyUsedError{Filename: internalFilename}
	}

	mediaMap[internalFilename] = source

	return filepath.Join(
		"..",
		mediaFolderName,
		internalFilename,
	), nil
}

Golang tags must be vMAJOR.MINOR.PATCH always.

The tag v0.7 is invalid, and it can't be importated into another project, it must have the patch number on it like this v0.7.0.

For further information please take a look into https://blog.golang.org/publishing-go-modules

On the writeEpub function of write.go, it seems that calling filepath.Join(rootEpubDir, mimetypeFilename) on a Windows machine would generate a path joined by a backslash \ instead of usual forward slash /.

Calling fs.WalkDir(filesystem, rootEpubDir, addFileToZip) later would result in comparing forward slash path of mimetype with the backslash version by filepath.Join(rootEpubDir, mimetypeFilename) which in the end resulting with another mimetype with deflate method.

Changing path == filepath.Join(rootEpubDir, mimetypeFilename) to filepath.Base(path) == mimetypeFilename would be a quick fix but there might be a better way to do this.

I am trying to compile my tool into webassembly.

The problem is that this library is using temporary files on the fs, and this is forbidden by webassembly.

I could be nice to add an option to deal with the media in a pseudo memory fs.

This would lead to a new write method that could take an io.Writer as a parameter (and eventually deprecating the WriteFile method).

I can make a POC when I have time if you are interested (and you can assign this issue to me)

This might make it easier to identify issues like #19

When I try to add a woff font, I get the following error:

panic: Unmatched file extension, media type not set for file: roboto-mono-v5-latin-300.woff

Is it possible to add support these fonts?

See https://www.w3.org/publishing/epub3/epub-spec.html#sec-cmt-supported

https://docs.travis-ci.com/user/migrate/open-source-repository-migration

It seems the shutdown date is currently 2021-06-15

On line 444 of write.go,
e.toc.addSubSection(relativePath, index, section.xhtml.Title(), relativeSubPath)
should be
e.toc.addSubSection(relativePath, index, child.xhtml.Title(), relativeSubPath)
or else the title of all child sections displayed on TOC would be the same with their parent

https://github.com/bmaupin/go-epub/blob/main/write.go#L444