maddyblue / esc Goto Github PK

I'm using the prefix flag and I noticed that when I run this

esc -o="static.go" -pkg="nictl" -prefix="docs/public" docs/public

on the command line the prefix is stripped but if I use go generate:

//go:generate esc -o="static.go" -pkg="nictl" -prefix="docs/public" docs/public

it isn't. The rest is working absolutely fine...

$ esc -o static.go static/

Generates:

var data = map[string]*file{
    "/static/mart.js": {compressed: "....", size: 519, local: "static/static/mart.js"},
}

Which is wrong as I do not have static/static/mart.js, and hence it fails.

static.go:14:6: don't use underscores in Go names; type _esc_localFS should be _escLocalFS
static.go:16:5: don't use underscores in Go names; var _esc_local should be _escLocal
static.go:18:6: don't use underscores in Go names; type _esc_staticFS should be _escStaticFS
static.go:20:5: don't use underscores in Go names; var _esc_static should be _escStatic
static.go:22:6: don't use underscores in Go names; type _esc_file should be _escFile
static.go:166:5: don't use underscores in Go names; var _esc_data should be _escData

Let me preface my feature request by first stating that I really have enjoyed using esc. It is simple, uses the Go toolchain (generate), and makes me happy.

I have run across a use case where a possible feature enhancement could make this tool a little easier to use for specific use cases. Below is a sample of some code that demonstrates a scenario where I have implemented a renderer for use in the Echo framework. Simply put it provides a Render method that tells the framework how to render requests.

In the LoadTemplates method is where the imaginary code comes in. In here I imagine the ability to do something similar to ioutil.ReadDir() and get a slice of FileInfo structs. This way I can then iterate over the files and compile them as templates.

package ui

import (
	"fmt"
	"html/template"
	"io"
	"path/filepath"
	"strings"

	"myproject/app/compiledassets"
	"github.com/labstack/echo"
)

var templates map[string]*template.Template

/*
TemplateRenderer describes a handlers for rendering layouts/pages
*/
type TemplateRenderer struct {
	templates *template.Template
}

/*
NewTemplateRenderer creates a new struct
*/
func NewTemplateRenderer(debugMode bool) *TemplateRenderer {
	result := &TemplateRenderer{}
	result.LoadTemplates(debugMode)

	return result
}

/*
Implements render interface expected by Echo 
*/
func (t *TemplateRenderer) Render(w io.Writer, name string, data interface{}, ctx echo.Context) error {
	var tmpl *template.Template
	var ok bool

	if tmpl, ok = templates[name]; !ok {
		return fmt.Errorf("Cannot find template %s", name)
	}

	return tmpl.ExecuteTemplate(w, "layout", data)
}

/*
Traverses the asset pages directory and compiles templates into a package level variable
*/
func (t *TemplateRenderer) LoadTemplates(debugMode bool) {
	templates = make(map[string]*template.Template)

	/*
	 * This code isn't real. It is what I imagine esc could look like
	 * to get a directory of assets. Perhaps returns a slice of 
	 * os.FileInfo structs
	 */
	files := compiledassets.DirContents("/www")

	for _, file := range files {
		if !file.IsDir() {
			basename := file.Name()
			trimmedName := strings.TrimSuffix(basename, filepath.Ext(basename))

			templates["mainLayout:"+trimmedName], _ = template.Must(
				template.New("layout").Parse(www.FSMustString(debugMode, "/www/myproject/layouts/mainLayout.gohtml")),
			).Parse(www.FSMustString(debugMode, "/www/myproject/pages/"+basename))
		}
	}
}

What are your thoughts on this? Thanks for reading!

Would you accept a PR to fix all golint errors and warnings?

With prefix specified, make sure that if both prefix/file.name and file.name are present, an error is produced.

I cannot use FS() directly since I need to do some templating on the index.html. This is what I'm doing:

// static
router.HandleFunc("/", MkIndexHandler(mc))
http.Handle("/js/", http.FileServer(_escDir(false, "/js")))

the index handler works fine:

func MkIndexHandler(mc *MeasurementCache) func(http.ResponseWriter, *http.Request) {
	mainTemplate, err := _escFSString(false, "/index.html")

However, any request to /js ends in http 404.

assets has the respective files and I'm using the head version with Readdir support:

var _escData = map[string]*_escFile{

    "/js/app.js": {
        name:    "app.js",
...

var _escDirs = map[string][]os.FileInfo{

	"assets": {
		_escData["/index.html"],
		_escData["/js"],
	},

	"assets/js": {
		_escData["/js/app.js"], ...
	},

I don't even see a call to _escDirectory.Open(). Would appreciate a hint if this is a supported scenario and what might be a potential mistake.

First of all, thanks for creating this! It works well and the API is clean and simple.

I used it to bundle wasm and js into an example app so that a server and all content is bundled into the app. Some of my content is generated dynamically, so delegating everything to

http.FileServer(FS(false))

didn't seem workable. I ended up writing handler functions like the following to allow setting proper content headers. Seems to work very well, though I don't know if that's the cleanest possible way to code it.

// fsSendStatic serves the requested file from the ESC FS.
func fsSendStatic(w http.ResponseWriter, r *http.Request) {
	_, err := w.Write(FSMustByte(false, r.URL.Path))
	if err != nil {
		log.Fatalf("Writing file %s returned error: %v", r.URL.Path, err)
	}
}

// fsSendStaticJS serves the requested javascript file from the ESC FS.
func fsSendStaticJS(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/javascript")
	fsSendStatic(w, r)
}

// fsSendStaticWasm serves the requested WebAssembly file  from the ESC FS.
func fsSendStaticWasm(w http.ResponseWriter, r *http.Request) {
	w.Header().Set("Content-Type", "application/wasm")
	fsSendStatic(w, r)
}

It took a bit of trial and error to get it working. Just wondering if it might save others some time if the README included an example showing how to serve a mixture of static and dynamic content.

$ esc -o static.go index.html style.css

http.Handle("/", http.FileServer(FS(Debug)))

I can access /style.css but not /index.html.

[amitu@AmitUs-MacBook-Air martd (master ✗)]$ curl -I localhost:54321/index.html
HTTP/1.1 301 Moved Permanently
Location: ./
Date: Mon, 24 Nov 2014 08:29:07 GMT
Content-Type: text/plain; charset=utf-8

[amitu@AmitUs-MacBook-Air martd (master ✗)]$ curl -I localhost:54321/
HTTP/1.1 404 Not Found
Content-Type: text/plain; charset=utf-8
Date: Mon, 24 Nov 2014 08:29:15 GMT
Content-Length: 19

[amitu@AmitUs-MacBook-Air martd (master ✗)]$ curl -I localhost:54321/style.css
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 1378
Content-Type: text/css; charset=utf-8
Date: Mon, 24 Nov 2014 08:31:26 GMT

[amitu@AmitUs-MacBook-Air martd (master ✗)]$

Hi,

Thank you so much for this project.

Many of us felt much pain when go-bindata unexpectedly went down and hundreds of projects stopped building. This is to verify that there are no plans for esc to repeat the same trick.

Would be great if there was FSGzippedByte method which would return the compressed representation of the data, afterall pretty much every browser sends Accept-Encoding: gzip.

Golang has made changes to how the gzip compression scheme works in Golang 1.8. So if you are embedding files into your program and you would like to support users who have earlier versions of Golang, you will need to manage inconsistent compression errors.

It would be nice to toggle compression on and off.

I've put together a small update with some additional documentation to the README in a branch found in my fork of this project. I would love some feedback before submitting a pull request. It can be found at https://github.com/adampresley/esc/tree/advanced-readme-example.

In some cases it is desirable to add files to the local fs and test without a need to rebuild an executable. Previously _ascLocalFS.Open() implementation needed the path to exist in the generated map of files in the fs or would fail. This change removes the dependency for the path to already exist in static FS generation.

Eventually It would be nice to change FS to take a string argument that would direct which file system path to use instead of defaulting to to predefined local path. so after deployment of binary an argument could be provided to allow user to specify a path to use for content.

Please See PR I have raised to begin discussion of this change.

The ModTime feature added in #13 causes our CI builds to fail due to the timestamps changing and causing our checks to think that the static.go and Tsc generated bosun.js file are out of date. I don't believe git syncs the last modified date, so anyone using multiple systems or having multiple contributors is likely to have the same issue.

Initially after we updated to the new version of esc (see bosun-monitor/bosun#1307 ) I though it was just an issue with TypeScript, but after ensuring it only updates the .js file when the content changes (see bosun-monitor/bosun#1308 ) we are still seeing issues with our CI builds.

I don't think we need the ModTime feature, so having a flag to disable it would meet our needs. @captncraig or I can probably create a PR to disable ModTime (or enable it if you want it disabled by default)

Hey,

This is the lowest of priorities I know... I have recently started using a library scanner that detects vulnerabilities in libraries that I use. It seems to have caught on to the usage of the embedded jquery in example/static.go

Would it be possible to update the example to something else?

Useful when multiple versions of static files are uses - dev, production.

// +build dev

I recently upgraded to the newest version of this module and I noticed that esc.Run() now requires an io.Writer. That's awesome, however old code requires a lot of template (see StackExchange/dnscontrol#515 for an example).

Would it make sense to have (for example) a nil writer result in the file being created automatically? This would help with migrating old code, and also would reduce the amount of template code needed.

Just a though!

The Open() function for the localFS (ie dev mode where original files are exposed) first checks that the file exists in the last esc-generated file list, and errors if it's not there. More serious refactoring & renaming of files would be permitted if this was simply a pass through to Open(). You wouldn't want this in production for security reasons though - perhaps it a new mode, DANGER_MODE.

I've been toying with having my CI pipeline re-generate the embedded front-end as a way to check I've remembered to check in any changes:

go get github.com/mjibson/esc
go generate ./frontend/...
git diff --exit-code ./frontend # --exit-code => exits w. 1 on changes, 0 otherwise

But I am getting lots of small changes like this

 	".": {
-		_escData["/icon.png"],
-		_escData["/index.html"],
 		_escData["/icon.svg"],
+		_escData["/index.html"],
+		_escData["/icon.png"],
 	},
 }

Yes, it is a map -- and the spec says they are un-ordered, and I guess that's kind of hard to change. But it would be nice if one could test this regularly, to make sure nothing is forgotten.

It would be useful to have the hashes of each files pre-computed and accessible with according methods so that we can send strong HTTP Etag headers for caching.

I am invoking esc like:

esc -o static.go -modtime=1483228800 -ignore=".*server$" static

I have a folder, which contains files I want to ignore (in the above example a binary called static/server). However, the regex passed into the -ignore flag only checks whether the foldername static matches the regex. Is this intended? I was hoping all the files in the folder static would be matched against the regex

The current tag/release is quite old now. I suggest creating a new release as the current version is much better then the old one.

I hit this data race in my CI:

==================
WARNING: DATA RACE
Read at 0x00c0000d2a68 by goroutine 7:
  github.com/hortbot/hortbot/internal/db/migrations/esc.(*_escFile).Name()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/esc/esc.go:130 +0x3e
  github.com/golang-migrate/migrate/v4/source/httpfs.(*PartialDriver).Init()
      /home/travis/gopath/pkg/mod/github.com/golang-migrate/migrate/[email protected]/source/httpfs/partial_driver.go:48 +0x34c
  github.com/golang-migrate/migrate/v4/source/httpfs.New()
      /home/travis/gopath/pkg/mod/github.com/golang-migrate/migrate/[email protected]/source/httpfs/driver.go:21 +0xaf
  github.com/hortbot/hortbot/internal/db/migrations.newMigrate()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/migrations.go:51 +0x77
  github.com/hortbot/hortbot/internal/db/migrations.Up()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/migrations.go:15 +0x8d
  github.com/hortbot/hortbot/internal/db/migrations_test.withDatabase()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/migrations_test.go:67 +0x11e
  github.com/hortbot/hortbot/internal/db/migrations_test.TestUp()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/migrations_test.go:33 +0x52
  testing.tRunner()
      /home/travis/.gimme/versions/go/src/testing/testing.go:993 +0x1eb

Previous write at 0x00c0000d2a68 by goroutine 77:
  github.com/hortbot/hortbot/internal/db/migrations/esc._escStaticFS.prepare.func1()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/esc/esc.go:59 +0xad
  sync.(*Once).doSlow()
      /home/travis/.gimme/versions/go/src/sync/once.go:66 +0x103
  sync.(*Once).Do()
      /home/travis/.gimme/versions/go/src/sync/once.go:57 +0x68
  github.com/hortbot/hortbot/internal/db/migrations/esc._escStaticFS.prepare()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/esc/esc.go:58 +0x160
  github.com/hortbot/hortbot/internal/db/migrations/esc._escStaticFS.Open()
      /home/travis/gopath/src/github.com/hortbot/hortbot/internal/db/migrations/esc/esc.go:78 +0x46
  github.com/hortbot/hortbot/internal/db/migrations/esc.(*_escStaticFS).Open()
      <autogenerated>:1 +0x5f
  github.com/golang-migrate/migrate/v4/source/httpfs.(*PartialDriver).ReadUp()
      /home/travis/gopath/pkg/mod/github.com/golang-migrate/migrate/[email protected]/source/httpfs/partial_driver.go:111 +0x1b9
  github.com/golang-migrate/migrate/v4/source/httpfs.(*driver).ReadUp()
      <autogenerated>:1 +0x5b
  github.com/golang-migrate/migrate/v4.(*Migrate).newMigration()
      /home/travis/gopath/pkg/mod/github.com/golang-migrate/migrate/[email protected]/migrate.go:837 +0x96
  github.com/golang-migrate/migrate/v4.(*Migrate).readUp()
      /home/travis/gopath/pkg/mod/github.com/golang-migrate/migrate/[email protected]/migrate.go:612 +0x22b

It appears as though there's a path where the static file is being returned before it has been prepared, but I can't see how at first glance.

I'm using the /text/template package for html rendering. Now I want to use esc for including the templates files in my binary. I'm looking for a good example, but I can't find it. Is there a way to access a file from the generated sources as an asset which will be accepted by the template.-functions?

tmpl, parseErr := template.ParseFiles(baseFile, headerFile, footerFile)
if parseErr != nil {
    http.Error(w, parseErr.Error(), http.StatusInternalServerError)
    return
}

if templErr := tmpl.Execute(w, renderObject); templErr != nil {
    http.Error(w, templErr.Error(), http.StatusInternalServerError)
}

Awesome project that we use daily! One thing we do with go generate is to invoke esc as part of our code generation step to bundle the assets and whatnot into a few "static.go"esq files. It would be very useful to be able to query the version of esc to validate its existence before attempting to use it.

That will really help with dependency management.

Add a comment on top of the autogenerated files—something like

// This file was autogenerated by esc (https://github.com/mjibson/esc), do not edit.

I am trying to figure out how to handle passing the 'output' file to a cross compiler so I can cross compile my binary while using this library.

I am fine with working with either goxc or gox for cross compiling so if you have a solution for either of those cross compilers, I would be interested in how you do it.

Thanks...

golang/go#13560 (comment)

A generated file should have a comment matching this regex: ^// Code generated .* DO NOT EDIT.$.

Hi there,

I just discovered this project. I have a similar one called vfsgen, that also generates a .go file that implements http.FileSystem and only uses standard library.

In its README, I have a list of alternatives because I think it's a good idea, and I'd like to add your project there. Can you help me understand in what ways this project is unique, what does it focus on/optimize for, and what would be a good description?

Thanks!