Code Monkey home page Code Monkey logo

django-manifest-loader's Introduction

Django Manifest Loader

Build Status Build Status contributions welcome

Reads a manifest file to import your assets into a Django template. Find the URL for a single asset or the URLs for multiple assets by using pattern matching against the file names. Path resolution handled using Django's built-in staticfiles app. Minimal configuraton, cache-busting, split chunks.

About

Turns this

{% load manifest %}
<script src="{% manifest 'main.js' %}"></script>

Into this

<script src="/static/main.8f7705adfa281590b8dd.js"></script>

Quick reference:

Manifest tag

{% load manifest %}

<script src="{% manifest 'main.js' %}"></script>

turns into

<script src="/static/main.8f7705adfa281590b8dd.js"></script>

Manifest match tag

{% load manifest %}

{% manifest_match '*.js' '<script src="{match}"></script>' %}

turns into

<script src="/static/vendors~main.3ad032adfa281590f2a21.js"></script>
<script src="/static/main.8f7705adfa281590b8dd.js"></script>

License

Django Manifest Loader is distributed under the 3-clause BSD license. This is an open source license granting broad permissions to modify and redistribute the software.

django-manifest-loader's People

Contributors

andersk avatar christopherlandaverde avatar modbender avatar rykener avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

django-manifest-loader's Issues

Better documentation needed for using with split chunks

There are two ways to use this, as a straight replacement for django's static templatetag, or as something a bit more involved that uses pattern matching to bring in all needed assets for a webpack instance making split chunks. Documentation about using it with split chunks needs to be created.

Production documentation?

What would be the best practices for deploying this to production or would the same setup in installation work seamlessly across environments?

Import all dependencies from single webpack bundle/entry automatically

I have webpack-bundle-tracker and django-webpack-loader in a working state after manually fixing them, but as part of my CI/CD it is not viable to do this.

A thing that I noticed this project does not do is link which npm packages are used in which bundles when you follow the tutorial which got me into this in the first place (which you also link in your readme) which makes all the npm packages into different .js files to save bandwidth with caching.

Here's an example of the output of [email protected] (note it does not do this in 0.4.3):

...
"chunks": {
        "globals": [
            "runtime.7389d0ed4f1baaf09d7f.js",
            "globals.cfeb587d501142b196c5.js"
        ],
        "board": [
            "npm.underscore.c7a03d459b7d101c847b.js",
            "npm.regexp.prototype.flags.2623abc18f6e36f1fbd2.js",
            "npm.object-is.38cd874c904b51db50c7.js",
            "npm.object-keys.dfaa60aa949e1df12b1c.js",
            "npm.has-symbols.05e790e10906ba453058.js",
            "npm.function-bind.2d6787cc6f5f48b3b50e.js",
            "npm.es-abstract.6c4cb5b8e3a358d794f8.js",
            "npm.is-regex.4a1384cbccb2368b5c84.js",
            "npm.is-date-object.39cb2bf1968347761930.js",
            "npm.is-arguments.ff60d7ce5200e7f3abb1.js",
            "npm.invert-color.1ed95cd7bb596a2a2a68.js",
            "npm.has.4bac64353b104fe339c3.js",
            "npm.fast-diff.809067b7dc8fbef37c87.js",
            "npm.extend.cf12587c8e830c228315.js",
            "npm.define-properties.a59ce35a4273a902a5e2.js",
            "npm.deep-equal.35ddf946ff5693b1dd89.js",
            "board.5d76531bfc5c8812e276.js"
        ],
        "drive": [
            "npm.quill.0f4eed0169b981e5c8e3.js",
            "npm.quill-cursors.c297e81a0f33e5574f5d.js",
            "npm.quill-delta.ca815e1b8e309922cae6.js",
            "drive.b3ce3a01d399c22a27e2.js"
        ]
    },
...

With the output above, I was able to do {% render_bundle 'board' %} and it would import all the files as seen above under the 'board' chunk.

I believe this is a mandatory feature, as I do not want to be keeping track and/or guessing which npm package I use in which bundle, thus forgetting to add or remove a script tag in my HTML.

I also do not know the significance of the runtime.js file webpack outputs when you have multiple entry files and runtimeChunk: 'single', in your webpack config, but I assume it should be imported when using any of the bundles?

You could even go further as to support webpack's dependOn: 'x', flag, and have the bundles that depend on other bundles automatically imported also, which is something the older project does not do.

With all that in mind, it should probably prevent importing the same file twice in one document?

Thanks for creating and maintaining this project. Please let me know if this feature is possible for you to implement.

Cannot interpret URL paths in manifest

I'm trying to run a Vue app in dev mode (i.e. vue serve from the CLI. My manifest correctly records URLs to the dev server, e.g.

{
  "chunk-vendors.js": "http://localhost:8080/chunk-vendors.js",
  "vue_app_01.js": "http://localhost:8080/vue_app_01.js",
  "vue_app_02.js": "http://localhost:8080/vue_app_02.js",
  "favicon.ico": "http://localhost:8080/favicon.ico",
  "img/logo.png": "http://localhost:8080/img/logo.png",
  "index.html": "http://localhost:8080/index.html"
}

However, it seems that django-manifest-loader is presuming all resources in the manifest are paths to Django staticfiles. My template renders:

<!-- manifest -->
 <script src="/static/http%3A/localhost%3A8080/vue_app_01.js"></script>  

<!-- manifest_match -->
<script src="/static/http%3A/localhost%3A8080/vue_app_01.js"/>

I'd like to be able to use django-manifest-loader to easily switch between dev mode and static build mode on my Vue app without altering any code. Is this possible?

Manifest output grouped by entry

WebpackManifestPlugin allows using a custom generate function, which could be used to generate a manifest with the following structure:

{ "entryName": [ "runtime.js", "vendor.js", "entry.js"] }

With this structure it would be possible to get all files required to render a given entry and maybe it could also be used to determine the order by which the files should be imported (related to #3).

Possibly related to #14?

Unit tesets needed

This library has no tests associated with it. This should change asap.

Issue with webpack-stats

First, great project!

For some reason, the manifest is not picking correctly the URL from my webpack-stats:

{
    "status": "done",
    "publicPath": "http://localhost:8080/",
    "chunks": {
        "page1": [
            {
                "name": "js/page1.js",
                "publicPath": "http://localhost:8080/js/page1.js",
                "path": "/Users/xxx/frontend/dist/js/page1.js"
            }
        ],
        "page2": [
            {
                "name": "js/page2.js",
                "publicPath": "http://localhost:8080/js/page2.js",
                "path": "/Users/xxx/frontend/dist/js/page2.js"
            }
        ]
    }
}```

Its generating a `/static/page1.js`

I tried with `{%manifest 'page1.js'%}` and `{%manifest 'page1'%}` , both generate the same.

Support for entrypoints with splitchunks

I have a webpack config that uses splitChunks to generate a number of files:

base-43214231.js
vendors~base~b5271302-c4dfeb1c1ad832b8000d.bundle.js

How do I ensure all files required are rendered for the correct entrypoint?

Failing to load entrypoint with DEBUG=True

Running with DEBUG=True, in the browser console, I have the following:

Loading failed for the <script> with source “http://devel-c7-tmp:8000/static/automain.3aae4e640403c1e57326.js”.

In the dist directory, there is a main.3aae4e640403c1e57326.js file. Perhpaps, that URL is exactly what it should be be? It is obviously not resolving to the ./dist/main.3aae4e640403c1e57326.js file as it should though.

In the template, I have <script src="{% manifest 'main.js' %}"></script>.

Any idea what is going on here?

Thanks

Can't assign asset paths to a variable in templates

in django's static template tag you can do

{% static 'main.js' as foobar %}

Even though the manifest tag is just a wrapper around the static tag, the modification that takes place will strip out any arguments after the name of the asset. The manifest tag should be modified to allow the as foobar to pass through

Path of dynamic chunks is relative

The chunks resulting from a dynamic import are incorrectly loaded into the page. Their path is relative to the page thus
causing 404 errors. To solve this I try to set the publicPath in Webpack config as suggested in webpack/webpack#7968

    output: {
        path: path.resolve(__dirname, 'dist'),
        publicPath: "/",
        filename: "[name].[contenthash].js",
    },

Unfortunately that raises a Django SuspiciousFileOperation:

The joined path (/main.9ce82b7ec94c17ac77f9.css) is located outside of the base path component

Jinja2 extension returns string quotes around output

For example:
{{ '*baseTenant*.js'|manifest_match('<script src="{match}"></script>') }}

results in:

"<script src="/static/baseAddMembership~baseTenant~f585d785-baebc8a04938f4824d09.bundle.js"></script>"

Which means the script is displayed as text rather than as a script element.

document how to use with webpack-dev-server

For debugging it is better to have webpack-dev-server active. Sadly it is a bit tricky to configure:
webpack.config.js

module.exports =  (env, options) =>({
 ...
  devServer: {
                              port: '8080'
 },
plugins: {

        new CleanWebpackPlugin({
            verbose: true,
//important elsewise minifest.json is deleted
            cleanOnceBeforeBuildPatterns: ['**/*', '!manifest.json'],
        }),
        new WebpackManifestPlugin({
            writeToFileEmit: true,
            publicPath: env['WEBPACK_SERVE'] ? 'http://localhost:8080/' : null,
        }),
})

it would be nice to include this trick in the documentation

My project uses this setup:
https://github.com/devkral/secretgraph

How to determine order of assets for splitChunks?

When webpack splits source files into chunks there is likely a preferred order to which those files should be imported into html with. In my testing thus far it doesn't seem to matter but I don't trust that. This needs further research to determine if there is a preferred order and if so, how to determine it from the manifest file.

WEBPACK_SETTINGS wants webpack's output directory, it should be able to infer it

# settings.py

STATICFILES_DIRS = [
    BASE_DIR / 'dist'
]

WEBPACK_SETTINGS = {
    'output_dir': BASE_DIR / 'dist'
}

It was originally setup this way to not assume which directory within STATICFILES_DIRS the manifest file is in. However we can perform some basic logic to find it. if len(STATICFILES_DIRS) == 1 then we know it'll be in STATICFILES_DIRS[0]. If it's longer than 1 then we can just iterate through and check.

Should still allow the user to set the output dir in order to not have to do all that checking.

Manifest has leading slash, but staticfiles expects none

Trying to use this in non-debug mode fails:

self = <django.contrib.staticfiles.storage.ManifestStaticFilesStorage object at 0x7fe1adb9b670>, name = '/css/app.f6e28323.css'

    def stored_name(self, name):
        parsed_name = urlsplit(unquote(name))
        clean_name = parsed_name.path.strip()
        hash_key = self.hash_key(clean_name)
        cache_name = self.hashed_files.get(hash_key)
        if cache_name is None:
            if self.manifest_strict:
>               raise ValueError(
                    "Missing staticfiles manifest entry for '%s'" % clean_name
                )
E               ValueError: Missing staticfiles manifest entry for '/css/app.f6e28323.css'

/usr/local/lib/python3.10/site-packages/django/contrib/staticfiles/storage.py:465: ValueError

From what I can tell, the issue is that django-manifest-loader attempts to find files in the staticfiles stoage using the path found in manifest.json. The issue is that paths in manifest.json have a leading slash, but paths in staticfiles.json do not.

E.g.:

$ jq < manifest.json| grep css.app
  "app.css": "/css/app.f6e28323.css",
$ jq < staticfiles.json |  grep f6e28323
    "css/app.f6e28323.css": "css/app.f6e28323.d95a1d9dad84.css",

[Question] Is there a way to include Django Admin assets?

Hi,

Thanks for this project, it's been a great help to me! I was wondering if there was an easy way to include the default admin assets in the Webpack build?

Currently I have custom CSS & JS being bundled through Webpack and rendering correctly with the help of the manifest loader but I am having a hard time figuring out how to include the admin files. Any help would be appreciated.

Thanks in advance!

Having trouble getting the loader to work with webpack assets produced by Vue.js

This topic may possibly be a duplicate of #41 but I didn't want to hijack the thread.
I am experiencing difficulties in trying to get the loader to work with webpack assets produced by Vue.js (@vue/cli 4.5.12).

My vue.config.js file is as follows:

const {CleanWebpackPlugin} = require('clean-webpack-plugin');
const {WebpackManifestPlugin} = require('webpack-manifest-plugin');

module.exports = {
    configureWebpack: {
        plugins: [
            new CleanWebpackPlugin(),
            new WebpackManifestPlugin({ publicPath: "" })
        ]
    },
    productionSourceMap: false,
    transpileDependencies: [
        'vuetify'
    ]
};

The generated manifest.json is as follows:

{
  "app.css": "css/app.7a4272b1.css",
  "app.js": "js/app.1ddd4c01.js",
  "chunk-vendors.css": "css/chunk-vendors.24611a9a.css",
  "chunk-vendors.js": "js/chunk-vendors.33990123.js",
  "favicon.ico": "favicon.ico",
  "index.html": "index.html"
}

The django template correctly loads both of the javascript files, but the styling is completely lost and the UI is garbled.

If I add the following to vue.config.js:

    css: { extract: false },

The css files are no longer generated as separate files.
The django template loads the javascript files and most of the styling is applied except for the fonts.
The mdi icons used in the UI are also completely lost.

I have checked to see what the assets in the dist directory produce using serve -s dist which results in the correct styling for the fonts.
The mdi icons are also present in the UI.

I would really appreciate any insight into what may be causing this or any pointers to what I could try to determine the problem.

Improve Documentation

Documentation is currently subpar, and would benefit from attention. Goals are to write full docs with Sphinx to then host on Read the Docs. Anyone who wants to help is welcome to do so.

django-manifest-loader and whitenoise

Hello developers;

I already have django-manifest-loader working on my project, but today I started to set up Whitenoise; And I am wondering if there is any wierd interaction between the two packages?

My current flow is:

  • Run Webpack with SplitChunks, compression and optimization into Django's /static/ folder;
  • Run Django collectstatic to send the contents from /static/ to STATIC_ROOT.

Set STATICFILES_STORAGE = "django.contrib.staticfiles.storage.ManifestStaticFilesStorage" (default) such that neither Django nor Whitenoise perform any sort of work on my static assets (Webpack does that for me);

Now, my problem arises from here onwards.

  • From what I got from Whitenoise docs, it expects {% static ... %} to be used.

    • Do you know if using {% manifest_match ... %} ruins Whitenoise's operation? Or there is no correlation at all.
  • Django-manifest-loader docs tell us we can set the options below:

      1. Should I set output_dir to be the path of STATIC_ROOT which happens to be an absolute path in my system?
      1. Should I set manifest_file to staticfiles.json, since it is also the approach used by Whitenoise?
MANIFEST_LOADER = {
    'output_dir': None,
    'manifest_file': 'manifest.json',
    'cache': env("SERVICE_ENV", default="dev") == "prd",
    'loader': DefaultLoader
}

Can't use with create react app etc

currently Django Manifest Loader can only consume the default manifest file as generated by WebpackManifestPlugin. Instead, it should have a series of different loaders it can use, so that a) loaders can be developed for other default types of manifest files, like create react apps manifest and b) so that users can write their own manifest loaders if the provided loaders don't work for them.

# settings.py

from manifest_loader.loaders import CREATE_REACT_APP

MANIFEST_LOADER = { 
    'loader': CREATE_REACT_APP,
}

Manifest tag doesn't accept variable argument

Trying to use a variable filename:

{% with page|add:".css" as file %}
    <script src="{% manifest file %}" />
{% endwith %}

Results in TemplateSyntaxError: 'manifest' tag's argument should be in quotes.

Hot reloading or automatic browser reload?

Thanks so much for this, I was using django-webpack-loader but this is much simpler!

Do you have a recommendation for how to achieve automatic browser reloading when the assets are updated? I know this can be achieved using browsersync or livereload, but I'm hoping to not have to run a third process (on top of django server and webpack watch) during development.

Thanks again!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.