Code Monkey home page Code Monkey logo

nuxt-payload-extractor's Introduction

Nuxt payload extractor

Nuxt.js module that makes nuxt generate command to store data payload in dist dir implementing full static generation. See it in action on my site: DreaMinder.pro

⚠️⚠️⚠️ This feature called "full static generated mode" has been released as a native Nuxt.js feature recently. You might want to try it out before using this module, it is much more powerful. ⚠️⚠️⚠️

Benefits

✓ If you use external API to fill your project with data, even after your site has been prerendered, you need to keep your API running to make client-side navigation possible. With this module your API data stores along with prerendered HTML, so the issue is solved

✓ Increases page download speed for crawlers

✓ Makes HTML source code look cleaner

✓ Decreases load on API server

✓ Makes prerendered and client rendered pages consistent in case API data changed after you deployed prerendered pages

✓ Decreases initial page download time x1.5-2 (which is actually doesn't affect perfomance)

Setup

  • Add nuxt-payload-extractor dependency
  • Define nuxt module:
{
  modules: [
   'nuxt-payload-extractor'
  ]
}
  • Add asyncData custom logic with $payloadURL helper
async asyncData({ $axios, $payloadURL, route }){
  //if generated and works as client navigation, fetch previously saved static JSON payload
  if(process.static && process.client && $payloadURL)
    return await $axios.$get($payloadURL(route))

  //your request logic
  let post = await $axios.$get(`/post.json`)
  return {
    post
  }
}
  • Run npm run generate

GraphQL usage

You'll need axios in your production bundle, your graphQL client is only invoked server-side, on 'generate' command.

async asyncData({ $axios, $payloadURL, route, app }) {
  if (process.static && process.client && $payloadURL) {
    return await $axios.$get($payloadURL(route))
  } else {
    let gqlData = await app.apolloProvider.defaultClient.query({
      query: gqlquery
    })
    return {
      gqlData
    }
  }
}

Options

You can blacklist specific paths using blacklist: [] options. They will be generated in native way. But you have to disable payload request inside of asyncData yourself. Check out example dir for details.

All payloads have timestamp applied to their names for better cache invalidation. You can turn them off by using versioning: false option. Keep in mind that timestamp changes on every generate run. Also, nuxt generate --no-build is not supported in this case.

Caveats

  • Are you using nested routes (with <nuxt-child />)? This case isn't compatible with nuxt-payload-extractor. It will only work if parent and child routes return different data-keys (ex. return { postsList } and return { singlePost }).
  • Are you filling your vuex store with page-specific data? It will break on client-side navigation. Use NuxtServerInit action for global vuex data or return your vuex data from asyncData to make it work.

How it works

  • Extracts <script>window.__NUXT__= ... </script> replacing it with <script src="payload.js">
  • Makes two files along with index.html of a prerendered page: payload.js for initial page load and payload.json for client-side navigation

nuxt-payload-extractor's People

Contributors

astagi avatar dreaminder avatar edimitchel avatar jani-kasa avatar weotch avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

nuxt-payload-extractor's Issues

idea: single payload.js to fetch unique payloads

Hey, I'm wondering if this is possible and if there would be any downsides?

Currently, there is a payload.js and payload.json generated for each page.
They both contain the same data but in different formats.

Would it be possible to have a global payload.js that simply grabs the payload.json file per page? That way the .json would work as usual and payload.js could be cached and only need to be downloaded once. It would cut down on the number of files generated as well.

Stops working with update to 1.1.0. Cannot find ./helpers

After update, this module no longer works and I get this error each time. The ./helpers file is in the node_modules/nuxt-payload-extractor/lib/... but it does not get extracted by nuxt and I keep getting this error.

ERROR in ./.nuxt/lib.plugin.82602734.js
Module not found: Error: Can't resolve './helpers' in 'C:\Users\evers\qmolab.nuxt'
@ ./.nuxt/lib.plugin.82602734.js 1:0-43 7:19-33
@ ./.nuxt/index.js
@ ./.nuxt/client.js
@ multi ./.nuxt/client.js

FATAL Nuxt build error 13:12:56

at WebpackBundler.webpackCompile (node_modules@nuxt\webpack\dist\webpack.js:5351:21)
at processTicksAndRejections (internal/process/task_queues.js:97:5)
at async WebpackBundler.build (node_modules@nuxt\webpack\dist\webpack.js:5300:5)
at async Builder.build (node_modules@nuxt\builder\dist\builder.js:5602:5)
at async Generator.initiate (node_modules@nuxt\generator\dist\generator.js:70:7)
at async Generator.generate (node_modules@nuxt\generator\dist\generator.js:40:5)
at async Object.run (node_modules@nuxt\cli\dist\cli-generate.js:107:24)
at async NuxtCommand.run (node_modules@nuxt\cli\dist\cli-index.js:2759:7)

Route ending in forward slash causes double forward slash in payload path

Sometimes route.path has a trailing slash and this results in a payload URL like https://www.example.com/foo//payload.json. Some servers handle this, however AWS S3 views this as a separate, non-existent, object key.

Something like this may work better:

return document.location.origin + route.path.replace(/\/+$/, "") + '/payload.json'

Support for vue-apollo smart queries?

This is more question than issue but maybe it deserves documenting. If you were using vue-apollo's smart queries instead of asyncData, how would you integrate the payload-extractor? Would you set the skip property from a method? Use middleware to control Apollo?

Tests

Hey 👋

I think some tests for the package would come handy ☺️

Running generate() under a cloud function

Hey @DreaMinder
I was hoping I could ask you some questions about your work on this issue, which was closed: nuxt/nuxt#3798

I'm trying to get generate() to work under a Google Cloud Function, and running into problems and would really appreciate talking to you about it.

Happy to pay for your time, and appreciate any help you can provide. Sorry to bug you via this project, I didn't know any better way to reach out.

Thanks!

Questions about window.__NUXT__ access

This is not an issue, it's just a question. This plugin saved my life to port an entire Wordpress multi language blog to Nuxt.js. I needed to statically render the entire blog and I used nuxt generate, anyway I don't know why asyncData inside my page got called anytime I navigated the site, forcing me to keep the Wordpress server API (they're inside a docker network). With your plugin I just load the payload from .json and everything is ok! I wonder why if I can do something like this to statically render my blog:

  async asyncData (context) {
    if (context.payload) {
      return { post: context.payload }
    } else {
      return { post: window.__NUXT__.data[0].posts[0] }
    }
  }

It looks a dirty solution but it works! I really don't understand why Vue uses window.__NUXT__ to hydrate the component when I go directly to the page and needs to fetch data again if I arrive on the page from a nuxt-link 🤔

Thanks in advance

Caching API calls on server?

Hello @DreaMinder ,

Not a issue here but I'm trying to find a way to avoid hammering my server API during static build. I have a website with 60+ pages and my builds partially fail because of multiple error 500. Some of the pages make multiple API calls.

async asyncData({ $axios, $payloadURL, route, store }) {
   if (process.static && process.client && $payloadURL) {
     return await $axios.$get($payloadURL(route))
   }

   if (store.state.marquee.length <= 0) {
     const { marquee } = await wpFetch('common')
     store.dispatch('LOAD_MARQUEE', marquee)
   }

   const { homepage } = await wpFetch('pages')
   const posts = await wpFetch('posts')

   return {
     ...homepage,
     agenda: await wpFetch('agenda'),
     articles: getArticles(posts, 5, true),
     frames: getFrames(posts, 1, true),
   }
 },

I searched for way to solve this, including in this topic but it's not working in static SSR mode. I tried the payload method too : it reduces my API calls during build but actually calls APIs again during client navigation so it isn't much of solution either.

Do you have your own method to approach to solve this or/and would you consider adding this as a feature to your module?

Any solution with Graphql?

Currently the module works with an axios request but is there a pattern that allows for Apollo with graphql?

[Multiple Requests] Documentation isn't clear how to

First I would like to thank you for the good work 👍

For my private website I'm using WordPress as a headless CMS to fetch data from. Therefore I have to make multiple requests like so:

async asyncData({$axios}) {
    let [[ index ], { name, description }, [ index_meta ], menus ] = await Promise.all([
        $axios.$get(`/index`),
        $axios.$get(`https://admin.tobias-thiele.de/api/`),
        $axios.$get(`/index?slug=index&_embed=true`),
        $axios.$get(`https://admin.tobias-thiele.de/api/routes/menu`)
    ]);


    name = name.replace(/&amp;/g, '&');
    description = description.replace(/&amp;/g, '&');

    let mainMenu = [];
    menus["main-menu"].forEach(menu => {
        mainMenu.push({
            title: menu.title,
            link: menu.classes[0]
        });
    });

    return {index, name, description, mainMenu, index_meta};
},

Could you please explain me how I could use your module with these kind of data?

Thanks ❤️

PS: I tried around but all I got is the following error:

image

asyncData and vuex

In my asyncData I usually don't return data, instead I assign the data to the vuex store. How is this possible with this module?

Question - Will this work with Async Fetch on Nuxt?

Hi,

Thank you for this package.

Could you please confirm will this work on async fetch on nuxt components(not page)?
As I do not want to query firestore after generating the pages.

I have my async fetch call in one of the component like this.

  async fetch() {
    const response = this.$fireStore.collection('recipes').limit(10)
    try {
      await response.get().then((querySnapshot) => {
               querySnapshot.forEach((doc) => { this.articles.push({  ...doc.data()       })
        })      })    } catch (e) {
      console.log('There is some error: ' + e)
    }  }

my Nuxt generate dynamic route as below.

 generate: {
    minify: false,
    async routes() {
      const { StoreDB } = require('./services/fireinit')
      const qs = await StoreDB.collection('recipes')
        .where('publish', '==', true)
        .orderBy('updated', 'desc')
        .limit(10)
        .get()
      return qs.docs.map((x) => `/${x.data().slug}`)
    }
  }

Your help is much appreciated.

Don't cache specific API calls

Is it possible to not cache specific API calls? Say I have an e-commerce shop, I don't mind all the product page responses are cached, but I'd prefer if "add to cart" response is not cached, and also product variations API request.

nuxt generate with custom 'dir' is not working

I am using nuxt generate with a custom directory - https://nuxtjs.org/api/configuration-generate/#dir.

I am generating the builds in dist/apps

The links for the payload thus points to the wrong directory.

Can this be fixed or is a workaround available? Ideally, the package should take this property into consideration or maybe work with relative URLs.

let path = route === '/' ? '' : route

Looking forward to your input @DreaMinder.

Thanks 😄

Payload cache issue

Hi,

First of all, thank you very much for this nuxt module. I found it by reading this.

I'm facing a cache problem whenever I rebuild my app.

Nuxt generate will name each js chunk with some random string (for example, 4e0f06a8ac4f255a8faf.js). However, the names for payload.json and payload.js never change between rebuilds (am I missing something here?) .

When, for example. payload.json changes between rebuilds and the browser has the older file in the cache, it will load the older file from disk and not the updated data from the server.

The client is getting stale data when he or she clicks a nuxt-link.

Any ideas on how to solve this?

nuxt-generate: "Payload-extractor had to duplicate some nested routes data in extracted payload"

I'm getting lots of Payload-extractor had to duplicate some nested routes data in extracted payload warnings when using nuxt-generate-cluster's nuxt-generate.

With Nuxt's built-in nuxt generate I get no such messages at all.
EDIT: Sometimes I do. Couldn't figure out the rule yet.

Could you please elaborate a bit on this message and its impact on build time and size? Also let me know if there is something to optimise on my end.

nuxt.config.js:

const axios = Axios.create({
  baseURL: process.env.API_URL
})

module.exports = {
  <...>
  generate: {
    workers: 8,
    async routes() {
      const [
        { data: authors },
        { data: yearsIssues },
        { data: issues },
        { data: articles }
      ] = await Promise.all([
        axios.get(`/all_authors_contribution`),
        axios.get(`/all_issues_by_year`),
        axios.get(`/all_issues`),
        axios.get(`/articles`)
      ])
      const authorsRoutes = authors.map(entry => ({
        route: `/authors/${entry.author.author_id}`,
        payload: entry
      }))
      const yearsRoutes = yearsIssues.map(yearIssues => ({
        route: `/archive/${yearIssues.year}`,
        payload: yearIssues.issues
      }))
      const issuesRoutes = issues.map(issue => ({
        route: `/archive/${issue.feature || issue.year}/${issue.order_within_year}`,
        payload: issue
      }))
      const articlesRoutes = articles
        .filter(article => !!article.issue)
        .map(article => ({
          route: `/archive/${article.issue.feature || article.issue.year}/${article.issue.order_within_year}/${article.section.order_within_issue}/${article.order_within_section}`,
          payload: article
        }))
      return [
        ...authorsRoutes,
        ...yearsRoutes,
        ...issuesRoutes,
        ...articlesRoutes
      ]
    },
    <...>
  }
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.