Code Monkey home page Code Monkey logo

shakapacker's Introduction

Shakapacker (v8)

Official, actively maintained successor to rails/webpacker.ShakaCode stands behind the long-term maintenance and development of this project for the Rails community.

  • ⚠️ See the 6-stable branch for Shakapacker v6.x code and documentation. ⚠️
  • See V8 Upgrade for upgrading from the v7 release.
  • See V7 Upgrade for upgrading from the v6 release.
  • See V6 Upgrade for upgrading from v5 or prior v6 releases.

Ruby specs Jest specs Rubocop JS lint

node.js Gem npm version

Shakapacker makes it easy to use the JavaScript pre-processor and bundler Webpack v5+ to manage frontend JavaScript in Rails. It can coexist with the asset pipeline, leaving Webpack responsible solely for frontend JavaScript, or can be used exclusively, making it also responsible for images, fonts, and CSS.

Check out 6.1.1+ for SWC and esbuild-loader support! They are faster than Babel!

See a comparison of Shakapacker with jsbundling-rails. For an in-depth discussion of choosing between shakapacker and jsbundling-rails, see the discussion Webpacker alternatives - which path should we go to? #8783 and the resulting PR Switch away from Webpacker to Shakapacker #10389.

For discussions, see our Slack Channel.


ShakaCode Support

ShakaCode focuses on helping Ruby on Rails teams use React and Webpack better. We can upgrade your project and improve your development and customer experiences, allowing you to focus on building new features or fixing bugs instead.

For an overview of working with us, see our Client Engagement Model article and how we bill for time.

We also specialize in helping development teams lower infrastructure and CI costs. Check out our project Control Plane Flow, which can allow you to get the ease of Heroku with the power of Kubernetes and big cost savings.

If you think ShakaCode can help your project, click here to book a call with Justin Gordon, the creator of React on Rails and Shakapacker.

Here's a testimonial of how ShakaCode can help from Florian Gößler of Blinkist, January 2, 2023:

Hey Justin 👋

I just wanted to let you know that we today shipped the webpacker to shakapacker upgrades and it all seems to be running smoothly! Thanks again for all your support and your teams work! 😍

On top of your work, it was now also very easy for me to upgrade Tailwind and include our external node_module based web component library which we were using for our other (more modern) apps already. That work is going to be shipped later this week though as we are polishing the last bits of it. 😉

Have a great 2023 and maybe we get to work together again later in the year! 🙌

Read the full review here.


Prerequisites

  • Ruby 2.7+
  • Rails 5.2+
  • Node.js 14+

Features

  • Rails view helpers that fully support Webpack output, including HMR and code splitting.
  • Convenient but not required webpack configuration. The only requirement is that your webpack configuration creates a manifest.
  • HMR with the shakapacker-dev-server, such as for hot-reloading React!
  • Automatic code splitting using multiple entry points to optimize JavaScript downloads.
  • Support for NPM, Yarn (classic and berry), PNPM, and Bun
  • Webpack v5+
  • ES6 with babel, SWC, or Esbuild
  • Asset compression, source-maps, and minification
  • CDN support
  • Extensible and configurable. For example, all major dependencies are specified as peers, so you can upgrade easily.

Optional support

Requires extra packages to be installed.

  • React
  • TypeScript
  • Stylesheets - Sass, Less, Stylus and Css, PostCSS
  • CoffeeScript

Installation

Rails v6+

With Rails v6+, skip JavaScript for a new app and follow below Manual Installation Steps to manually add the shakapacker gem to your Gemfile.

rails new myapp --skip-javascript

Note, Rails 6 installs the older v5 version of webpacker unless you specify --skip-javascript.

Add shakapacker gem to your Gemfile:

bundle add shakapacker --strict

Then run the following to install Shakapacker:

./bin/bundle install
./bin/rails shakapacker:install

Before initiating the installation process, ensure you have committed all the changes. While installing Shakapacker, there might be some conflict between the existing file content and what Shakapacker tries to copy. You can either approve all the prompts for overriding these files or use the FORCE=true environment variable before the installation command to force the override without any prompt.

Shakapacker uses the package_json gem to handle updating the package.json and interacting with the underlying package manager of choice for managing dependencies and running commands; the package manager is managed using the packageManager property in the package.json, otherwise falling back to the value of PACKAGE_JSON_FALLBACK_MANAGER if set or otherwise npm.

If packageManager is not set when running shakapacker:install, Shakapacker will set it based on the lockfile and the result of calling --version on the inferred manager; if no lockfile is present, then npm be used unless you choose to explicitly set the PACKAGE_JSON_FALLBACK_MANAGER to your preferred package manager.

Note

The packageManager property is only used to determine the package manager to use, based primarily on its name. The version (if present) is only used to determine if Yarn Classic or Yarn Berry should be used, but is otherwise not checked, nor is corepack used to ensure that the package manager is installed.

It is up to the developer to ensure that the desired package manager is actually install at the right version, which can be done using corepack or by other means.

See here for a list of the supported package managers and more information; note that package_json does not handle ensuring the manager is installed.

If you wish to use Yarn PnP you will need to configure Babel using a babel.config.js file rather than via package.json - see customizing Babel Config for examples on how to do this.

Note

The rest of the documentation will only reference npm when providing commands such as to install optional packages except in cases where a particular package manager requires a very different command; otherwise it should be safe to just replace npm with the name of your preferred package manager when running the command

Note, in v6+, most JS packages are peer dependencies. Thus, the installer will add the packages:

  • @babel/core
  • @babel/plugin-transform-runtime
  • @babel/preset-env
  • @babel/runtime
  • babel-loader
  • compression-webpack-plugin
  • terser-webpack-plugin
  • webpack
  • webpack-assets-manifest
  • webpack-cli
  • webpack-merge
  • webpack-sources
  • webpack-dev-server

Previously, these "webpack" and "babel" packages were direct dependencies for shakapacker. By making these peer dependencies, you have control over the versions used in your webpack and babel configs.

Concepts

At its core, Shakapacker's essential function is to:

  1. Provide configuration by a single file used by both Rails view helpers and JavaScript webpack compilation code.
  2. Provide Rails view helpers, utilizing this configuration file so that a webpage can load JavaScript, CSS, and other static assets compiled by webpack, supporting bundle splitting, fingerprinting, and HMR.
  3. Provide a community-supported, default webpack compilation that generates the necessary bundles and manifest, using the same configuration file. This compilation can be extended for any needs.

Usage

Configuration and Code

You will need your file system to correspond to the setup of your config/shakapacker.yml file.

Suppose you have the following configuration:

shakapacker.yml

default: &default
  source_path: app/javascript
  source_entry_path: packs 
  public_root_path: public
  public_output_path: packs
  nested_entries: false
# And more

And that maps to a directory structure like this:

app/javascript:
  └── packs:               # sets up webpack entries
  │   └── application.js   # references ../src/my_component.js
  │   └── application.css
  └── src:                 # any directory name is fine. Referenced files need to be under source_path
  │   └── my_component.js
  └── stylesheets:
  │   └── my_styles.css
  └── images:
      └── logo.svg
public/packs                # webpack output

Webpack intelligently includes only necessary files. In this example, the file packs/application.js would reference ../src/my_component.js

nested_entries allows you to have webpack entry points nested in subdirectories. This defaults to true as of shakapacker v7. With nested_entries: false, you can have your entire source_path used for your source (using the source_entry_path: /) and you place files at the top level that you want as entry points. nested_entries: true allows you to have entries that are in subdirectories. This is useful if you have entries that are generated, so you can have a generated subdirectory and easily separate generated files from the rest of your codebase.

To enable/disable the usage of contentHash in any node environment (specified using the NODE_ENV environment variable), add/modify useContentHash with a boolean value in config/shakapacker.yml. This feature is disabled for all environments except production by default. You may not disable the content hash for a NODE_ENV of production as that would break the browser caching of assets. Notice that despite the possibility of enabling this option for the development environment, it is not recommended.

Setting custom config path

You can use the environment variable SHAKAPACKER_CONFIG to enforce a particular path to the config file rather than the default config/shakapacker.yml.

View Helpers

The Shakapacker view helpers generate the script and link tags to get the webpack output onto your views.

Be sure to consult the API documentation in the source code of helper.rb.

Note: For your styles or static assets files to be available in your view, you would need to link them in your "pack" or entry file. Otherwise, Webpack won't know to package up those files.

View Helpers javascript_pack_tag and stylesheet_pack_tag

These view helpers take your shakapacker.yml configuration file and the resulting webpack compilation manifest.json and generate the HTML to load the assets.

You can then link the JavaScript pack in Rails views using the javascript_pack_tag helper. If you have styles imported in your pack file, you can link them by using stylesheet_pack_tag:

<%= javascript_pack_tag 'application' %>
<%= stylesheet_pack_tag 'application' %>

The javascript_pack_tag and stylesheet_pack_tag helpers will include all the transpiled packs with the chunks in your view, which creates HTML tags for all the chunks.

You can provide multiple packs and other attributes. Note, defer defaults to showing.

<%= javascript_pack_tag 'calendar', 'map', 'data-turbo-track': 'reload' %>

The resulting HTML would look like this:

<script src="/packs/vendor-16838bab065ae1e314.js" data-turbo-track="reload" defer></script>
<script src="/packs/calendar~runtime-16838bab065ae1e314.js" data-turbo-track="reload" defer></script>
<script src="/packs/calendar-1016838bab065ae1e314.js" data-turbo-track="reload" defer"></script>
<script src="/packs/map~runtime-16838bab065ae1e314.js" data-turbo-track="reload" defer></script>
<script src="/packs/map-16838bab065ae1e314.js" data-turbo-track="reload" defer></script>

In this output, both the calendar and map codes might refer to other common libraries. Those get placed in something like the vendor bundle. The view helper removes any duplication.

Note, the default of "defer" for the javascript_pack_tag. You can override that to false. If you expose jquery globally with expose-loader, by using import $ from "expose-loader?exposes=$,jQuery!jquery" in your app/javascript/application.js, pass the option defer: false to your javascript_pack_tag.

Important: Pass all your pack names as multiple arguments, not multiple calls, when using javascript_pack_tag and the stylesheet_pack_tag. Otherwise, you will get duplicated chunks on the page.

<%# DO %>
<%= javascript_pack_tag 'calendar', 'map' %>

<%# DON'T %>
<%= javascript_pack_tag 'calendar' %>
<%= javascript_pack_tag 'map' %>

While this also generally applies to stylesheet_pack_tag, you may use multiple calls to stylesheet_pack_tag if, say, you require multiple <style> tags for different output media:

<%= stylesheet_pack_tag 'application', media: 'screen' %>
<%= stylesheet_pack_tag 'print', media: 'print' %>

View Helper append_javascript_pack_tag, prepend_javascript_pack_tag and append_stylesheet_pack_tag

If you need to configure your script pack names or stylesheet pack names from the view for a route or partials, then you will need some logic to ensure you call the helpers only once with multiple arguments. The new view helpers, append_javascript_pack_tag and append_stylesheet_pack_tag can solve this problem. The helper append_javascript_pack_tag will queue up script packs when the javascript_pack_tag is finally used. Similarly,append_stylesheet_pack_tag will queue up style packs when the stylesheet_pack_tag is finally used.

Main view:

<% append_javascript_pack_tag 'calendar' %>
<% append_stylesheet_pack_tag 'calendar' %>

Some partial:

<% append_javascript_pack_tag 'map' %>
<% append_stylesheet_pack_tag 'map' %>

And the main layout has:

<%= javascript_pack_tag 'application' %>
<%= stylesheet_pack_tag 'application' %>

is the same as using this in the main layout:

<%= javascript_pack_tag 'calendar', 'map', application' %>
<%= stylesheet_pack_tag 'calendar', 'map', application' %>

However, you typically can't do that in the main layout, as the view and partial codes will depend on the route.

Thus, you can distribute the logic of what packs are needed for any route. All the magic of splitting up the code and CSS was automatic!

Important: These helpers can be used anywhere in your application as long as they are executed BEFORE (javascript/stylesheet)_pack_tag respectively. If you attempt to call one of these helpers after the respective (javascript/stylesheet)_pack_tag, an error will be raised.

The typical issue is that your layout might reference some partials that need to configure packs. A good way to solve this problem is to use content_for to ensure that the code to render your partial comes before the call to javascript_pack_tag.

<% content_for :footer do 
   render 'shared/footer' %>
   
<%= javascript_pack_tag %>

<%= content_for :footer %>

There is also prepend_javascript_pack_tag that will put the entry at the front of the queue. This is handy when you want an entry in the main layout to go before the partial and main layout append_javascript_pack_tag entries.

Main view:

<% append_javascript_pack_tag 'map' %>

Some partial:

<% append_javascript_pack_tag 'map' %>

And the main layout has:

<% prepend_javascript_pack_tag 'main' %>
<%= javascript_pack_tag 'application' %>

is the same as using this in the main layout:

<%= javascript_pack_tag 'main', 'calendar', 'map', application' %>

For alternative options for setting the additional packs, see this discussion.

View Helper: asset_pack_path

If you want to link a static asset for <img /> tag, you can use the asset_pack_path helper:

<img src="<%= asset_pack_path 'static/logo.svg' %>" />

View Helper: image_pack_tag

Or use the dedicated helper:

<%= image_pack_tag 'application.png', size: '16x10', alt: 'Edit Entry' %>
<%= image_pack_tag 'picture.png', srcset: { 'picture-2x.png' => '2x' } %>

View Helper: favicon_pack_tag

If you want to create a favicon:

<%= favicon_pack_tag 'mb-icon.png', rel: 'apple-touch-icon', type: 'image/png' %>

View Helper: preload_pack_asset

If you want to preload a static asset in your <head>, you can use the preload_pack_asset helper:

<%= preload_pack_asset 'fonts/fa-regular-400.woff2' %>

Images in Stylesheets

If you want to use images in your stylesheets:

.foo {
  background-image: url('../images/logo.svg')
}

Server-Side Rendering (SSR)

Note, if you are using server-side rendering of JavaScript with dynamic code-splitting, as is often done with extensions to Shakapacker, like React on Rails, your JavaScript should create the link prefetch HTML tags that you will use, so you won't need to use to asset_pack_path in those circumstances.

Development

Shakapacker ships with two binstubs: ./bin/shakapacker and ./bin/shakapacker-dev-server. Both are thin wrappers around the standard webpack.js and webpack-dev-server.js executables to ensure that the right configuration files and environmental variables are loaded based on your environment.

Note: older Shakapacker installations had set a missing NODE_ENV in the binstubs. Please remove this for versions 6.5.2 and newer.

Automatic Webpack Code Building

Shakapacker can be configured to automatically compile on demand when needed using compile option in the shakapacker.yml. This happens when you refer to any of the pack assets using the Shakapacker helper methods. This means that you don't have to run any separate processes. Compilation errors are logged to the standard Rails log. However, this auto-compilation happens when a web request is made that requires an updated webpack build, not when files change. Thus, that can be painfully slow for front-end development in this default way. Instead, you should either run the bin/shakapacker --watch or run ./bin/shakapacker-dev-server during development.

The compile: true option can be more useful for test and production builds.

Compiler strategies

Shakapacker ships with two different strategies that are used to determine whether assets need recompilation per the compile: true option:

  • digest - This strategy calculates SHA1 digest of files in your watched paths (see below). The calculated digest is then stored in a temp file. To check whether the assets need to be recompiled, Shakapacker calculates the SHA1 of the watched files and compares it with the one stored. If the digests are equal, no recompilation occurs. If the digests are different or the temp file is missing, files are recompiled.
  • mtime - This strategy looks at the last "modified at" timestamps of both files AND directories in your watched paths. The timestamp of the most recent file or directory is then compared with the timestamp of manifest.json file generated. If the manifest file timestamp is newer than one of the most recently modified files or directories in the watched paths, no recompilation occurs. If the manifest file is older, files are recompiled.

The mtime strategy is generally faster than the digest one, but it requires stable timestamps, this makes it perfect for a development environment, such as needing to rebuild bundles for tests, or if you're not changing frontend assets much.

In production or CI environments, the digest strategy is more suitable, unless you are using incremental builds or caching and can guarantee that the timestamps will not change after e.g. cache restore. However, many production or CI environments will explicitly compile assets, so compile: false is more appropriate. Otherwise, you'll waste time either checking file timestamps or computing digests.

You can control what strategy is used by the compiler_strategy option in shakapacker.yml config file. By default mtime strategy is used in development environment, digest is used elsewhere.

Note

If you are not using the shakapacker-dev-server, your packs will be served by the Rails public file server. If you've enabled caching (Rails application config.action_controller.perform_caching setting), your changes will likely not be picked up due to Cache-Control header being set and assets being cached in the browser memory.

For more details see issue 88: Caching issues in Development since migrating to Shakapacker.

If you want to use live code reloading, or you have enough JavaScript that on-demand compilation is too slow, you'll need to run ./bin/shakapacker-dev-server. This process will watch for changes in the relevant files, defined by shakapacker.yml configuration settings for source_path, source_entry_path, and additional_paths, and it will then automatically reload the browser to match. This feature is also known as Hot Module Replacement.

Common Development Commands

# webpack dev server
./bin/shakapacker-dev-server

# watcher
./bin/shakapacker --watch --progress

# standalone build
./bin/shakapacker --progress

Once you start this webpack development server, Shakapacker will automatically start proxying all webpack asset requests to this server. When you stop this server, Rails will detect that it's not running and Rails will revert back to on-demand compilation if you have the compile option set to true in your config/shakapacker.yml

You can use environment variables as options supported by webpack-dev-server in the form SHAKAPACKER_DEV_SERVER_<OPTION>. Please note that these environmental variables will always take precedence over the ones already set in the configuration file, and that the same environmental variables must be available to the rails server process.

SHAKAPACKER_DEV_SERVER_PORT=4305 SHAKAPACKER_DEV_SERVER_HOST=example.com SHAKAPACKER_DEV_SERVER_INLINE=true SHAKAPACKER_DEV_SERVER_HOT=false ./bin/shakapacker-dev-server

By default, the webpack dev server listens on localhost:3035 in development for security purposes. However, if you want your app to be available on port 4035 over local LAN IP or a VM instance like vagrant, you can set the port and host when running ./bin/shakapacker-dev-server binstub:

SHAKAPACKER_DEV_SERVER_PORT=4305 SHAKAPACKER_DEV_SERVER_HOST=0.0.0.0 ./bin/shakapacker-dev-server

Note: You need to allow webpack-dev-server host as an allowed origin for connect-src if you are running your application in a restrict CSP environment (like Rails 5.2+). This can be done in Rails 5.2+ in the CSP initializer config/initializers/content_security_policy.rb with a snippet like this:

Rails.application.config.content_security_policy do |policy|
  policy.connect_src :self, :https, 'http://localhost:3035', 'ws://localhost:3035' if Rails.env.development?
end

Note: Don't forget to prefix ruby when running these binstubs on Windows

Webpack Configuration

First, you don't need to use Shakapacker's webpack configuration. However, the shakapacker NPM package provides convenient access to configuration code that reads the config/shakapacker.yml file which the view helpers also use. If you have your customized webpack configuration, at the minimum, you must ensure:

  1. Your output files go to the right directory
  2. Your output includes a manifest, via package webpack-assets-manifest that maps output names (your 'packs') to the fingerprinted versions, including bundle-splitting dependencies. That's the main secret sauce of Shakapacker!

The webpack configuration used by Shakapacker lives in config/webpack/webpack.config.js; this makes it easy to customize the configuration beyond what's available in config/shakapacker.yml by giving you complete control of the final configuration. By default, this file exports the result of generateWebpackConfig which handles generating a webpack configuration based on config/shakapacker.yml.

The easiest way to modify this config is to pass your desired customizations to generateWebpackConfig which will use webpack-merge to merge them with the configuration generated from config/shakapacker.yml:

// config/webpack/webpack.config.js
const { generateWebpackConfig } = require('shakapacker')

const options = {
  resolve: {
      extensions: ['.css', '.ts', '.tsx']
  }
}

// This results in a new object copied from the mutable global
module.exports = generateWebpackConfig(options)

The shakapacker package also exports the merge function from webpack-merge to make it easier to do more advanced customizations:

// config/webpack/webpack.config.js
const { generateWebpackConfig, merge } = require('shakapacker')

const webpackConfig = generateWebpackConfig()

const options = {
  resolve: {
    extensions: ['.css', '.ts', '.tsx']
  }
}

module.exports = merge(options, webpackConfig)

This example is based on an example project

Shakapacker gives you a default configuration file config/webpack/webpack.config.js, which, by default, you don't need to make any changes to config/webpack/webpack.config.js since it's a standard production-ready configuration. However, you will probably want to customize or add a new loader by modifying the webpack configuration, as shown above.

You might add separate files to keep your code more organized.

// config/webpack/custom.js
module.exports = {
  resolve: {
    alias: {
      jquery: 'jquery/src/jquery',
      vue: 'vue/dist/vue.js',
      React: 'react',
      ReactDOM: 'react-dom',
      vue_resource: 'vue-resource/dist/vue-resource'
    }
  }
}

Then require this file in your config/webpack/webpack.config.js:

// config/webpack/webpack.config.js
// use the new NPM package name, `shakapacker`.
const { generateWebpackConfig } = require('shakapacker')

const customConfig = require('./custom')

module.exports = generateWebpackConfig(customConfig)

If you need access to configs within Shakapacker's configuration, you can import them like so:

// config/webpack/webpack.config.js
const { generateWebpackConfig } = require('shakapacker')

const webpackConfig = generateWebpackConfig()

console.log(webpackConfig.output_path)
console.log(webpackConfig.source_path)

// Or to print out your whole webpack configuration
console.log(JSON.stringify(webpackConfig, undefined, 2))

You may want to modify the rules in the default configuration. For instance, if you are using a custom svg loader, you may want to remove .svg from the default file loader rules. You can search and filter the default rules like so:

const fileRule = config.module.rules.find(rule => rule.test.test('.svg'));
// removing svg from asset file rule's test RegExp
fileRule.test = /\.(bmp|gif|jpe?g|png|tiff|ico|avif|webp|eot|otf|ttf|woff|woff2)$/
// changing the rule type from 'asset/resource' to 'asset'. See https://webpack.js.org/guides/asset-modules/
fileRule.type = 'asset'

Babel configuration

By default, you will find the Shakapacker preset in your package.json. Note, you need to use the new NPM package name, shakapacker.

"babel": {
  "presets": [
    "./node_modules/shakapacker/package/babel/preset.js"
  ]
},

Optionally, you can change your Babel configuration by removing these lines in your package.json and adding a Babel configuration file](https://babeljs.io/docs/en/config-files) to your project. For an example of customization based on the original, see Customizing Babel Config.

SWC configuration

You can try out experimental integration with the SWC loader. You can read more at SWC usage docs.

Please note that if you want opt-in to use SWC, you can skip React integration instructions as it is supported out of the box.

esbuild loader configuration

You can try out experimental integration with the esbuild-loader. You can read more at esbuild-loader usage docs.

Please note that if you want opt-in to use esbuild-loader, you can skip React integration instructions as it is supported out of the box.

Integrations

Shakapacker out of the box supports JS and static assets (fonts, images etc.) compilation. To enable support for CoffeeScript or TypeScript install relevant packages:

React

See here for detailed instructions on how to configure Shakapacker to bundle a React app (with optional HMR).

See also Customizing Babel Config for an example React configuration.

TypeScript

npm install typescript @babel/preset-typescript

Babel won’t perform any type-checking on TypeScript code. To optionally use type-checking run:

npm install fork-ts-checker-webpack-plugin

Add tsconfig.json

{
  "compilerOptions": {
    "declaration": false,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "lib": ["es6", "dom"],
    "module": "es6",
    "moduleResolution": "node",
    "sourceMap": true,
    "target": "es5",
    "jsx": "react",
    "noEmit": true
  },
  "exclude": ["**/*.spec.ts", "node_modules", "vendor", "public"],
  "compileOnSave": false
}

Then modify the webpack config to use it as a plugin:

// config/webpack/webpack.config.js
const { generateWebpackConfig } = require("shakapacker");
const ForkTSCheckerWebpackPlugin = require("fork-ts-checker-webpack-plugin");

module.exports = generateWebpackConfig({
  plugins: [new ForkTSCheckerWebpackPlugin()],
});

CSS

To enable CSS support in your application, add the following packages:

npm install css-loader style-loader mini-css-extract-plugin css-minimizer-webpack-plugin

Optionally, add the CSS extension to webpack config for easy resolution.

// config/webpack/webpack.config.js
const { generateWebpackConfig } = require('shakapacker')

const customConfig = {
  resolve: {
    extensions: ['.css']
  }
}

module.exports = generateWebpackConfig(customConfig)

To enable PostCSS, Sass or Less support, add CSS support first and then add the relevant pre-processors:

Postcss

npm install postcss postcss-loader

Optionally add these two plugins if they are required in your postcss.config.js:

npm install postcss-preset-env postcss-flexbugs-fixes

Sass

npm install sass-loader

You will also need to install Dart Sass, Node Sass or Sass Embedded to pick the implementation to use. sass-loader will automatically pick an implementation based on installed packages.

Please refer to sass-loader documentation and individual packages repos for more information on all the options.

Dart Sass
npm install sass
Node Sass
npm install node-sass
Sass Embedded
npm install sass-embedded

Less

npm install less less-loader

Stylus

npm install stylus stylus-loader

CoffeeScript

npm install coffeescript coffee-loader

Other frameworks

Please follow Webpack integration guide for the relevant framework or library,

  1. Svelte
  2. Angular
  3. Vue

For example to add Vue support:

// config/webpack/rules/vue.js
const { VueLoaderPlugin } = require('vue-loader')

module.exports = {
  module: {
    rules: [
      {
        test: /\.vue$/,
        loader: 'vue-loader'
      }
    ]
  },
  plugins: [new VueLoaderPlugin()],
  resolve: {
    extensions: ['.vue']
  }
}
// config/webpack/webpack.config.js
const { generateWebpackConfig, merge } = require('shakapacker')

const webpackConfig = generateWebpackConfig()

const vueConfig = require('./rules/vue')

module.exports = merge(vueConfig, webpackConfig)

Custom Rails environments

Out of the box Shakapacker ships with - development, test and production environments in config/shakapacker.yml however, in most production apps extra environments are needed as part of the deployment workflow. Shakapacker supports this out of the box from version 3.4.0+ onwards.

You can choose to define additional environment configurations in shakapacker.yml,

staging:
  <<: *default

  # Production depends on precompilation of packs prior to booting for performance.
  compile: false

  # Cache manifest.json for performance
  cache_manifest: true

  # Compile staging packs to a separate directory
  public_output_path: packs-staging

Otherwise, Shakapacker will use the production environment as a fallback environment for loading configurations. Please note, NODE_ENV can either be set to production, development or test. This means you don't need to create additional environment files inside config/shakapacker/* and instead use shakapacker.yml to load different configurations using RAILS_ENV.

For example, the below command will compile assets in production mode but will use staging configurations from config/shakapacker.yml if available or use fallback production environment configuration:

RAILS_ENV=staging bundle exec rails assets:precompile

And, this will compile in development mode and load configuration for the cucumber environment if defined in shakapacker.yml or fallback to production configuration

RAILS_ENV=cucumber NODE_ENV=development bundle exec rails assets:precompile

Please note, binstubs compiles in development mode however rake tasks compiles in production mode.

# Compiles in development mode unless NODE_ENV is specified, per the binstub source
./bin/shakapacker
./bin/shakapacker-dev-server

# Compiles in production mode by default unless NODE_ENV is specified, per `lib/tasks/shakapacker/compile.rake`
bundle exec rails assets:precompile
bundle exec rails shakapacker:compile

Upgrading

You can run the following commands to upgrade Shakapacker to the latest stable version. This process involves upgrading the gem and related JavaScript packages:

# check your Gemfile for version restrictions
bundle update shakapacker

# overwrite your changes to the default install files and revert any unwanted changes from the install
rails shakapacker:install

# using npm
npm install shakapacker@latest
npm install webpack-dev-server@latest

# using yarn classic
yarn upgrade shakapacker --latest
yarn upgrade webpack-dev-server --latest

# using yarn berry
yarn up shakapacker@latest
yarn up webpack-dev-server@latest

# using pnpm
pnpm up shakapacker@latest
pnpm up webpack-dev-server@latest

# Or to install the latest release (including pre-releases)
npm install shakapacker@next

Also, consult the CHANGELOG for additional upgrade links.

Paths

By default, Shakapacker ships with simple conventions for where the JavaScript app files and compiled webpack bundles will go in your Rails app. All these options are configurable from config/shakapacker.yml file.

The configuration for what webpack is supposed to compile by default rests on the convention that every file in app/javascript/(default) or whatever path you set for source_entry_path in the shakapacker.yml configuration is turned into their own output files (or entry points, as webpack calls it). Therefore you don't want to put any file inside app/javascript directory that you do not want to be an entry file. As a rule of thumb, put all files you want to link in your views inside "app/javascript/" directory and keep everything else under subdirectories like app/javascript/controllers.

Suppose you want to change the source directory from app/javascript to frontend and output to assets/packs. This is how you would do it:

# config/shakapacker.yml
source_path: frontend # packs are the files in frontend/
public_output_path: assets/packs # outputs to => public/assets/packs

Similarly, you can also control and configure webpack-dev-server settings from config/shakapacker.yml file:

# config/shakapacker.yml
development:
  dev_server:
    host: localhost
    port: 3035

If you have hmr turned to true and inline_css is not false, then the stylesheet_pack_tag generates no output, as you will want to configure your styles to be inlined in your JavaScript for hot reloading. During production and testing, the stylesheet_pack_tag will create the appropriate HTML tags.

If you want to have HMR and separate link tags, set hmr: true and inline_css: false. This will cause styles to be extracted and reloaded with the mini-css-extract-plugin loader. Note that in this scenario, you do not need to include style-loader in your project dependencies.

Additional paths

If you are adding Shakapacker to an existing app that has most of the assets inside app/assets or inside an engine, and you want to share that with webpack modules, you can use the additional_paths option available in config/shakapacker.yml. This lets you add additional paths that webpack should look up when resolving modules:

additional_paths: ['app/assets', 'vendor/assets']

You can then import these items inside your modules like so:

// Note it's relative to parent directory i.e. app/assets
import 'stylesheets/main'
import 'images/rails.png'

Assets put in these folders will also have their path stripped just like with the source_path.

Example:

A file in app/assets/images/image.svg with additional_paths: ['app/assets'] will result in static/images/image.svg

Note: Please be careful when adding paths here otherwise it will make the compilation slow, consider adding specific paths instead of the whole parent directory if you just need to reference one or two modules

Also note: While importing assets living outside your source_path defined in shakapacker.yml (like, for instance, assets under app/assets) from within your packs using relative paths like import '../../assets/javascripts/file.js' will work in development, Shakapacker won't recompile the bundle in production unless a file that lives in one of it's watched paths has changed (check out Shakapacker::MtimeStrategy#latest_modified_timestamp or Shakapacker::DigestStrategy#watched_files_digest depending on strategy configured by compiler_strategy option in shakapacker.yml). That's why you'd need to add app/assets to the additional_paths as stated above and use import 'javascripts/file.js' instead.

Deployment

Shakapacker hooks up a new shakapacker:compile task to assets:precompile, which gets run whenever you run assets:precompile. If you are not using Sprockets, shakapacker:compile is automatically aliased to assets:precompile. Similar to sprockets both rake tasks will compile packs in production mode but will use RAILS_ENV to load configuration from config/shakapacker.yml (if available).

This behavior is optional & can be disabled by either setting a SHAKAPACKER_PRECOMPILE environment variable to false, no, n, or f, or by setting a shakapacker_precompile key in your shakapacker.yml to false. (source code)

When compiling assets for production on a remote server, such as a continuous integration environment, it's recommended to ensure the exact versions specified in your lockfile are installed:

# using npm
npm ci

# using yarn classic
yarn install --frozen-lockfile

# using yarn berry
yarn install --immutable

# using pnpm
pnpm install --frozen-lockfile

# using bun
bun install --frozen-lockfile

If you are using a CDN setup, Shakapacker does NOT use the ASSET_HOST environment variable to prefix URLs for assets during bundle compilation. You must use the SHAKAPACKER_ASSET_HOST environment variable instead (WEBPACKER_ASSET_HOST if you're using any version of Webpacker or Shakapacker before Shakapacker v7).

Example Apps

Troubleshooting

See the doc page for Troubleshooting.

Contributing

We encourage you to contribute to Shakapacker! See CONTRIBUTING for guidelines about how to proceed. We have a Slack discussion channel.

License

Shakapacker is released under the MIT License.

Supporters

The following companies support our Open Source projects, and ShakaCode uses their products!



JetBrains ScoutAPM Control Plane
BrowserStack Rails Autoscale Honeybadger Reviewable

shakapacker's People

Contributors

ahangarha avatar aried3r avatar dependabot[bot] avatar dhh avatar g-rath avatar gauravtiwari avatar guillaumebriday avatar guilleiguaran avatar jakeniemiec avatar javan avatar judahmeek avatar justin808 avatar lazylester avatar maschwenk avatar merqlove avatar palkan avatar pustovalov avatar rafaelfranca avatar renchap avatar rossta avatar schpet avatar sirius248 avatar smondal avatar swrobel avatar t27duck avatar tagliala avatar tomdracz avatar tricknotes avatar yhirano55 avatar ytbryan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

shakapacker's Issues

Easier granular opt-out of defaults

Ruby version: 3.1.1
Rails version: 7.0.2
Webpacker version: 6.1.1

Desired behavior:
I want to tweak the style-loader options 1 but otherwise opt-in to the excellent webpack defaults provided.

Actual behavior:
As far as I can tell it's a little harder to do this now that loaders/plugins etc aren't named. I'm currently digging through the list of rules and matching on the "test" values, then mutation them, which feels a little fragile (esp b/c webpack is so flexible in how it lets you define configuration.)

Some ideas:

  • Fancier moduleExists: right now this function is used pretty heavily to determine what defaults to use. Instead of just checking for the existence of a module, it could tie in with some config and always return false if the user wants to opt-out of certain defaults.
  • More use of getCustomConfig: swc and esbuild integrations allow the user to add a specially named config file within their app that can override defaults (littering the config folder with js snippets doesn't seem great though, see below.)

How does the user specify the config?

  • Add more options to webpacker.yml: this is done right now for a few options (inline_css, webpack_loader) but maybe better to keep this file mostly about things ruby and webpack both need to know about.
  • Have shakapacker export a function that takes some user config options and returns a default webpack config with those applied.

Alternatively:

  • Document some specific recipes using webpack-merge to accomplish this, to the extent it is supported.
  • Add some names or other metadata to defaults for easier manipulation (to the extent webpack supports doing so.)

Footnotes

  1. Specifically I want to add a nonce so that its HMR functionality doesn't create a bunch of spurious CSP violations.

Default configurations do not work for custom environments

I was recently updating dependencies in one of my projects and ran into a couple of problems with shakapacker, mostly because of introduction of breaking changes without explicit notice. For example, in v6.3.0-rc.1 webpacker_precompile setting was added, right now it says that this is similar to ENV["WEBPACKER_PRECOMPILE"], yet unless webpacker_precompile is set to true compile command won't be ran.

Another example is v6.3.0, it introduced new config compiler_strategy, it is marked as improvement and there is no mention in CHANGELOG that this improvement is in fact a breaking change, and you have to update your configs because there is no default option.

I think it would be great if there were default option for new settings, or at least breaking changes were marked as such and there was an info about migration.

Run bundle install automatically after releasing a new. gem version

As it stands, running release task https://github.com/shakacode/shakapacker/blob/master/rakelib/release.rake doesn't do anything to update local Gemfile.lock file with new version.

Missing Gemfile.lock update means that the tests fail (as we're running bundle install with deployment flag https://github.com/shakacode/shakapacker/runs/6512746968?check_suite_focus=true) and we need to run bundle install and commit result after each release to make sure everything works as expected.

We should modify or create a new rake task that commits the new version change and runs bundle install after gem is released to streamline this process.

cc @justin808

New Rails 7 project errors with undefined method `deep_symbolize_keys'

I am trying to set up a new Rails 7 app to compile JSX for React. Unfortunately Rails 7 totally changed how things work, so I found this project which looks like it could help. However, I followed the instructions on the readme, but get this

Ruby version: 3.1.0
Rails version: 7.0.1
Webpacker version: shakapacker 6.4

Expected behavior: rails s starts up

Actual behavior: errors
"/Users/me/.rbenv/versions/3.1.0/lib/ruby/gems/3.1.0/gems/shakapacker-6.4.0/lib/webpacker/configuration.rb:110:in load': undefined method deep_symbolize_keys' for nil:NilClass (NoMethodError)"

Small, reproducible repo: Make a new rails 7 app, install this project, and it gives this error.

Receiving WebSocket connect to 'wss://hostname:3035/ws' failed

During deployments to dev/production servers, everything works perfectly fine with websockets. Now, I am trying to use shakapacker on Docker and I receive "Unable to process ChannelName#action" Is there anything specific that I have to do, in order to communicate with web sockets using this version? I'm running Redis on the same container as the rails app and I am able to connect to it using 127.0.0.1. Every time that I use docker, I see [webpack-dev-server] Trying to reconnect.....? WebSocketClient @ WebSocketClient.js.

Screen Shot 2022-02-04 at 7 40 03 PM

swc transpilation not working

Ruby version: 3.1.0
Rails version: 7.0.2
Webpacker version: 6.2.0
swc: 1.2.161

Expected behavior:

in config/swc.config.js I've attempted to configure a browserlist:

const swcConfig = {
  options: {
    env: {
      targets: '> 0.5%, last 2 versions, Firefox ESR, not dead'
    }
  }
}

Since this matches IE 11 among other older browsers, I would expect template literals to be transpiled into legacy ES5 concat statements but they are still present in the pack output.

Actual behavior:

swc does not seem to be correctly transforming the output.

Is this expected to work? It looks like template literals are correctly transformed on https://swc.rs/playground

append_javascript_pack_tag inserts (debugging?) text into the DOM

A bug is a crash or incorrect behavior. If you have a debugging or troubleshooting question, please open a discussion on Discussions Tab.

Ruby version: 2.7.6
Rails version: (6.1.6)
Webpacker version: 6.3.0-rc.1

Expected behavior:

In app/views/layouts/hello_world.html.erb, we have:
<%= append_javascript_pack_tag 'application' %>
<%= javascript_pack_tag 'hello-world-bundle' %>

This should simply load the packs

Actual behavior:

This seems to work, but also inserts ["application"] into the DOM, where it's visible on the top of the page.

Small, reproducible repo:
https://github.com/sschafercdd/react_on_rails_demo_ssr_hmr/tree/demobug

Remove addition of node_modules to Rails.application.config.assets.paths

This change added to v6 is probably no longer necessary. Instead, we can probably document why one would do this.

rails/webpacker@47b1e06

if (asset_config_path = Rails.root.join("config/initializers/assets.rb")).exist?
  say "Add node_modules to the asset load path"
  append_to_file asset_config_path, <<-RUBY
# Add node_modules folder to the asset load path.
Rails.application.config.assets.paths << Rails.root.join("node_modules")
RUBY
end

Option to not use style-loader

Ruby version: 2.7.4
Rails version: ~> 5.2.6
Webpacker version: 6.1.1

Desired behavior:

mini-css-extract-plugin is capable of HMR now. I'd like the ability to opt into just using it all the time, even when HMR is set in webpacker.yml. I'd like to get style-loader out of the dev pipeline so that my development and production environments more closely match, while retaining HMR. It turns out this isn't too hard!

Actual behavior:

Currently, hmr: true in webpacker.yml sets Webpack.inlining_css?, which prevents stylesheet_pack_tag form emitting <link> tags for CSS, on the assumption that style-loader is in play. I monkey patched my install with:

class Webpacker::Instance
  def inlining_css?
    false
  end
end

And my Webpack css loader with:

// Swap all style-loader rules for mini-css-extract-plugin, even if HMR. MCEP supports HMR now.
const MiniCssExtractPlugin = require("mini-css-extract-plugin");
webpackConfig.module.rules.forEach((rule) => {
  if ("test.css".match(rule.test) || "test.scss".match(rule.test)) {
    rule.use = rule.use.map((loader) => {
      if (loader === "style-loader") {
        return MiniCssExtractPlugin.loader;
      } else {
        return loader;
      }
    });
  }
});

and it works just dandy.

The fix should be as simple as allowing the explicit disabling of inlining css (perhaps inlineCss: false in webpacker.yml), then updating the Webpack::Instance#inlining_css? method and its partner in shakapacker/package/inliningCss.js to respect the flag if it's set.

[Feature request] all config in js, remove config/webpacker.yml

Currently some webpacker config located in config/webpacker.yml and some in config/webpack/*.js

This leads to a problem: when I build assets. I need both nodejs and ruby environment.

This troubled me when I have to build assets in docker.

So can we move all webpacker config to config/webpack/*.js ?

Two advantages:

  • Simplify the build environment.
  • Make webpacker more thin, closer to webpack.

yarn 2 + pnp modules?

Ruby version: 3.0
Rails version: 6.1
Webpacker version: 6.0.2

Desired behavior: yarn 2 + pnp modules.

Actual behavior: I'm not clear on whether this is supposed to work out of the box or not. There was a webpacker issue that was closed before migrating to this repository.

Setting different URL for pack tags

I'm migrating from Webpacker 5 to Shakapacker and option development.dev_server.public: localhost:3035 in config/webpacker.yaml is no longer working.

Since we use separate Docker container to run webpacker-dev-server, I need the pack tags URLs to contain the different port.

I also tried setting dev_server.devMiddleware.publicPath but that doesn't have any effect either.

According to the getPublicPath function,

// Ensure that the publicPath includes our asset host so dynamic imports

I tried setting WEBPACKER_ASSET_HOST environment variable, which worked, but doesn't feel like a proper solution.

Suggestions for the name of this project

Warning, the final name of this fork and evolution of rails/webpacker is still pending waiting on community feedback.

What name would make sense so the name is not confusing with old rails/webpacker?

Since the old repository will not be transferred, we need a new name.

Don't enhance precompile with yarn install

Running yarn:install during asset:precompile might be bit too magical. In the same way as we don't run bundle install, we shouldn't run yarn install.

yarn:install being ran adds some time to the compile. On Heroku also, by default it will reinstall all packages even if they have been previously cached. Not good.

Prepping of Shakapacker 7 might be good time to get rid of this behaviour:

TODO:

Shakapacker conflicts with vue-loader

Ruby version: ruby 2.6.6p146 (2020-03-31) [x86_64-darwin17]
Rails version: Rails 6.1.4.1
webpack: 5.70.0
webpack-cli: 4.9.2
webpack-dev-server 4.7.4

Expected behavior

Vue files are correctly handled when using the vue-loader.

Actual behavior

The "raw" rule incorrectly tricks the vue-loader's foo.html.vue check (https://github.com/vuejs/vue-loader/blob/7123a37ff68eaaebec515b029fb602cb4d8cf498/lib/plugin-webpack5.js#L76-L80) causing the vue-loader to believe that it is misconfigured.

The easiest work-around is to add to the exclude configuration of the "raw" rule as follows:

/\.(js|mjs|jsx|ts|tsx|vue\.html)$/

export XXX was not found in 'xxx' (module has no exports)

After upgrading from webpacker 5.x to shakapacker 6.3.0 I'm getting such an error. It happens for single package only, all other components/node_modules resolved/compiled correctly. Before the upgrade the same node module used to be working properly.

Previous versions:
ruby: 2.5.5
node: 14.x
webpacker: 5.4.3
webpack: 4.44.2

Current versions:
ruby: 2.7.6
node: 16.x
shakapacker: 6.3.0
webpack: 5.72.1

Default babel preset for shakapacker produces incorrect behavior by running in "loose" mode

Hi Shakapacker,

The default babel preset includes a non-default option, "loose" mode, which changes the behavior of some javascript statements so that they defer from their native behaviors.

This can be really surprising, because the code in your bundle is doing something different from what you see when you run the same code in the Dev Console.

Ruby version:

ruby -v

ruby 2.6.2p47 (2019-03-13 revision 67232) [x86_64-darwin21]`

Rails version:

./bin/rails -v

Rails 6.0.4.7

Webpacker version:

./bin/webpacker -v

webpack: 5.72.0
webpack-cli: 4.9.2
webpack-dev-server 4.8.1

Expected behavior:

The code in the bundle should be consistent with the native behavior. For example, to get a unique array of values out of an array with duplicates, you can spread over a Set:

const dups = [1, 1, 2, 2, 3, 3];
const uniq = [...new Set(dups)];

console.log(uniq) // [1, 2, 3]

The result is an array of unique values from the original input.

Actual behavior:

In "loose" mode, the compiled output changes the logic slightly, and the raw source looks like this:

[].concat(new Set(dups))

Which produces a different result:

console.log(uniq) // [new Set([1, 2, 3])]

The array never iterates over the Set, the spread operation was ignored. Turning off "loose" mode makes it produce the correct result.

Small, reproducible repo:

  1. Go to this Babel REPL
  2. Open DevTools Console to observe the output
  3. Toggle "Loose" on and off to see the change it produces in the compiled output and in the console.log output

Alternatively:

  1. Follow the Installation steps in the README
  2. Copy the code snippet above into app/javascript/application.js and run the app.

The culprit is this line in the preset: https://github.com/shakacode/shakapacker/blob/master/package/babel/preset.js#L28

Workaround:

You can manually fix the babel config by removing the babel preset from package.json and creating a new babel.config.js to toggle "loose" mode off:

// babel.config.js
module.exports = function (api) {
  // fetch the default babel config
  const defaultConfigFunc = require("shakapacker/package/babel/preset.js");
  const resultConfig = defaultConfigFunc(api);

  // "@babel/preset-env" is always the 1st preset
  const [[, options]] = resultConfig.presets;
  options.loose = false;

  // you can also make any other babel config changes that you need here
  return resultConfig;
};

running the react_on_rails or shakapacker install along with build:css (cssbuilding) under Rails 7

Ruby version: 3.1.2
Rails version: 7.0.3
Webpacker version: n/a

steps to reproduce:

rails new HelloShakapackerRails7H --skip-javascript --css=bootstrap --database=postgresql
• generate welcome controller & add bootstrap layout
• start server with ./bin/dev, confirm that bootstrap CSS is working but with non-working JS
bundle add react_on_rails --strict
rails generate react_on_rails:install
this seems to wipe away my CSS watcher and sass watcher in Procfile.dev

https://github.com/jasonfb/HelloShakapackerRails7H/commit/7aee0142b05a41b9717941d66894056af6556f7c

If I run ./bin/dev in this state, I get:

% ./bin/dev
18:59:18 rails.1     | started with pid 17550
18:59:18 wp-client.1 | started with pid 17551
18:59:18 wp-server.1 | started with pid 17552
18:59:18 wp-client.1 | sh: bin/webpacker-dev-server: No such file or directory
18:59:18 wp-server.1 | sh: bin/webpacker: No such file or directory
18:59:18 wp-server.1 | exited with code 127
18:59:18 system      | sending SIGTERM to all processes
18:59:18 wp-client.1 | exited with code 127
18:59:18 rails.1     | terminated by SIGTERM

(admittedly, in its current state I have not yet run or even added shakapacker, so it makes sense that webpacker is unavailable).

see app at:
https://github.com/jasonfb/HelloShakapackerRails7H

The example app @justin808 pointed me to is for Rails 6 and I think what I am failing to grasp is how to build a react_on_rails app using the new ./bin/dev tools in Rails 7 (specifically, the sass watcher + setting the "build" script to be the bin/webpacker-dev-server so they can be started simultaneously using ./bin/dev the new Rails 7 way)

the installers seem to be overwriting my Procfile.dev. should I put the sass watcher back into the Procfile.dev?

thank you!

V7 Items

Breaking Changes Requiring a Major Version Bump

  1. Change references from webpacker to shakapacker #157
  2. Change the configuration so that the configuration runs with a function invocation of a passed configuration object rather than using globals.

Other Suggestions (not required for v7)

  1. Change to use rspec for new tests rather than minitest
  2. Support a dummy test app for better tests

ENHANCEMENT: Support or switch to SWC

SWC might dramatically speed up build times.

SWC is 20x faster than Babel on a single thread and 70x faster on four cores.

Does anybody want to try a PR that swaps Babel for SWC?

SyntaxError: Invalid regular expression

I added a new component file in my project at app/javascript/components/xxx.js after compiling the project with rbenv exec bundle exec rake assets:clean assets:precompile command I got the error as below

ERROR -- : SyntaxError: Invalid regular expression: /^(https?://)?((([a-zd]([a-zd-]*[a-zd])*).)+[a-z]{2,}|((d{1,3}.){3}d{1,3}))(:d+)?(/[-a-zd%_.~+]*)*(?[&a-zd%_.~+=-]*)?(#[-a-zd_]*)?$/: Invalid group
    at new RegExp (<anonymous>)
    at /home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:412:24
    at Hook.eval [as call] (eval at create (/home/myusername/myproject/node_modules/tapable/lib/HookCodeFactory.js:19:10), <anonymous>:7:16)
    at JavascriptParser.evaluateExpression (/home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:3193:25)
    at JavascriptParser.getRenameIdentifier (/home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:1398:23)
    at JavascriptParser.walkVariableDeclaration (/home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:2090:31)
    at JavascriptParser.walkStatement (/home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:1598:10)
    at JavascriptParser.walkStatements (/home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:1459:9)
    at /home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:1633:9
    at JavascriptParser.inBlockScope (/home/myusername/myproject/node_modules/webpack/lib/javascript/JavascriptParser.js:3091:3)

After checked and verified I found that the error is coming from import Dante from "Dante2" that containing in component file

import React from "react";
import ReactDOM from "react-dom";
import Dante from "Dante2";

....

Can anyone advise how can I fix this?

Ruby version: 3.1.2
Rails version: 7.0.3
Shakapacker version: 6.3

assets:clean / webpacker:clean task deletes the latest version of files when asset_host is set

Ruby version: 2.7.6
Rails version: 6.1.6
Webpacker version: 6.4.1

Expected behavior:

As documented by a comment in the implementation file itself, running the webpacker:clean task (or the enhanced assets:clean task) should always keep the latest version of compiled assets as indicated by manifest.json.

There is no expectation that setting config.asset_host (or the longer name of config.action_controller.asset_host) in production.rb should change this behaviour.

Actual behavior:

If a host is set for config.asset_host, then the webpacker:clean task will sometimes remove even the latest version of assets, leading to 404s for those assets. I think this is down to the age of the asset file.

Steps to reproduce:

  1. Set config.asset_host to any host in production.rb, eg config.asset_host = example.com
  2. Run RAILS_ENV=production bundle exec rails assets:precompile
  3. Wait 1 second
  4. Run RAILS_ENV=production bundle exec rake webpacker:clean[2,1]
  5. See that all your new assets just got removed!

(To save you from looking, the '1' argument in the command sets the maximum age of assets that are no longer considered the "latest" version to 1 second.)

Cause:

Setting config.asset_host causes full URLs to be inserted into the manifest.json, eg:

{
  // ... other assets ...
  "application.js": "//example.com/packs/js/application-50abb05db229c31c1e24.js",
  // ... other assets ...
}

which is already documented in the troubleshooting docs relating to another issue.

I couldn't locate the PR/motivation for this change to manifest.json, but as a reminder, prior to shakapacker/webpacker v6 manifest.json did not include the asset_host.

Unfortunately the implementation of webpacker:clean was not updated to handle this, so it is expecting to get local file paths but getting remote URLs from manifest.json. Since none of the local assets paths match the remote URLs, this in turns means that no local assets are kept due to being the current version, and they are only ever kept if they meet the criteria for keeping a backup.

def current_version
packs = manifest.refresh.values.map do |value|
value = value["src"] if value.is_a?(Hash)
next unless value.is_a?(String)
File.join(config.root_path, "public", "#{value}*")
end.compact
Dir.glob(packs).uniq
end

In particular, on line 74 value is a URL so sticking this on the end of the project root path produces a half-local-path half-URL string.

Workaround:

Like the linked troubleshooting issue, setting the WEBPACKER_ASSET_HOST env variable to an empty string causes the manifest.json to revert to having relative paths, so the webpacker:clean task then works correctly.

Thoughts on priority:

This is quite a nasty issue as things might work absolutely fine when initially upgrading to shakapacker, but then start failing in weird ways on subsequent builds.

You can get very strange behaviours since all of the logic is based off of the file age, so in my case running assets:clean removed a foo.js file while leaving the equivalent foo.js.gz file intact, causing much head scratching due things working in some browsers and not others.

How do I set up HMR for a React project using Shakapacker?

Hi,

I'm having trouble setting up a React project with HMR.

I tried following the steps outlined here: https://github.com/shakacode/shakapacker/blob/master/docs/customizing_babel_config.md

Whenever I make a change to a CSS file, HMR happens as expected.

However, when I edit a React component, I see the following error message:

[HMR] Cannot apply update. Need to do a full reload! 
[HMR] Aborted because ./app/javascript/application.js is not accepted
Update propagation: ./app/javascript/application.js
...

What am I doing wrong?

Is it possible to set things up so that when I edit a React component, the changes are reflected on the page without a full refresh and that no warning is shown in the console? I'm using Rails 7 and React 18 (although I was getting the same error under React 17)

Grateful for any help.

Steps to reproduce
  1. Create a new Rails app:
rails new myapp --skip-javascript
cd myapp
bundle add shakapacker --strict
./bin/bundle install
./bin/rails webpacker:install
yarn add react react-dom @babel/preset-react
yarn add css-loader style-loader mini-css-extract-plugin css-minimizer-webpack-plugin
yarn add --dev @pmmmwh/react-refresh-webpack-plugin react-refresh
  1. Generate controller
rails g controller site index
echo '<div id="root"></div>' > app/views/site/index.html.erb
  1. Create CSS file:
touch app/javascript/App.css
  1. app/javascript/application.js:
import React from 'react';
import { createRoot } from 'react-dom/client';
import './App.css';

const container = document.getElementById('root');
const root = createRoot(container);

document.addEventListener('DOMContentLoaded', () => {
  root.render(<p>Hello, World!!</p>);
});
  1. app/javascript/App.css:
p { color: blue; }
  1. Enable HMR in config/webpacker.yml:
hmr: true
  1. Remove the Babel configuration from package.json
  2. Create a babel.config.js file in the root of project and add the sample config.
  3. Start servers:
rails s
./bin/webpacker-dev-server
  1. Hit: http://localhost:3000/site/index
  2. Edit React component in application.js and observe the error in the browser console.

Incorrect reference to the next release tag

We're referencing shakapacker@next in
https://github.com/shakacode/shakapacker/blob/master/README.md?plain=1#L612
and https://github.com/shakacode/shakapacker/blob/master/lib/install/template.rb#L63-L64

However, there is no next tag in the NPM registry, we've got rc instead. Compare shakapacker vs rails/webpacker.

Yarn v1 behaves fine and just asks to pick a different release instead. Yarn v3 fails miserably though and refuses to install anything.

Looks like we're using release-it and not supplying the tag name so tag guessing game happens at https://github.com/release-it/release-it/blob/master/lib/plugin/npm/npm.js#L202-L210 and just grabs first tag that is not latest.

Are we able to retag the RC releases with next so everything falls into place @justin808? If not, then we can settle on rc and update the code.

Intermittent Webpacker.dev_server.running? behavior

This bug report was originally filed against rails/webpacker #3269. I'm opening it here even though I've only reproduced this bug with rails/webpacker because I see the problematic default timeout value is identical here as it was in rails/webpacker.

Ruby version: 2.7.3p183
Rails version: 6.0.4.6
Webpacker version: 4.1.0

Expected behavior: Webpacker.dev_server.running? should return consistent answers

Actual behavior: Webpacker.dev_server.running? returns true and false intermittently

As of rails/webpacker#753

Webpacker uses by default a socket connect timeout of 1/100th of a second. I have seen situations developing locally where this is too fast and I get intermittent return values. Thankfully, the value is configurable, but going back to the PR where the change was made, I don't see any conversation specifically around the change from 1 full second to 1/100th of a second. It isn't clear that there was a reason for the change to the default value and basically I think the default value should be longer. Happy to put in a PR, but I'm putting in this issue first to see if there are any objections.

Followup: Actually, I do see a blip of a conversation inline in that PR now, but it seems to be in reference to "slower systems" and I have seen this on a few modern systems now, one of them being a brand new M1 macbook.

SWC Config Output is Strange, Wrong-Seeming

Was playing around with the swc option (neat!) and encountered this. Happy to take a pass at fixing.

Ruby version: 3.1.1
Rails version: 7.0.2.2
Webpacker version: 6.1.1

Expected behavior:

The following, rather unambitious Javascript, when compiled with webpack_loader: "swc" and no further customization to the web shakapacker config, should compile to Javascript that works.

Input Javascript

class B {}

class Thing extends B {
  constructor(n) {
    super()
    this.name = n
  }
}

const t = new Thing("name")

Actual behavior:

It compiles to to something very strange that doesn't polyfill classes correctly.

Output Javascript

// ... elided polyfill stuff

var B = function B() {
    "use strict";
};
var Thing = /*#__PURE__*/ function(B) {
    "use strict";
    _inherits(Thing, B);
    function Thing(n) {
        var _this;
        _this = B.call(this); // "_this" is always going to be undefined here
        _this.name = n; // so this line will always be an error
        return _this;
    }
    return Thing;
}(B);

var t = new Thing("name");

The swc config output looks like the below. The stuff under env looks like maybe it was copied directly from the Babel config. This is exacerbated by the fact that shakapacker merges its default swc config with the custom one you supply.

Generated SWC Config

{
  "jsc": {
    "parser": {
      "syntax": "ecmascript",
      "jsx": false
    }
  },
  "sourceMaps": true,
  "env": {
    "coreJs": "3.8",
    "loose": true,
    "exclude": [
      "transform-typeof-symbol"
    ],
    "mode": "entry"
  }
}

Small, reproducible repo:

No repo yet, but to reproduce:

  • Go here https://swc.rs/playground.
  • Paste the "Input Javascript" above into the "Input" field.
  • Click on the "Edit as JSON" button.
  • Add a console.log somewhere in your shakapacker config to output your webpack config. Grab the SWC part (or just grab the it from the above) and paste the results in.

Consider relaxing the version check and making it more consistent

It's great to see that we've got ensure_consistent_versioning but I feel like it's missing the mark a bit and has inconsistent behaviour.

Sample of current output for reference:

**ERROR** Webpacker: Your node package version for shakapacker contains a ^ or ~
Detected: ^6.4.1
     gem: 6.4.1
Ensure the installed version of the gem is the same as the version of
your installed node package. Do not use >= or ~> in your Gemfile for shakapacker.
Do not use ^ or ~ in your package.json for shakapacker.
Run `yarn add shakapacker --exact` in the directory containing folder node_modules.
  1. it does not enforce the use of an exact version in the Gemfile, but does in package.json
  2. it's message says "Do not use >= or ~> in your Gemfile for shakapacker.", which imo is silly because you definitely should be using at least a major version constraint (e.g. ~> 6.0)
    • if 1. was fixed the message would become more accurate, but if 1. is fixed by enforcing an exact version then the message should say "any constraint" rather than just two types of constraints
  3. the fact that we have ensure_consistent_versioning should mean we can use non-exact versions since we'll be told if the gem and npm package do not match

It's really that last point that I want to talk about: generally it's considered bad practice to depend on an exact version as it makes it harder to do upgrades (especially for security patches) and it's generally a smell of a brittle setup if patch versions (which are meant to be the most non-breaking of changes) matter.

I totally get why it's important to ensure the gem and package are on the same version for this library and that's why I think having ensure_consistent_versioning is the way to go because it should actually allow us to avoid needing to use exact constraints as if something results in the gem being updated, we'll get told we have to update the package, and vice versa.

Obscure errors when peer dependency warnings are not respected

We had obscure errors in the browser:

Failed to load resource: net::ERR_CONTENT_DECODING_FAILED

This was caused by a very old version of the peer dependency webpack-compression-plugin.

We could have prevented this by checking for unmet peer dependency warnings when running yarn.

How can we make the use of peer dependencies less hazardous?

I added a note to the Troubleshooting and Upgrade documents:

Bundler does not load webpacker.rb or the rest of the gem

A bug is a crash or incorrect behavior. If you have a debugging or troubleshooting question, please open a discussion on the Rails forum, category webpacker

Ruby version: 3.0.3
Rails version: 6.1.x
Webpacker version: shakapacker 6.0.0.rc.12

Expected behavior:

Having gem "shakapacker", "6.0.0.rc.12" in the Gemfile properly loads up the new gem

Actual behavior:

The gemspec file is called shakapacker.gemspec and expects a shakeapcker.rb file in lib/ to load up. But the file in lib/ is still webpacker.rb so nothing in the gem appears to be loaded up.

Small, reproducible repo:

On an existing Rails app on Rail's webpacker 6 rc6 gem, change the gem in the Gemfile to gem "shakapacker", "6.0.0.rc.12" and go through any minor changes to work with rc12. Running the app will result in no method javascript_pack_tag error because nothing actually is bootstrapped.

A workaround that appears to be fine locally is to change the entry in the Gemfile to

gem "shakapacker", "6.0.0.rc.12", require: "webpacker"

Which properly has bundler require the webpacker.rb file and the rest of the gem.

Attempt D : With jsbundling **WIP**

A bug is a crash or incorrect behavior. If you have a debugging or troubleshooting question, please open a discussion on Discussions Tab.

Ruby version: 3.1.2
Rails version: 7.0.3
Webpacker version: 6.4.1

So in my last attempt (https://github.com/jasonfb/HelloShakapakerRails7C) I mistakenly started with an importmap-rails app. In this attempt, I started with a jsbundling/cssbundling app but got stuck even faster --- in particular, the webpacker installer overwrites the jsbundling config which enables the ./bin/dev rails development environment

In this attempt, which you can find here https://github.com/jasonfb/HelloShakapackerRails7D, I started with rails new HelloShakapackerRails7D --javascript=webpack --css=bootstrap

When I ran rails webpacker:install, the package.json file removes these packages

  • "@hotwired/stimulus": "^3.0.1",
  • "@hotwired/turbo-rails": "^7.1.3",
  • "@popperjs/core": "^2.11.5",
  • "bootstrap": "^5.1.3",
  • "bootstrap-icons": "^1.8.3",
  • "sass": "^1.52.3",

and also the scripts for the dev environment are removed too:


-  "scripts": {
-    "build": "webpack --config webpack.config.js",
-    "build:css": "sass ./app/assets/stylesheets/application.bootstrap.scss ./app/assets/builds/application.css --no-source-map --load-path=node_modules"
-  }

I added these back manually using yarn add @hotwired/stimulus @hotwired/turbo-rails @popperjs/core bootstrap bootstrap-icons sass and adding the script tags back by hand

Small, reproducible repo:
https://github.com/jasonfb/HelloShakapackerRails7D

Webpacker can't find subdir/image.png in manifest.json

Ruby version: 3.03
Rails version: 7.0.1
Webpacker version: 6.0.2 (same for js and gem)

Expected behavior:

  • put an image file into /app/javascript/images/subdir/image.png
  • use image_pack_tag "subdir/image.png" in view
  • image should be found by webpacker and rendered

Actual behavior:
Error is raised: Webpacker can't find subdir/image.png in .../manifest.json

Small, reproducible repo:
https://github.com/honzasterba/packer-test

What should I do about `Module not found: Error: Can't resolve '@hotwired/stimulus-loading'`

I realize it is mixing separate paradigms but I have some sections of my app using Stimulus and want to mix Rails 7 stimulus + Turbo defaults with react_on_rails (not in the same sections of the app which I realize would be a little crazy)... I just happen to want both in the same app.

my application.js file is like so:

// this was the default Rails setup
// Configure your import map in config/importmap.rb. Read more: https://github.com/rails/importmap-rails
import "@hotwired/turbo-rails"
import "controllers"


// added from Shakapacker docs with modifications for my React component 'Logo'
import React from 'react';
import {createRoot } from 'react-dom/client';
import Logo from './components/basic_elements/logo/Logo';

const container = document.getElementById('root');
const root = createRoot(container);

document.addEventListener('DOMContentLoaded', () => {
  root.render(<Logo name="World" />);
});

In addition to the standard install instructions, I had to add @hotwired/turbo-rails, @hotwired/stimulus, and styled-components using yarn

However, i seem to be stuck on this one:
when I run bin/webpacker-dev-server

ERROR in ./app/javascript/controllers/index.js 4:0-70
Module not found: Error: Can't resolve '@hotwired/stimulus-loading' in '/Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/controllers'

this SO post says that this node package ('@hotwired/stimulus-loading') was built for importmap, and I should try @hotwired/stimulus-webpack-helpers. I attempted to add this with yarn add @hotwired/stimulus-webpack-helpers but it does not solve the issue.

FWIW, my intention for this app was not to use importmap but I will use it in parallel with shakapacker assuming that not a terrible idea(?) and the path of least resistance. what do you think?

Ruby version: 3.1.2
Node Version: 16.13.1
Rails version: 7.0.3
Webpacker version: 6.3 via 6.2.1 upgrade using workaround from ISSUE#123

FULL ERROR:

% bin/webpacker-dev-server                   
<i> [webpack-dev-server] Project is running at:
<i> [webpack-dev-server] Loopback: http://localhost:3035/, http://127.0.0.1:3035/
<i> [webpack-dev-server] Content not from webpack is served from '/Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/public' directory
<i> [webpack-dev-server] 404s will fallback to '/index.html'
assets by path static/ 22.3 KiB
  asset static/Refactor-spinner-0_290x290-bc16e4b5dbd527ac51d0.png 17.4 KiB [emitted] [immutable] [from: app/javascript/images/Refactor-spinner-0_290x290.png] (auxiliary name: application)
  asset static/120-deg-arrow1-1cba889ed3cfce3ce26b.svg 1.49 KiB [emitted] [immutable] [from: app/javascript/images/120-deg-arrow1.svg] (auxiliary name: application)
  asset static/120-deg-arrow2-b6402b9ca3a35e39af6e.svg 1.4 KiB [emitted] [immutable] [from: app/javascript/images/120-deg-arrow2.svg] (auxiliary name: application)
  asset static/right-arrow-f71b9a76417162cf96db.svg 1.03 KiB [emitted] [immutable] [from: app/javascript/images/right-arrow.svg] (auxiliary name: application)
  asset static/120-deg-arrow3-19ff131865d761915c9f.svg 980 bytes [emitted] [immutable] [from: app/javascript/images/120-deg-arrow3.svg] (auxiliary name: application)
assets by path js/*.js 1.56 MiB
  assets by chunk 1.52 MiB (id hint: vendors)
    asset js/vendors-node_modules_hotwired_stimulus_dist_stimulus_js-node_modules_hotwired_turbo-rails_app-4383ca.js 1.49 MiB [emitted] (id hint: vendors) 1 related asset
    asset js/actioncable.chunk.js 29.5 KiB [emitted] (name: actioncable) (id hint: vendors) 1 related asset
  asset js/application.js 23.1 KiB [emitted] (name: application) 1 related asset
  asset js/runtime.js 12.5 KiB [emitted] (name: runtime) 1 related asset
asset manifest.json 1.51 KiB [emitted]

ERROR in ./app/javascript/controllers/index.js 4:0-70
**Module not found: Error: Can't resolve '@hotwired/stimulus-loading' in** '/Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/controllers'
resolve '@hotwired/stimulus-loading' in '/Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/controllers'
  Parsed request is a module
  using description file: /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/package.json (relative path: ./app/javascript/controllers)
    Field 'browser' doesn't contain a valid alias configuration
    resolve as module
      looking for modules in /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript
        single file module
          using description file: /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/package.json (relative path: ./app/javascript/@hotwired/stimulus-loading)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading.js doesn't exist
            .jsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading.jsx doesn't exist
            .mjs
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading.mjs doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading.ts doesn't exist
            .tsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading.tsx doesn't exist
            .coffee
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading.coffee doesn't exist
        /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/@hotwired/stimulus-loading doesn't exist
      /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/controllers/node_modules doesn't exist or is not a directory
      /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/javascript/node_modules doesn't exist or is not a directory
      /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/app/node_modules doesn't exist or is not a directory
      looking for modules in /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules
        single file module
          using description file: /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/package.json (relative path: ./node_modules/@hotwired/stimulus-loading)
            no extension
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading doesn't exist
            .js
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading.js doesn't exist
            .jsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading.jsx doesn't exist
            .mjs
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading.mjs doesn't exist
            .ts
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading.ts doesn't exist
            .tsx
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading.tsx doesn't exist
            .coffee
              Field 'browser' doesn't contain a valid alias configuration
              /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading.coffee doesn't exist
        /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapackerRails7C/node_modules/@hotwired/stimulus-loading doesn't exist
      /Users/jason/Work/_LEARNING/ReactOnRails/node_modules doesn't exist or is not a directory
      /Users/jason/Work/_LEARNING/node_modules doesn't exist or is not a directory
      /Users/jason/Work/node_modules doesn't exist or is not a directory
      /Users/jason/node_modules doesn't exist or is not a directory
      /Users/node_modules doesn't exist or is not a directory
      /node_modules doesn't exist or is not a directory

webpack 5.72.1 compiled with 1 error in 593 ms

This is reproduced here:

https://github.com/jasonfb/HelloShakapakerRails7C

Caching issues in Development since migrating to Shakapacker

We used to run a pretty standard Webpacker 5 setup, which in Development resulted in HTML like this:

<head>
  <link rel="stylesheet" media="all" href="/packs/css/application-e793e27b.css" data-turbo-track="reload" />
  <script src="/packs/js/application-57eff0528424c89ab096.js" data-turbo-track="reload" defer="defer"></script>
  <!-- etc. -->

We then migrated to (equally plain/default) Shakapacker 6 setup, which in Development now results in HTML like this:

<head>
  <link rel="stylesheet" media="all" href="/packs/css/application.css" data-turbo-track="reload" />
  <script src="/packs/js/runtime.js" data-turbo-track="reload" defer="defer"></script>
  <script src="/packs/js/vendors-node_modules_webpack-dev-server_client_index_js_protocol_ws_3A_hostname_localhost_por-54b5bb.js" data-turbo-track="reload" defer="defer"></script>
  <script src="/packs/js/vendors-node_modules_babel_runtime_regenerator_index_js-node_modules_hotwired_turbo-rails_app-56e6f5.js" data-turbo-track="reload" defer="defer"></script>
  <script src="/packs/js/application.js" data-turbo-track="reload" defer="defer"></script>
  <!-- etc. -->

We expected SplitChunks to be enabled here, however the contenthash has also been removed from some filenames, like runtime.js and most problematically application.js/application.css.

This is resulting in caching issues where the e.g. application.js file gets updated by Shakapacker during development, but the browser serves a cached version even after page refresh since the filename hasn't changed.

I can fix this with the following config/webpack/webpack.config.js, but it adds the missing hash only to the JS file, not also the CSS:

const { webpackConfig, merge } = require('shakapacker')

const customConfig = {
  // Add contenthash to filenames in development to prevent browser serving cached files
  output: {
    filename: '[name].[contenthash].js'
  }
}

module.exports = merge(webpackConfig, customConfig)

Is this an expected change in Shakapacker? It doesn't seem a good default?

Edit: Looks like this might be a performance thing? Still, this is unexpected when migrating from Webpacker and probably not a good default.

Rails 7.0.3 - Webpacker configuration file not found when running `rails webpacker:install` (shakapacker v6.3)

Ruby version: 3.1.2
Rails version: 7.0.3
Shakapacker version: 6.3.0

Steps to reproduce:
rails new HelloShakapaker1 --skip-javascript --database=postgresql
bundle add shakapacker --strict
rails webpacker:install

for some reason I seem to have something missed here, this task tells me No such file or directory config/webpacker.yml but as far as I understand this command should create this for me no? Maybe this is a Rails 7 thing or perhaps I missed something?

example app is here
https://github.com/jasonfb/HelloShakapakerRails7C

thank you!

% rails webpacker:install
The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`.
/Users/jason/.rvm/rubies/ruby-2.7.2/lib/ruby/2.7.0/net/protocol.rb:66: warning: already initialized constant Net::ProtocRetryError
/Users/jason/.rvm/gems/ruby-2.7.2/gems/net-protocol-0.1.3/lib/net/protocol.rb:68: warning: previous definition of ProtocRetryError was here
/Users/jason/.rvm/rubies/ruby-2.7.2/lib/ruby/2.7.0/net/protocol.rb:206: warning: already initialized constant Net::BufferedIO::BUFSIZE
/Users/jason/.rvm/gems/ruby-2.7.2/gems/net-protocol-0.1.3/lib/net/protocol.rb:208: warning: previous definition of BUFSIZE was here
/Users/jason/.rvm/rubies/ruby-2.7.2/lib/ruby/2.7.0/net/protocol.rb:503: warning: already initialized constant Net::NetPrivate::Socket
/Users/jason/.rvm/gems/ruby-2.7.2/gems/net-protocol-0.1.3/lib/net/protocol.rb:504: warning: previous definition of Socket was here
rails aborted!
Webpacker configuration file not found /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapakerRails7C/config/webpacker.yml. Please run rails webpacker:install Error: No such file or directory @ rb_check_realpath_internal - /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapakerRails7C/config/webpacker.yml
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:112:in `rescue in load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:101:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:98:in `data'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:93:in `fetch'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:33:in `webpacker_precompile?'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/tasks/webpacker/clean.rake:14:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `block in run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/application.rb:505:in `block in run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine/railties.rb:15:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine/railties.rb:15:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/application.rb:505:in `run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:464:in `load_tasks'
/Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapakerRails7C/Rakefile:6:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands/rake/rake_command.rb:20:in `block in perform'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands/rake/rake_command.rb:18:in `perform'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/command.rb:51:in `invoke'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands.rb:18:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
bin/rails:4:in `<main>'

Caused by:
Errno::ENOENT: No such file or directory @ rb_check_realpath_internal - /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapakerRails7C/config/webpacker.yml
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/compile_cache/yaml.rb:313:in `realpath'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/compile_cache/yaml.rb:313:in `load_file'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:105:in `rescue in load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:102:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:98:in `data'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:93:in `fetch'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:33:in `webpacker_precompile?'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/tasks/webpacker/clean.rake:14:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `block in run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/application.rb:505:in `block in run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine/railties.rb:15:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine/railties.rb:15:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/application.rb:505:in `run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:464:in `load_tasks'
/Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapakerRails7C/Rakefile:6:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands/rake/rake_command.rb:20:in `block in perform'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands/rake/rake_command.rb:18:in `perform'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/command.rb:51:in `invoke'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands.rb:18:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
bin/rails:4:in `<main>'

Caused by:
ArgumentError: unknown keyword: :aliases
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/compile_cache/yaml.rb:307:in `load_file'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:103:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:98:in `data'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:93:in `fetch'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/webpacker/configuration.rb:33:in `webpacker_precompile?'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/shakapacker-6.3.0/lib/tasks/webpacker/clean.rake:14:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `block in run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:661:in `run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/application.rb:505:in `block in run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine/railties.rb:15:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine/railties.rb:15:in `each'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/application.rb:505:in `run_tasks_blocks'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/engine.rb:464:in `load_tasks'
/Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapakerRails7C/Rakefile:6:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:39:in `load'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands/rake/rake_command.rb:20:in `block in perform'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands/rake/rake_command.rb:18:in `perform'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/command.rb:51:in `invoke'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/railties-7.0.3/lib/rails/commands.rb:18:in `<main>'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
/Users/jason/.rvm/gems/ruby-2.7.2/gems/bootsnap-1.11.1/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
bin/rails:4:in `<main>'
(See full trace by running task with --trace)
jason@MBA22 /Users/jason/Work/_LEARNING/ReactOnRails/HelloShakapakerRails7C [main]
% 

5♠︎

Easier handling of multiple pack_tag invocations

Continuing from discussions like rails/webpacker#3068

We are now raising an error when pack_tag helpers are used multiple times during the rendering. This solves hard to debug issues with double script loading, but it's fairly heavy handed approach, especially if we don't provide easy way to solve those issues. At the moment, users are required to roll out some custom logic to get all their packs in a single place.

We should look and see if we can improve the experience here in any way. Ideally calling *_pack_tag multiple times would "just" work, but there's complexities here around load order, ideal placement of runtime check etc

As a first step, maybe simply adding a view helper to register additional packs that then get added in pack_tag call is enough to improve the experience?

Suggestions definitely welcome!

Assets not built for tests in CI

When compile in Test environment is set to true, Shakapacker automatically builds to /public/packs-test.

webpacker.yml

test:
  <<: *default
  compile: true

However in CI e.g. CircleCI this does not happen, tests run without automatically building the assets.

  • Is it correct that Shakapacker does not automatically build in this CI environment, where RAILS_ENV etc. is set to test? This was slightly unexpected when I changed this setting (in place of manually calling Webpacker.compile in our test helpers).
  • If it is correct, what is the official way of building assets for tests in a CI environment? Heroku runs bundle exec rake assets:precompile in Production, is it the same command in CI? Happy to update the docs with the recommended setup here.

Can't execute Rails Code in js.erb files

I am trying to get js.erb files to work with shakapacker. But so far i had no luck. As far as i know there is no installer for erb (bundle exec webpacker:install:erb) anymore like in previous webpacker versions, so this should be supported out of the box, am i correct?
what could i be doing wrong here?

Currently i have the stock webpack.config.js

// /app/views/concept_maps/edit.html.erb

<% append_javascript_pack_tag 'test' %> 

This is my output:

Uncaught Error: Module build failed (from ./node_modules/babel-loader/lib/index.js):
SyntaxError: /home/windhowl/Dokumente/CoMapEd/app/javascript/packs/test.js.erb: Unexpected token (1:0)

> 1 | <%logger.debug "this is rails code"%>
    | ^
    at instantiate (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:72:32)
    at constructor (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:358:12)
    at Parser.raise (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:3341:19)
    at Parser.unexpected (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:3379:16)
    at Parser.parseExprAtom (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:13083:24)
    at Parser.parseExprSubscripts (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:12650:23)
    at Parser.parseUpdate (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:12629:21)
    at Parser.parseMaybeUnary (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:12600:23)
    at Parser.parseMaybeUnaryOrPrivate (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:12394:61)
    at Parser.parseExprOps (/home/windhowl/Dokumente/CoMapEd/node_modules/@babel/parser/lib/index.js:12401:23)

Ruby version: 2.7
Rails version: 7.0.2
Shakapacker version: 6.3

Allow setting webpack mode to "development" in tests

Ruby version: 3.1.2
Rails version: 7.0.3
Shakapacker version: 6.2.1

JS debugger statements are useful, including when debugging system tests. But in test environment Shakapacker sets webpack to mode: 'production' which strips debugger statements.

Have you considered providing an option in webpacker.yml to set mode to development in test environment, to avoid a custom webpack config override?

Or perhaps tests should run with webpack mode: 'development' by default? As the builds are faster, with sourcemaps etc.?

Wrong CSS url with assets in subdirectories on Windows

Ruby version: 3.1
Rails version: 7.0
Webpacker version: 6.2.1

Expected behavior: CSS urls must not contain backslashes as path delimiters when using assets from subdirectories.

Actual behavior: If for example a CSS font url is used on Windows, the final asset url contains a mixture of forward and backward slashes. Backslashes are escape characters in CSS, resulting in invalid CSS and subsequent 404 errors.

The bug was introduced with af48749

Details and fix:

In package/rules/file.js there is a Webpack hook defined to override the filename of the asset generator.

When Webpack calls this, pathData.filename contains a normalized asset path with forward slashes, even on Windows.
(The code in the filename function correctly assumes this at the replace and split calls.)

  filename: (pathData) => {              // pathData.filename contains forward slashes only, even on Windows
      const folders = dirname(pathData.filename)
        .replace(`${sourcePath}/`, '')   // Hardcoded forward slash, correct
        .split('/')                      // Hardcoded forward slash, correct
        .slice(1)

      const foldersWithStatic = join('static', ...folders)  // Inserts backslashes on Windows -> Invalid CSS name

      return `${foldersWithStatic}/[name]-[hash][ext][query]`  // Hardcoded forward slash, correct.
    }

Since this is called from a Webpack context with already normalized path delimiters I'd say it's reasonable to just forgo path.join and just use Array.join, i.e.

const foldersWithStatic = ['static', ...folders].join('/')

If you like I can create a pull request for this.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.