Comments (18)
So if anyone arrives here having troubles
✅ Bundling and self-hosting with Webpack 5 / Next js
this is a way to make it work.
When you simply import and use shiki like this
import { getHighlighter } from "shiki";
getHighlighter().then(highlighter => /* do stuff */);
You might get the error
TypeError: WebAssembly: Response has unsupported MIME type 'text/html; charset=utf-8' expected 'application/wasm'
This is not really a MIME-Type-problem. What is happening here is that the browser bundle loads a wasm file (and some other files) from the default location
/dist/onig.wasm
/dist/themes/[yourtheme].json
and those files are not there. You can change the path as described here, but you might also want to serve those files yourself. For this, you can copy the npm package to your static files in next
next.config.js
const path = require("path");
const CopyPlugin = require("copy-webpack-plugin");
const nextConfig = {
webpack: (config, { webpack }) => {
/**
* Copying the whole npm package of shiki to static/shiki because it
* loads some files from a "cdn" in the browser (semi-hacky)
* @see https://github.com/shikijs/shiki#specify-a-custom-root-directory
*/
config.plugins.push(
new CopyPlugin({
patterns: [
{
from: path.resolve(path.dirname(require.resolve("shiki")), ".."),
to: "static/shiki/",
},
],
})
);
return config;
},
};
module.exports = nextConfig;
And then set the CDN to your own served folder
- import { getHighlighter } from "shiki";
+ import { getHighlighter, setCDN } from "shiki";
+ setCDN("/_next/static/shiki/");
getHighlighter().then(highlighter => /* do stuff */);
from shiki.
@canibanoglu Even if that's doable, is it desirable at all? Loading several MB of WASM and then the grammars to do syntax highlighting on client side?
I initially thought of shiki as an library, on either a static site generator system or locally, to faithfully reproduce VS Code's highlighting. I still think that's useful enough. If you need to dynamically do syntax highlighting in browser, maybe Monaco or Highlight.js would be better choices?
from shiki.
Author of onigasm
here. There is absolutely no problem running it in browsers, in fact, it was designed to be run in browsers for one of my own projects (now defunct).
It works perfectly fine with webpack
with minimal config changes, just define a rule in webpack config to use file-loader
for any import asking for *.wasm
file.
Input
// src/index.ts
await loadWASM(await import('onigasm/lib/onigasm.wasm'))`
Output
// out/bundle.eu21df1.js
loadWASM('build/onigasm.8ug81e3.wasm').then(...)`
Note: Webpack might try to use it's on WASM loader, despite the explicit rule. If that happens, fix this like this - webpack/webpack#7264 (comment) The entire thread is a great read
from shiki.
Oh, I quite agree with you there. I was just thinking that changing the way onigasm
is loaded slightly would enable people who were inclined to bundle it for browser use could do so without too much trouble.
Instead of reading the WASM file from the file system with fs
, we could use a loader to import
it into code and webpack
to take care of bundling for browser use if asked. Haven't tried anything yet though.
On a kinda related note: Is there a reason for not using node-oniguruma
instead of onigasm
for non-browser use?
from shiki.
It should be possible, but I haven't had time for it. You should start by making Onigasm runnable in browsers.
from shiki.
If you're using Vue and gridsome, there's a plugin for it. https://github.com/EldoranDev/gridsome-plugin-remark-shiki @ipelekhan
from shiki.
If you're using Vue and gridsome, there's a plugin for it. https://github.com/EldoranDev/gridsome-plugin-remark-shiki @ipelekhan
That library actually still runs on node.
from shiki.
Hmm, I looked into this a bit but I'm left a bit confused. As it is shiki
depends on onigasm
instead of node-oniguruma
. The onigLibs.ts
file loads the WASM and takes care of everything but due to the way it reads the binary file it won't bundle with webpack
due to the errors reported by @ipelekhan. If the goal is to depend solely on onigasm
then we need to make sure that this works on both web and node without any adjustments by the end user by replacing the fs
-dependent code segments.
One idea could be to introduce webpack
to the project and depend on loaders to pull the raw data from files instead of using fs
calls. Thoughts?
from shiki.
vscode-textmate depends on either onigasm or oniguruma so I was using it.
from shiki.
Related: microsoft/monaco-editor#1915
from shiki.
Blocked by zikaari/onigasm#20
Hopefully onigasm
can use an isomorphic lru-cache
alternative. I'll even make one with a dispose
hook if they need me to
from shiki.
This is in with 0.9.0 - see the readme and #109
from shiki.
So if anyone arrives here having troubles
✅ Bundling and self-hosting with Webpack 5 / Next js
this is a way to make it work.
When you simply import and use shiki like this
import { getHighlighter } from "shiki"; getHighlighter().then(highlighter => /* do stuff */);You might get the error
TypeError: WebAssembly: Response has unsupported MIME type 'text/html; charset=utf-8' expected 'application/wasm'
This is not really a MIME-Type-problem. What is happening here is that the browser bundle loads a wasm file (and some other files) from the default location
/dist/onig.wasm
/dist/themes/[yourtheme].json
and those files are not there. You can change the path as described here, but you might also want to serve those files yourself. For this, you can copy the npm package to your static files in next
next.config.js
const path = require("path"); const CopyPlugin = require("copy-webpack-plugin"); const nextConfig = { webpack: (config, { webpack }) => { /** * Copying the whole npm package of shiki to static/shiki because it * loads some files from a "cdn" in the browser (semi-hacky) * @see https://github.com/shikijs/shiki#specify-a-custom-root-directory */ config.plugins.push( new CopyPlugin({ patterns: [ { from: path.resolve(path.dirname(require.resolve("shiki")), ".."), to: "static/shiki/", }, ], }) ); return config; }, }; module.exports = nextConfig;And then set the CDN to your own served folder
- import { getHighlighter } from "shiki"; + import { getHighlighter, setCDN } from "shiki"; + setCDN("/_next/static/shiki/"); getHighlighter().then(highlighter => /* do stuff */);
I'm trying to do like you suggested, but I'm getting a lot of errors in nextjs.
from shiki.
It outputs no file nord.json at .... .next/server/app/(blog)/article/themes/nord.json
from shiki.
next 13 server components
from shiki.
I found this issue looking for how to use shiki with Next.js, if you did as well the discussion you are looking for is likely here: #398
from shiki.
If the problem is WASM bundling in Webpack this works - zikaari/onigasm#2 (comment)
from shiki.
So if anyone arrives here having troubles
✅ Bundling and self-hosting with Webpack 5 / Next js
this is a way to make it work.
When you simply import and use shiki like this
import { getHighlighter } from "shiki"; getHighlighter().then(highlighter => /* do stuff */);You might get the error
TypeError: WebAssembly: Response has unsupported MIME type 'text/html; charset=utf-8' expected 'application/wasm'
This is not really a MIME-Type-problem. What is happening here is that the browser bundle loads a wasm file (and some other files) from the default location
/dist/onig.wasm
/dist/themes/[yourtheme].json
and those files are not there. You can change the path as described here, but you might also want to serve those files yourself. For this, you can copy the npm package to your static files in next
next.config.js
const path = require("path"); const CopyPlugin = require("copy-webpack-plugin"); const nextConfig = { webpack: (config, { webpack }) => { /** * Copying the whole npm package of shiki to static/shiki because it * loads some files from a "cdn" in the browser (semi-hacky) * @see https://github.com/shikijs/shiki#specify-a-custom-root-directory */ config.plugins.push( new CopyPlugin({ patterns: [ { from: path.resolve(path.dirname(require.resolve("shiki")), ".."), to: "static/shiki/", }, ], }) ); return config; }, }; module.exports = nextConfig;And then set the CDN to your own served folder
- import { getHighlighter } from "shiki"; + import { getHighlighter, setCDN } from "shiki"; + setCDN("/_next/static/shiki/"); getHighlighter().then(highlighter => /* do stuff */);
This worked for me! Thanks @LukasBombach
from shiki.
Related Issues (20)
- `@shikijs/vitepress-twoslash`: Use twoslash in the case of using imported code snippets. HOT 5
- Row of `language-*` on a single `code` element HOT 1
- Vue 当没有写 `template` 时无法正常高亮 HOT 1
- @shikijs/markdown-it: code block is wrong when entering a non-existent language HOT 2
- Twoslash node query blocks (^?) no longer display on separate lines since 0.x
- `remove-notation-escape`: notation escape syntax’s normalize transformer HOT 2
- Rehype: default language
- "TypeError: onigBinding.UTF8ToString is not a function" after some time / parallel requests in Next.js HOT 10
- Inconsistent behavior between `@shikijs/twoslash` and `@shikijs/vitepress-twoslash`
- Diff syntax highlighting doesn't work with css variables theme
- `light-dark()` CSS function for dual themes HOT 3
- how to use codeToHtml + transformer (meta) ? HOT 5
- Detect Notation Transformers dynamically
- Properly disposing of highlighters HOT 1
- Describe the bug here HOT 2
- `@shikijs/rehype`: support `langAlias` options
- Optimization suggestions for the 'hast' type in Shiki.js HOT 2
- `@shikijs/rehype` - "30 instances have been created" HOT 6
- Return the grammar state after tokenizing HOT 3
- shiki/wasm dependency not found HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from shiki.