sonicdoe / detect-character-encoding Goto Github PK

View Code? Open in Web Editor NEW

84.0 2.0 15.0 58.41 MB

Detect character encoding using ICU

License: Other

Python 13.69% C++ 57.22% JavaScript 29.09%

nodejs icu javascript c-plus-plus encoding character-encoding charset detect

detect-character-encoding's Introduction

detect-character-encoding

Detect character encoding using ICU

Tip: If you don’t need ICU in particular, consider using ced, which is based on Google’s lighter compact_enc_det library.

Installation

$ npm install detect-character-encoding

detect-character-encoding is a C++ addon. Therefore, you may need to install various build tools. Check node-gyp’s readme for more information.

Usage

const fs = require('fs');
const detectCharacterEncoding = require('detect-character-encoding');

const fileBuffer = fs.readFileSync('file.txt');
const charsetMatch = detectCharacterEncoding(fileBuffer);

console.log(charsetMatch);
// {
//   encoding: 'UTF-8',
//   confidence: 60
// }

detect-character-encoding may return null if no charset matches.

Supported operating systems

macOS Sonoma
Ubuntu 22.04 and 20.04
Debian 12, 11, and 10

detect-character-encoding does not support 32-bit operating systems.

Supported character sets

As listed in ICU’s user guide:

UTF-8
UTF-16BE
UTF-16LE
UTF-32BE
UTF-32LE
Shift_JIS
ISO-2022-JP
ISO-2022-CN
ISO-2022-KR
GB18030
Big5
EUC-JP
EUC-KR
ISO-8859-1
ISO-8859-2
ISO-8859-5
ISO-8859-6
ISO-8859-7
ISO-8859-8
ISO-8859-9
windows-1250
windows-1251
windows-1252
windows-1253
windows-1254
windows-1255
windows-1256
KOI8-R
IBM420
IBM424

License

detect-character-encoding is licensed under the BSD 2-clause license but includes third-party software under different licenses. See LICENSE.md for the full license text.

detect-character-encoding's People

Contributors

Stargazers

Watchers

Forkers

cronopio andythink gitstoy mhertsch wanjarus jevo duduluu arun1895 rajeshdude0 twisterking eduilson flyingdr antonsacred gerhobbelt

detect-character-encoding's Issues

install failed

hi:
i install failed, message as below, thanks.

env:

macOS: 10.12.1 (16B2657)
nodejs: v8.1.4

error screenshot:

Module doesn't work on Docker

When running the module in a Docker container using this config:

FROM mhart/alpine-node:8.1.4

RUN apk update
RUN apk add --no-cache git openssh python make g++ libc6-compat

# copy project files
COPY . /src

# install node_modules
RUN cd /src; npm install

# run
CMD ["node", "/src/app/app.js"]
EXPOSE 3002

I get this error:

/src/node_modules/bindings/bindings.js:83
        throw e
        ^

Error: Error relocating /src/node_modules/detect-character-encoding/build/Release/icuWrapper.node: ucnv_close_58: symbol not found
    at Object.Module._extensions..node (module.js:598:18)
    at Module.load (module.js:503:32)
    at tryModuleLoad (module.js:466:12)
    at Function.Module._load (module.js:458:3)
    at Module.require (module.js:513:17)
    at require (internal/module.js:11:18)
    at bindings (/src/node_modules/bindings/bindings.js:76:44)
    at Object.<anonymous> (/src/node_modules/detect-character-encoding/index.js:3:37)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)

It works locally (OSX) and on Travis (linux i think), and I have tried it with versions of node from 0.10.48 all the way up to 8.1.4. Are there unlisted dependencies that you can think of?

cant build on node v12.3.1

node: v12.3.1
node-gyp: 3.8.0
npm: 6.9.0


/root/.node-gyp/12.3.1/include/node/v8.h:3359:26: note: declared here
                     bool Set(Local<Value> key, Local<Value> value));
                          ^
/root/.node-gyp/12.3.1/include/node/v8config.h:322:3: note: in definition of macro 'V8_DEPRECATE_SOON'
   declarator __attribute__((deprecated(message)))
   ^~~~~~~~~~
../icuWrapper.cpp:67:96: warning: 'bool v8::Object::Set(v8::Local<v8::Value>, v8::Local<v8::Value>)' is deprecated: Use maybe version [-Wdeprecated-declarations]
  obj->Set(Nan::New<v8::String>("confidence").ToLocalChecked(), Nan::New<v8::Number>(confidence));
                                                                                                ^
In file included from /root/.node-gyp/12.3.1/include/node/v8-internal.h:14:0,
                 from /root/.node-gyp/12.3.1/include/node/v8.h:25,
                 from /root/.node-gyp/12.3.1/include/node/node.h:63,
                 from ../icuWrapper.cpp:1:
/root/.node-gyp/12.3.1/include/node/v8.h:3359:26: note: declared here
                     bool Set(Local<Value> key, Local<Value> value));
                          ^
/root/.node-gyp/12.3.1/include/node/v8config.h:322:3: note: in definition of macro 'V8_DEPRECATE_SOON'
   declarator __attribute__((deprecated(message)))
   ^~~~~~~~~~
../icuWrapper.cpp: In function 'void Init(v8::Local<v8::Object>)':
../icuWrapper.cpp:75:72: error: no matching function for call to 'v8::FunctionTemplate::GetFunction()'
   Nan::New<v8::FunctionTemplate>(DetectCharacterEncoding)->GetFunction());
                                                                        ^
In file included from /root/.node-gyp/12.3.1/include/node/node.h:63:0,
                 from ../icuWrapper.cpp:1:
/root/.node-gyp/12.3.1/include/node/v8.h:5947:46: note: candidate: v8::MaybeLocal<v8::Function> v8::FunctionTemplate::GetFunction(v8::Local<v8::Context>)
   V8_WARN_UNUSED_RESULT MaybeLocal<Function> GetFunction(
                                              ^~~~~~~~~~~
/root/.node-gyp/12.3.1/include/node/v8.h:5947:46: note:   candidate expects 1 argument, 0 provided
icuWrapper.target.mk:126: recipe for target 'Release/obj.target/icuWrapper/icuWrapper.o' failed
make: *** [Release/obj.target/icuWrapper/icuWrapper.o] Error 1
make: Leaving directory '/mystuff/node_modules/detect-character-encoding/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/root/.nvm/versions/node/v12.3.1/lib/node_modules/npm/node_modules/npm-lifecycle/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:200:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Linux 4.15.0-45-generic
gyp ERR! command "/root/.nvm/versions/node/v12.3.1/bin/node" "/root/.nvm/versions/node/v12.3.1/lib/node_modules/npm/node_modules/npm-lifecycle/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /mystuff/node_modules/detect-character-encoding
gyp ERR! node -v v12.3.1
gyp ERR! node-gyp -v v3.8.0

node-gyp rebuild fails on ubuntu 14.04

node-gyp rebuild fails on ubuntu 14.04
node version: 5.6.0
npm version: 3.6.0

icuWrapper.node was compiled against a different Node.js version

I got this error on electron, and tried npm rebuild but nothing changed. How can I fix this?

'/Users/.../node_modules/detect-character-encoding/build/Release/icuWrapper.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 67. This version of Node.js requires
NODE_MODULE_VERSION 73. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).
    at process.func (electron/js2c/asar.js:155)
    at process.func [as dlopen] (electron/js2c/asar.js:155)
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:828)
    at Object.func (electron/js2c/asar.js:155)
    at Object.func [as .node] (electron/js2c/asar.js:155)
    at Module.load (internal/modules/cjs/loader.js:645)
    at Function.Module._load (internal/modules/cjs/loader.js:560)
    at Module.require (internal/modules/cjs/loader.js:685)
    at require (internal/modules/cjs/helpers.js:16)
    at bindings (/Users/funatsu/lab/KarteCSVConverter/node_modules/bindings/bindings.js:112)

Support Context Aware flag in Node

First of all, I have to say this is one of the most reliable character encoding detection package I managed to find, which is why I really want to include it in my Electron project. That said, I want to point out that in the future (not so near but still foreseeable), the Electron team will require native modules to be using NAPI or be Context Aware, as detailed in electron/electron#18397. Could you please consider this direction?

Module version mismatch. Expected 50, got 48

I try use detect-character-encoding in Desktop app with Electron.
But i have this error:

Uncaught Error: Module version mismatch. Expected 50, got 48.

/app/node_modules/bindings/bindings.js:83

Detect encoding in variable

Hello.

I am wondering if it is possible to directly convert a string to UTF8 without saving it in a file ?

Thanks.

Failed to install on debian.

Got the following errors:

`
node_modules/detect-character-encoding/build/Release/icuWrapper.node: undefined symbol: ucnv_open_61

`
node_modules/detect-character-encoding/build/Release/icuWrapper.node: undefined symbol: ucnv_open_59

Does not work under Node.js v6 on macOS

node-gyp rebuild produces quite a few error: no type named 'nullptr_t' in namespace 'std' errors.

Originally reported in #9.

I can't install detect-character-encoding.

nodejs version : v5.3.0
npm version : 3.5.2
Operating System : ArchLinux 64bit

% npm install --save detect-character-encoding

> [email protected] install /home/elisa/repos/mine/convert-to-utf8/node_modules/detect-character-encoding
> node-gyp rebuild

gyp: name 'icu_ver_major' is not defined while evaluating condition 'icu_ver_major == 54' in vendor/icu/icu.gyp while loading dependencies of binding.gyp while trying to load binding.gyp

Could not install detect-character-encoding

I've added detect-character-encoding to my project, but I'm not being able to compile it. I get the following error message:

error /home/domanski/Projects/visitown-api/node_modules/detect-character-encoding: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments: 
Directory: /home/domanski/Projects/visitown-api/node_modules/detect-character-encoding
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | linux | x64
gyp info find Python using Python version 3.8.6 found at "/usr/bin/python3"
gyp info spawn /usr/bin/python3
gyp info spawn args [
gyp info spawn args   '/home/domanski/.config/yarn/global/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/home/domanski/Projects/visitown-api/node_modules/detect-character-encoding/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/domanski/Projects/visitown-api/node_modules/detect-character-encoding/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/domanski/.config/yarn/global/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/home/domanski/.cache/node-gyp/12.18.2/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/home/domanski/.cache/node-gyp/12.18.2',
gyp info spawn args   '-Dnode_gyp_dir=/home/domanski/.config/yarn/global/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/home/domanski/.cache/node-gyp/12.18.2/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/home/domanski/Projects/visitown-api/node_modules/detect-character-encoding',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/home/domanski/Projects/visitown-api/node_modules/detect-character-encoding/build'
  TOUCH Release/obj.target/vendor/icu/icu_implementation.stamp
  TOUCH Release/obj.target/vendor/icu/icu_uconfig.stamp
  TOUCH Release/obj.target/vendor/icu/icu_uconfig_target.stamp
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/ucsdet.o
make: g++: No such file or directory
make: *** [vendor/icu/icui18n.target.mk:142: Release/obj.target/icui18n/vendor/icu/i18n/ucsdet.o] Error 127
make: Leaving directory '/home/domanski/Projects/visitown-api/node_modules/detect-character-encoding/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/home/domanski/.config/yarn/global/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (events.js:315:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:275:12)
gyp ERR! System Linux 5.8.0-33-generic
gyp ERR! command "/usr/bin/node" "/home/domanski/.yarn/bin/node-gyp" "rebuild"
gyp ERR! cwd /home/domanski/Projects/visitown-api/node_modules/detect-character-encoding

My environment is:
UBUNTU 20.10
Node: 12.19
NPM: 6.14.8
Yarn: 1.22.5

I guess that it has something to do with ICU, but I don't know what to do to set it up of fix it. I've just bought a new laptop and it is a fresh new ubuntu installation. In my previous laptop (with same Ubuntu version) my project used to compile with no problems.

Can anyone help me?

Does not work under Node.js v6 on Alpine Linux

Error: Error relocating /usr/src/app/build/Release/icuWrapper.node: utrace_cleanup_59: symbol not found

Originally reported in #8 (comment).

Unable to install on ubuntu 18.04

please see the error log and guide me, I have tried everything but it is not working

/home/rafay/.meteor/packages/meteor W20191030-19:28:01.750(5)? W20191030-19:28:01.750(5)? W20191030-19:28:01.750(5)? (STDERR)
W20191030-19:28:01.750(5)? W20191030-19:28:01.751(5)? W20191030-19:28:01.751(5)? W20191030-19:28:01.751(5)? W20191030-19:28:01.751(5)? W20191030-19:28:01.751(5)? (STDERR) W20191030-19:28:01.751(5)? (STDERR) W20191030-19:28:01.752(5)? (STDERR) W20191030-19:28:01.752(5)? (STDERR) W20191030-19:28:01.752(5)? (STDERR) W20191030-19:28:01.752(5)? (STDERR) W20191030-19:28:01.757(5)? (STDERR) W20191030-19:28:01.757(5)? (STDERR) W20191030-19:28:01.758(5)? (STDERR) W20191030-19:28:01.758(5)? (STDERR) W20191030-19:28:01.758(5)? (STDERR) W20191030-19:28:01.758(5)? (STDERR) W20191030-19:28:01.758(5)? (STDERR) W20191030-19:28:01.759(5)? (STDERR) W20191030-19:28:01.759(5)? (STDERR) W20191030-19:28:01.759(5)? (STDERR) W20191030-19:28:01.759(5)? (STDERR) W20191030-19:28:01.760(5)? (STDERR) W20191030-19:28:01.760(5)? (STDERR) W20191030-19:28:01.760(5)? (STDERR) W20191030-19:28:01.760(5)? (STDERR) W20191030-19:28:01.761(5)? (STDERR) -tool/.1.8.1.ani1yi.p0f9s++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:280
(STDERR) throw(ex);
(STDERR) ^
(STDERR) Error: The module '/home/rafay/Documents/work/denis/node_modules/detect-character-encoding/build/Release/icuWrapper.node'
(STDERR) was compiled against a different Node.js version using
(STDERR) NODE_MODULE_VERSION 64. This version of Node.js requires
(STDERR) NODE_MODULE_VERSION 57. Please try re-compiling or re-installing
(STDERR) the module (for instance, using npm rebuild or npm install).
at Object.Module._extensions..node (module.js:682:18)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at bindings (/home/rafay/Documents/work/denis/node_modules/bindings/bindings.js:112:48)
at Object. (/home/rafay/Documents/work/denis/node_modules/detect-character-encoding/index.js:3:37)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Module.require (module.js:597:17)
at require (internal/module.js:11:18)
at npmRequire (/home/rafay/Documents/work/denis/.meteor/local/build/programs/server/npm-require.js:133:10)
at Module.useNode (packages/modules-runtime.js:664:18)
at index.js (packages/modules.js:790:8)
at fileEvaluate (packages/modules-runtime.js:336:7)
at Module.require (packages/modules-runtime.js:238:14)
at require (packages/modules-runtime.js:258:21)
at utils.js (imports/api/server/utils.js:5:33)

The install script fails

gyp WARN install got an error, rolling back install
gyp ERR! configure error 
gyp ERR! stack Error: 404 response downloading https://nodejs.org/download//v8.1.1/node-v8.1.1-headers.tar.gz
gyp ERR! stack     at Request.<anonymous> (/home/admin/.nvm/versions/node/v8.1.1/lib/node_modules/npm/node_modules/node-gyp/lib/install.js:210:14)
gyp ERR! stack     at emitOne (events.js:120:20)
gyp ERR! stack     at Request.emit (events.js:210:7)
gyp ERR! stack     at Request.onRequestResponse (/home/admin/.nvm/versions/node/v8.1.1/lib/node_modules/npm/node_modules/request/request.js:1074:10)
gyp ERR! stack     at emitOne (events.js:115:13)
gyp ERR! stack     at ClientRequest.emit (events.js:210:7)
gyp ERR! stack     at HTTPParser.parserOnIncomingClient (_http_client.js:564:21)
gyp ERR! stack     at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)
gyp ERR! stack     at TLSSocket.socketOnData (_http_client.js:453:20)
gyp ERR! stack     at emitOne (events.js:115:13)
gyp ERR! System Linux 3.16.0-4-amd64
gyp ERR! command "/home/admin/.nvm/versions/node/v8.1.1/bin/node" "/home/admin/.nvm/versions/node/v8.1.1/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/admin/pg5/node_modules/detect-character-encoding
gyp ERR! node -v v8.1.1
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok 
npm WARN @0.0.5-alpha No description
npm WARN @0.0.5-alpha No repository field.
npm WARN @0.0.5-alpha No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/admin/.npm/_logs/2017-06-14T08_54_22_338Z-debug.log

Install fails in Windows

Hope this helps:

> [email protected] install C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding
> node-gyp rebuild
C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding>if not defined npm_config_node_gyp (node "C:\Users\m\AppData\Roaming\npm\node_modules\npm\node_modules\npm-lifecycle\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node "C:\Users\m\AppData\Roaming\npm\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" rebuild ) 
Warning: Missing input files:
C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\vendor\icu\..\..\..\vendor\icu\iculslocs.cc
C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\vendor\icu\..\..\..\vendor\icu\no-op.cc
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  csdetect.cpp
  csmatch.cpp
  csr2022.cpp
  csrecog.cpp
  csrmbcs.cpp
  csrsbcs.cpp
  csrucode.cpp
  csrutf8.cpp
c:\users\m\appdata\roaming\npm\node_modules\detect-character-encoding\vendor\icu\common\unicode/uclean.h(152): warning C4229: anachronism used: modifiers on data are ignored (compiling source file ..\..\..\vendor\icu\i18n\csdetect.cpp) [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\vendor\icu\icui18n.vcxproj]
  inputext.cpp
  ucln_in.cpp
  ucsdet.cpp
  win_delay_load_hook.cc
c:\users\m\appdata\roaming\npm\node_modules\detect-character-encoding\vendor\icu\common\unicode/uclean.h(152): warning C4229: anachronism used: modifiers on data are ignored (compiling source file ..\..\..\vendor\icu\i18n\ucln_in.cpp) [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\vendor\icu\icui18n.vcxproj]
  icui18n.vcxproj -> C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\Release\\icui18n.lib
  cmemory.cpp
  uarrsort.cpp
  ucln_cmn.cpp
  uenum.cpp
  umutex.cpp
  uobject.cpp
  utypes.cpp
  win_delay_load_hook.cc
c:\users\m\appdata\roaming\npm\node_modules\detect-character-encoding\vendor\icu\common\unicode/uclean.h(152): warning C4229: anachronism used: modifiers on data are ignored (compiling source file ..\..\..\vendor\icu\common\ucln_cmn.cpp) [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\vendor\icu\icuucx.vcxproj]
c:\users\m\appdata\roaming\npm\node_modules\detect-character-encoding\vendor\icu\common\unicode/uclean.h(152): warning C4229: anachronism used: modifiers on data are ignored (compiling source file ..\..\..\vendor\icu\common\cmemory.cpp) [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\vendor\icu\icuucx.vcxproj]
c:\users\m\appdata\roaming\npm\node_modules\detect-character-encoding\vendor\icu\common\unicode/uclean.h(152): warning C4229: anachronism used: modifiers on data are ignored (compiling source file ..\..\..\vendor\icu\common\umutex.cpp) [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\vendor\icu\icuucx.vcxproj]
  icuucx.vcxproj -> C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\Release\\icuucx.lib
  icuWrapper.cpp
  win_delay_load_hook.cc
     Creating library C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\Release\icuWrapper.lib and object C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\Release\icuWrapper.exp
icui18n.lib(csmatch.obj) : error LNK2001: unresolved external symbol ucnv_open_59 [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\icuWrapper.vcxproj]
icui18n.lib(csmatch.obj) : error LNK2001: unresolved external symbol ucnv_toUChars_59 [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\icuWrapper.vcxproj]
icui18n.lib(csmatch.obj) : error LNK2001: unresolved external symbol ucnv_close_59 [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\icuWrapper.vcxproj]
icuucx.lib(uenum.obj) : error LNK2001: unresolved external symbol u_UCharsToChars_59 [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\icuWrapper.vcxproj]
icuucx.lib(uenum.obj) : error LNK2001: unresolved external symbol u_charsToUChars_59 [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\icuWrapper.vcxproj]
icuucx.lib(ucln_cmn.obj) : error LNK2001: unresolved external symbol utrace_cleanup_59 [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\icuWrapper.vcxproj]
C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\Release\icuWrapper.node : fatal error LNK1120: 6 unresolved externals [C:\Users\m\AppData\Roaming\npm\node_modules\detect-character-encoding\build\icuWrapper.vcxproj]

A native crash

########CRASH STACK:
#0 0x00007fbf4050ea21 in icu_59::CharsetMatch::getName() const ()
from /home/tuna/papilio/node_modules/detect-character-encoding/build/Release/icuWrapper.node
#1 0x00007fbf4050d35b in DetectCharacterEncoding(Nan::FunctionCallbackInfov8::Value const&) ()
from /home/tuna/papilio/node_modules/detect-character-encoding/build/Release/icuWrapper.node
#2 0x00007fbf4050cf96 in Nan::imp::FunctionCallbackWrapper(v8::FunctionCallbackInfov8::Value const&) ()
from /home/tuna/papilio/node_modules/detect-character-encoding/build/Release/icuWrapper.node

########USE THIS DEMO TO REAPPEARS:
function main() {
console.log("=====START=====");
let rawData = new Buffer('AD', 'hex');
const detectCharacterEncoding = require('detect-character-encoding');
const charsetMatch = detectCharacterEncoding(rawData);
console.log("=====END=====", charsetMatch.encoding);
}

########FIXUP DIFF:
diff --git a/icuWrapper.cpp b/icuWrapper.cpp
index a228fe0..8d0bba3 100644
--- a/icuWrapper.cpp
+++ b/icuWrapper.cpp
@@ -34,6 +34,12 @@ NAN_METHOD(DetectCharacterEncoding) {

    charsetMatch = ucsdet_detect(charsetDetector, &errorCode);

```
  if(NULL == charsetMatch) {
```

          Nan::ThrowError("Failed to charset not match.");

           ucsdet_close(charsetDetector);

```
           return;
```
```
  }
```

  if(U_FAILURE(errorCode)) {
          Nan::ThrowError("Failed to detect charset.");
          ucsdet_close(charsetDetector);

No default jsDelivr CDN file set

This package doesn't have a default file set. You can set it via jsdelivr, browser, or main field in package.json

node:8.12-alpine: utrace_cleanup_59: symbol not found

Hi,

I'm trying to move our docker images to Alpine, but I've run into a road-block with [email protected]

Reproduce:

> docker run --rm -it node:8.12-alpine sh -c 'apk add python build-base && npm i detect-character-encoding && node -e "require(\"detect-character-encoding\")"'

Output from above command:

fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/28) Installing binutils (2.30-r5)
(2/28) Installing libmagic (5.32-r0)
(3/28) Installing file (5.32-r0)
(4/28) Installing gmp (6.1.2-r1)
(5/28) Installing isl (0.18-r0)
(6/28) Installing libgomp (6.4.0-r9)
(7/28) Installing libatomic (6.4.0-r9)
(8/28) Installing pkgconf (1.5.3-r0)
(9/28) Installing mpfr3 (3.1.5-r1)
(10/28) Installing mpc1 (1.0.3-r1)
(11/28) Upgrading libstdc++ (6.4.0-r8 -> 6.4.0-r9)
(12/28) Installing gcc (6.4.0-r9)
(13/28) Installing musl-dev (1.1.19-r10)
(14/28) Installing libc-dev (0.7.1-r0)
(15/28) Installing g++ (6.4.0-r9)
(16/28) Installing make (4.2.1-r2)
(17/28) Installing fortify-headers (0.9-r0)
(18/28) Installing build-base (0.5-r1)
(19/28) Installing libbz2 (1.0.6-r6)
(20/28) Installing expat (2.2.5-r0)
(21/28) Installing libffi (3.2.1-r4)
(22/28) Installing gdbm (1.13-r1)
(23/28) Installing ncurses-terminfo-base (6.1_p20180818-r1)
(24/28) Installing ncurses-terminfo (6.1_p20180818-r1)
(25/28) Installing ncurses-libs (6.1_p20180818-r1)
(26/28) Installing readline (7.0.003-r0)
(27/28) Installing sqlite-libs (3.24.0-r0)
(28/28) Installing python2 (2.7.15-r1)
Executing busybox-1.28.4-r1.trigger
OK: 212 MiB in 42 packages

> [email protected] install /node_modules/detect-character-encoding
> node-gyp rebuild

make: Entering directory '/node_modules/detect-character-encoding/build'
  TOUCH Release/obj.target/vendor/icu/icu_implementation.stamp
  TOUCH Release/obj.target/vendor/icu/icu_uconfig.stamp
  TOUCH Release/obj.target/vendor/icu/icu_uconfig_target.stamp
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csdetect.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csmatch.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csr2022.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csrecog.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csrmbcs.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csrsbcs.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csrucode.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/csrutf8.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/inputext.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/ucln_in.o
  CXX(target) Release/obj.target/icui18n/vendor/icu/i18n/ucsdet.o
  AR(target) Release/obj.target/vendor/icu/icui18n.a
  COPY Release/icui18n.a
  CXX(target) Release/obj.target/icuucx/vendor/icu/common/cmemory.o
  CXX(target) Release/obj.target/icuucx/vendor/icu/common/uarrsort.o
  CXX(target) Release/obj.target/icuucx/vendor/icu/common/ucln_cmn.o
  CXX(target) Release/obj.target/icuucx/vendor/icu/common/uenum.o
  CXX(target) Release/obj.target/icuucx/vendor/icu/common/umutex.o
  CXX(target) Release/obj.target/icuucx/vendor/icu/common/uobject.o
  CXX(target) Release/obj.target/icuucx/vendor/icu/common/utypes.o
  AR(target) Release/obj.target/vendor/icu/icuucx.a
  COPY Release/icuucx.a
  CXX(target) Release/obj.target/icuWrapper/icuWrapper.o
  SOLINK_MODULE(target) Release/obj.target/icuWrapper.node
  COPY Release/icuWrapper.node
  TOUCH Release/obj.host/vendor/icu/icu_implementation.stamp
  TOUCH Release/obj.host/vendor/icu/icu_uconfig.stamp
  TOUCH Release/obj.host/vendor/icu/icui18n.stamp
make: Leaving directory '/node_modules/detect-character-encoding/build'
npm WARN saveError ENOENT: no such file or directory, open '/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/package.json'
npm WARN !invalid#1 No description
npm WARN !invalid#1 No repository field.
npm WARN !invalid#1 No README data
npm WARN !invalid#1 No license field.

+ [email protected]
added 3 packages from 9 contributors and audited 3 packages in 5.16s
found 0 vulnerabilities

/node_modules/bindings/bindings.js:88
        throw e
        ^

Error: Error relocating /node_modules/detect-character-encoding/build/Release/icuWrapper.node: utrace_cleanup_59: symbol not found
    at Object.Module._extensions..node (module.js:682:18)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at bindings (/node_modules/bindings/bindings.js:81:44)
    at Object.<anonymous> (/node_modules/detect-character-encoding/index.js:3:37)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)

I'm seen the same plus a lot of warnings during on node:10-alpine.

Might be related to #11 but that is for Node v6.

Add support for use in browser

If this worked client-side, the File API could be used to read a file locally and detect everything locally which would avoid the need to upload anything to a server.

It looks like this relies on C++ code. Any chance this could be turned into a fully JavaScript-powered library in the future?