andipaetzold / mdb-reader Goto Github PK
View Code? Open in Web Editor NEWJavaScript library to read data from Access databases
Home Page: https://npm.im/mdb-reader
License: MIT License
JavaScript library to read data from Access databases
Home Page: https://npm.im/mdb-reader
License: MIT License
Currently, tests run against x.0.0 and the latest version of LTS releases. This should be changed to only run against the initial LTS version and the latest release of an LTS version.
This is a breaking change
Hi @andipaetzold
Thanks for this very useful libary to migrate old legacy data cross-platform!
Comparing the output of your library to the output of mdb-tools, I detected some discrepancy in the datetime parsing.
Example:
const td = 32234.339699074073;
const daysDiff = 25569;
return new Date((td - daysDiff) * 86400 * 1000);
This results in 1988-04-01T08:09:09.999Z
instead of 1988-04-01T08:09:10.000Z
due to the poor floating point arithmetic in JavaScript.
I suggest rounding the milliseconds passed to the Date constructor.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
@sveltejs/adapter-auto
, @sveltejs/kit
, @sveltejs/vite-plugin-svelte
, @types/chai
, @types/mocha
, @types/node
, eslint
, mocha
, semantic-release
, svelte-check
, tslib
, typescript
)These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.
@typescript-eslint/eslint-plugin
, @typescript-eslint/parser
).github/workflows/compressed-size.yml
actions/checkout v4
preactjs/compressed-size-action v2
ubuntu 22.04
.github/workflows/push.yml
actions/checkout v4
actions/setup-node v4
actions/checkout v4
actions/setup-node v4.0.3
actions/checkout v4
actions/setup-node v4
ubuntu 22.04
ubuntu 22.04
ubuntu 22.04
examples/browser/package.json
buffer 6.0.3
mdb-reader 3.0.0
rimraf 6.0.1
examples/sveltekit/package.json
mdb-reader 3.0.0
@sveltejs/adapter-auto 3.2.2
@sveltejs/kit 2.5.18
@sveltejs/vite-plugin-svelte 3.1.1
svelte 4.2.18
svelte-check 3.8.5
tslib 2.6.3
typescript 5.5.4
vite 5.3.5
vite-plugin-node-polyfills 0.22.0
examples/to-json/package.json
mdb-reader 3.0.0
package.json
browserify-aes ^1.0.0
create-hash ^1.0.0
fast-xml-parser ^4.0.0
@semantic-release/changelog 6.0.3
@semantic-release/git 10.0.1
@tsconfig/node18 18.2.4
@tsconfig/strictest 2.0.5
@types/chai 4.3.16
@types/mocha 10.0.7
@types/mocha-each 2.0.4
@types/node 18.19.42
@typescript-eslint/eslint-plugin 7.18.0
@typescript-eslint/parser 7.18.0
chai 5.1.1
eslint 8.57.0
mocha 10.7.0
mocha-each 2.0.1
rimraf 6.0.1
semantic-release 24.0.0
ts-node 10.9.2
typescript 5.5.4
.nvmrc
node 20
I have a french MDB, using probably not an UTF-8 encoding. If I read the table 'tTour', i see:
{ NoTour: 1, NomTour: 'Engagement' }
{ NoTour: 2, NomTour: 'S�rie' }
{ NoTour: 3, NomTour: '16�me f.' }
{ NoTour: 4, NomTour: '8�me f.' }
{ NoTour: 5, NomTour: 'Quart f.' }
{ NoTour: 6, NomTour: 'Semi f.' }
{ NoTour: 7, NomTour: 'Finale'
I suspect an iso8859-15.
the mdb source:
https://u.pcloud.link/publink/show?code=XZnIlkVZLG3ffaEEef4XDRskx2F5qyBGbHYV
Is there an option to set the encoding source ?
Hello,
Currently, I'm trying to integrate mdb-reader in a vue application (vue2 with vuetify) and I encounter some issues because you use the ?? and ?. operators. These operators are pretty recent and not supported in many browser. Is it really needed in this library.
Best regards,
Hi, I am trying to use this awesome package in Nestjs, but I receive this error, please help to solve this issue, thank you in advance.
const mdb_reader_1 = require("mdb-reader").default; ^ Error [ERR_REQUIRE_ESM]: require() of ES Module E:\api\node_modules\mdb-reader\lib\node\index.js from E:\api\dist\workshop\workshop.service.js not supported. Instead change the require of index.js in E:\api\dist\workshop\workshop.service.js to a dynamic import() which is available in all CommonJS modules. at Object.<anonymous> (E:\api\dist\workshop\workshop.service.js:12:22) at Object.<anonymous> (E:\api\dist\workshop\workshop.module.js:11:28) at Object.<anonymous> (E:\api\dist\app.module.js:17:27) at Object.<anonymous> (E:\api\dist\main.js:4:22)
I am reading a table with 9855 records. I want to read 1000 at a time.
If I pass rowOffset:0, rowLimit:1000 I get 1000 records, if I pass rowOffset 1000, rowLimit: 1000 I get 0(zero) records
table.name:Products, table.rowCount:9855
table.getData({rowOffset:0,rowLimit:1000});
returned: 1000 records
table.getData({rowOffset:1000,rowLimit:1000});
returned: 0 records
and if I pass rowOffset 1000, rowLimit: 2000 it trows an error. So, I am not sure how it is supposed to work.
table.name:Products, table.rowCount:9855
table.getData({rowOffset:0,rowLimit:1000});
returned: 1000 records
table.getData({rowOffset:1000,rowLimit:2000});
(node:20761) UnhandledPromiseRejectionWarning: RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 4094. Received 6400
at boundsError (internal/buffer.js:81:9)
at Buffer.readUInt16LE (internal/buffer.js:238:5)
at Buffer.readUIntLE (internal/buffer.js:175:17)
at D.getDataFromPage (/Users/fpw/Projects/migrandama/mystore/mdbImport2/node_modules/mdb-reader/lib/index.js:1:10704)
at D.getData (/Users/fpw/Projects/migrandama/mystore/mdbImport2/node_modules/mdb-reader/lib/index.js:1:10030)
The library currently always reads all records of a data page and then filters it using Array.slice
. This means a performance loss, especially for large table records.
This should be improved such that only data is read that is actually returned.
I am unable to use const MDBParser = require("mdb-reader")
. I don't wanna change all other imports of other libraries being used.
First of all, thank you for this great library. It is tremendously helpful for client-side exploration of MS Access databases.
I noticed that the browser-based example is using parcel to bundle the assets. You are probably aware of this, but it is possible to eliminate the build/bundling step by referencing a CDN. For example:
import { Buffer } from 'https://cdn.jsdelivr.net/npm/buffer@6/+esm'
import MDBReader from 'https://cdn.jsdelivr.net/npm/[email protected]/+esm' // @latest or @2 can be used instead
This could prove helpful for importing this library directly in the browser or other JavaScript runtimes that don't use a package manager.
Hi,
Currently, I am using mdb-reader in the project, and after setting up Dependabot I receive an alert for a security issue that related to package fast-xml-parser
. Can you update package fast-xml-parser
to version 4.2.4?
Here are the commands I ran:
git clone https://github.com/andipaetzold/mdb-reader.git
cd mdb-reader/examples/browser
npm install
npm run dev
Here's the output:
> [email protected] dev
> rimraf dist && parcel
Server running at http://localhost:1234
× Build failed.
@parcel/core: Failed to resolve 'mdb-reader' from './examples/browser/src/index.mjs'
/path/to/mdb-reader/examples/browser/src/index.mjs:2:23
1 | import { Buffer } from "buffer/";
> 2 | import MDBReader from "mdb-reader";
> | ^^^^^^^^^^^^
3 |
4 | const button = document.getElementById("button");
@parcel/resolver-default: Could not load './lib/browser/index.js' from module 'mdb-reader' found
in package.json#browser
/path/to/mdb-reader/examples/browser/node_modules/mdb-reader/package.json:6:16
5 | "main": "lib/node/index.js",
> 6 | "browser": "lib/browser/index.js",
> | ^^^^^^^^^^^^^^^^^^^^^^ './lib/browser/index.js' does not exist, did you mean './src/crypto/index.ts'?'
7 | "types": "lib/types/index.d.ts",
8 | "sideEffects": false,
Parcel is shutting down...
Mode: Browser (Chrome)
mdb-reader version: 2.2.2
Node version: 14.19.1
Buffer: 6.0.3
When using this library, and passing in an MDB file from the test examples, e.g. test.mdb (test/data/v2000) when constructing a reader, I receive the error: Range Error: offset is not uint
.
After debugging in Chrome, this is caused by: complexOrLong.js.readComplexOrLong(). This is calling buffer.readInt32LE() without passing an offset in. Other articles suggests that offset has a default version of 0 https://www.geeksforgeeks.org/node-js-buffer-readint32le-method/.
The buffer package however requires a value to be passed in https://github.com/feross/buffer/blob/795bbb5bda1b39f1370ebd784bea6107b087e3a7/index.js#L1127.
In this case should this package's code just be passing 0 in as the parameter value for offset?
If I manually edit those functions (integer.js, complexOrLong.js, float.js) in the node_modules folder for this package to pass 0 in as the index, the MDBReader is constructed and I can then query the data within it. Is there a mismatch between node versions?
I have this error with node 14.5 version:
file:///C:/Users/laure/Documents/Perso/Sources/live/node_modules/mdb-reader/lib/codec-handler/handlers/office/agile/EncryptionDescriptor.js:1
import { XMLParser } from "fast-xml-parser";
^^^^^^^^^
SyntaxError: The requested module 'fast-xml-parser' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
For example:
import pkg from 'fast-xml-parser';
const { XMLParser } = pkg;
at ModuleJob._instantiate (internal/modules/esm/module_job.js:98:21)
at async ModuleJob.run (internal/modules/esm/module_job.js:137:5)
at async Loader.import (internal/modules/esm/loader.js:162:24)
at async formattedImport (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\esm-utils.js:7:14)
at async Object.exports.loadFilesAsync (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\esm-utils.js:55:20)
at async singleRun (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\cli\run-helpers.js:125:3)
at async Object.exports.handler (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\cli\run.js:362:5)
It seems the problem comes with 'src/codec-handler/handlers/office/agile/EncryptionDescriptor.js' file.
Best regards,
See #169
I'm trying to use MDBReader
in the browser for a project that I'm serving with Web Dev Server, but keep hitting the following error:
Error while handling server request.
PluginError: Could not resolve import "crypto".
I've also tried directly referencing the built browser version
import { MDBReader } from 'mdb-reader/lib/browser/index.js';
but this yields a different error in lib/browser/environment/index.js
where browserifyAES
fails to import because it doesn't provide a default export.
Am I importing something incorrectly in either scenario?
Fields that contain over ~1893 characters are getting cut very short, returning a string that somewhat resembles what should have been returned. For example, if the long string started with "Version 1 ... " followed by large amounts of text, its returning "Versgr " and nothing else
I'm trying to read the table tRelais in this mdb:
https://u.pcloud.link/publink/show?code=XZu5EHVZVqot9BBBU7FMwQHIVSJ4zHIwB9i7
If I read this table with "sysinfotool MDB file viewer 20.0", I read a full table with no empty data. But, if I read with this library, I found many empty object:
{NoEngageRelais: 1487, NoEngagement: 44242, NoSerieTour: 8096, NoAthlete: 13310, NoOrdreRelais: null}
{NoEngageRelais: 1488, NoEngagement: 44242, NoSerieTour: 8096, NoAthlete: null, NoOrdreRelais: null} ...
The field NoOrdreRelais has the type octet
in french language... It is a small int with 8bits.
My script:
function loadRelais (reader) {
let cmtrelays = new Map()
const table = reader.getTable('tRelais')
let rows = table.getData()
for (let row of rows) {
console.log(row)
let uid = row.NoEngageRelais
let noEngagement = row.NoEngagement
let serieTour = row.NoSerieTour
let athUID = row.NoAthlete
let ord = row.NoOrdreRelais
cmtrelays.set(uid, { uid, noEngagement, serieTour, athUID, ord })
}
return cmtrelays
}
Hi, i am trying to run a script i wrote on a linux server but get an error (Windows works perfectly).
file:///root/mdb-reader/node_modules/mdb-reader/lib/node/MDBReader.js:14
this.db = new Database(this.buffer, password ?? "");
^
SyntaxError: Unexpected token '?'
Any idea how to fix this? Appeciate the help.
When I try to load this using the Browser example, I get:
Uncaught RangeError: offset is not uint
at checkOffset (index.js:1097:1)
at Uint8Array.readInt32LE (index.js:1220:1)
at readComplexOrLong (complexOrLong.js:2:1)
at readFieldValue (index.js:43:1)
at Table.getDataFromPage (Table.js:242:1)
at Table.getData (Table.js:132:1)
at new MDBReader (MDBReader.js:15:1)
How to read password protected mdb files? Does your library support this or any clue to add this capability?
Hello,
If you look into the table tSerieTour and look the column HoraireSerieTour, you have dates ending by 0. When I read the table with mdb-reader, I got:
Sat Dec 30 1899 15:14:21 GMT+0009 (Central European Standard Time) object
There is a delta of 9 minutes and 21seconds. My computer is GMT+1 (Paris, Bruxelles, ...)
If you set the computer clock with GTM+0, there's no bug.
MDB Sample File: https://drive.google.com/file/d/1G51j83Hj2CH92TAAnHHZyI8_rmzG9wn5/view
I am trying to read the data from Batch table but the values end up with something random "Qty": "109951162.7777"
whereas it has to be 40. The Batch table has only 4 rows.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.