andipaetzold / mdb-reader Goto Github PK

I am reading a table with 9855 records. I want to read 1000 at a time.
If I pass rowOffset:0, rowLimit:1000 I get 1000 records, if I pass rowOffset 1000, rowLimit: 1000 I get 0(zero) records

table.name:Products, table.rowCount:9855
table.getData({rowOffset:0,rowLimit:1000});
returned: 1000 records
table.getData({rowOffset:1000,rowLimit:1000});
returned: 0 records

and if I pass rowOffset 1000, rowLimit: 2000 it trows an error. So, I am not sure how it is supposed to work.

table.name:Products, table.rowCount:9855
table.getData({rowOffset:0,rowLimit:1000});
returned: 1000 records
table.getData({rowOffset:1000,rowLimit:2000});
(node:20761) UnhandledPromiseRejectionWarning: RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 4094. Received 6400
    at boundsError (internal/buffer.js:81:9)
    at Buffer.readUInt16LE (internal/buffer.js:238:5)
    at Buffer.readUIntLE (internal/buffer.js:175:17)
    at D.getDataFromPage (/Users/fpw/Projects/migrandama/mystore/mdbImport2/node_modules/mdb-reader/lib/index.js:1:10704)
    at D.getData (/Users/fpw/Projects/migrandama/mystore/mdbImport2/node_modules/mdb-reader/lib/index.js:1:10030)

I am unable to use const MDBParser = require("mdb-reader"). I don't wanna change all other imports of other libraries being used.

MDB Sample File: https://drive.google.com/file/d/1G51j83Hj2CH92TAAnHHZyI8_rmzG9wn5/view

I am trying to read the data from Batch table but the values end up with something random "Qty": "109951162.7777" whereas it has to be 40. The Batch table has only 4 rows.

Mode: Browser (Chrome)
mdb-reader version: 2.2.2
Node version: 14.19.1
Buffer: 6.0.3

When using this library, and passing in an MDB file from the test examples, e.g. test.mdb (test/data/v2000) when constructing a reader, I receive the error: Range Error: offset is not uint.

After debugging in Chrome, this is caused by: complexOrLong.js.readComplexOrLong(). This is calling buffer.readInt32LE() without passing an offset in. Other articles suggests that offset has a default version of 0 https://www.geeksforgeeks.org/node-js-buffer-readint32le-method/.

The buffer package however requires a value to be passed in https://github.com/feross/buffer/blob/795bbb5bda1b39f1370ebd784bea6107b087e3a7/index.js#L1127.

In this case should this package's code just be passing 0 in as the parameter value for offset?

If I manually edit those functions (integer.js, complexOrLong.js, float.js) in the node_modules folder for this package to pass 0 in as the index, the MDBReader is constructed and I can then query the data within it. Is there a mismatch between node versions?

I have this error with node 14.5 version:

file:///C:/Users/laure/Documents/Perso/Sources/live/node_modules/mdb-reader/lib/codec-handler/handlers/office/agile/EncryptionDescriptor.js:1
import { XMLParser } from "fast-xml-parser";
         ^^^^^^^^^
SyntaxError: The requested module 'fast-xml-parser' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
For example:
import pkg from 'fast-xml-parser';
const { XMLParser } = pkg;
    at ModuleJob._instantiate (internal/modules/esm/module_job.js:98:21)
    at async ModuleJob.run (internal/modules/esm/module_job.js:137:5)
    at async Loader.import (internal/modules/esm/loader.js:162:24)
    at async formattedImport (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\esm-utils.js:7:14)
    at async Object.exports.loadFilesAsync (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\esm-utils.js:55:20)
    at async singleRun (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\cli\run-helpers.js:125:3)
    at async Object.exports.handler (C:\Users\laure\Documents\Perso\Sources\live\node_modules\mocha\lib\cli\run.js:362:5)

It seems the problem comes with 'src/codec-handler/handlers/office/agile/EncryptionDescriptor.js' file.

Best regards,

Here are the commands I ran:

git clone https://github.com/andipaetzold/mdb-reader.git
cd mdb-reader/examples/browser
npm install
npm run dev

Here's the output:

> [email protected] dev
> rimraf dist && parcel

Server running at http://localhost:1234
× Build failed.

@parcel/core: Failed to resolve 'mdb-reader' from './examples/browser/src/index.mjs'

  /path/to/mdb-reader/examples/browser/src/index.mjs:2:23
    1 | import { Buffer } from "buffer/";
  > 2 | import MDBReader from "mdb-reader";
  >   |                       ^^^^^^^^^^^^
    3 |
    4 | const button = document.getElementById("button");

@parcel/resolver-default: Could not load './lib/browser/index.js' from module 'mdb-reader' found
in package.json#browser

  /path/to/mdb-reader/examples/browser/node_modules/mdb-reader/package.json:6:16
    5 |     "main": "lib/node/index.js",
  > 6 |     "browser": "lib/browser/index.js",
  >   |                ^^^^^^^^^^^^^^^^^^^^^^ './lib/browser/index.js' does not exist, did you mean './src/crypto/index.ts'?'
    7 |     "types": "lib/types/index.d.ts",
    8 |     "sideEffects": false,

Parcel is shutting down...

Hello,

If you look into the table tSerieTour and look the column HoraireSerieTour, you have dates ending by 0. When I read the table with mdb-reader, I got:
Sat Dec 30 1899 15:14:21 GMT+0009 (Central European Standard Time) object
There is a delta of 9 minutes and 21seconds. My computer is GMT+1 (Paris, Bruxelles, ...)

If you set the computer clock with GTM+0, there's no bug.

mdb with problem

Hello,

Currently, I'm trying to integrate mdb-reader in a vue application (vue2 with vuetify) and I encounter some issues because you use the ?? and ?. operators. These operators are pretty recent and not supported in many browser. Is it really needed in this library.

Best regards,

Hi @andipaetzold
Thanks for this very useful libary to migrate old legacy data cross-platform!

Comparing the output of your library to the output of mdb-tools, I detected some discrepancy in the datetime parsing.
Example:

const td = 32234.339699074073;
const daysDiff = 25569;
return new Date((td - daysDiff) * 86400 * 1000);

This results in 1988-04-01T08:09:09.999Z instead of 1988-04-01T08:09:10.000Z due to the poor floating point arithmetic in JavaScript.

I suggest rounding the milliseconds passed to the Date constructor.

The library currently always reads all records of a data page and then filters it using Array.slice. This means a performance loss, especially for large table records.

This should be improved such that only data is read that is actually returned.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore: update dependency vite to v5.0.13 [security]
• chore: update dependency @sveltejs/kit to v2.4.3 [security]
• chore: update actions/setup-node action to v4.0.2
• chore: update dev dependencies (non-major) (@sveltejs/adapter-auto, @sveltejs/vite-plugin-svelte, @tsconfig/node18, @tsconfig/strictest, @types/chai, @types/node, @typescript-eslint/eslint-plugin, @typescript-eslint/parser, chai, eslint, mocha, rimraf, semantic-release, svelte, svelte-check, typescript, vite-plugin-node-polyfills)
• chore: update dependency eslint to v9
• chore: update dependency semantic-release to v24
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore: update dependency @types/node to v20

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Hi,
Currently, I am using mdb-reader in the project, and after setting up Dependabot I receive an alert for a security issue that related to package fast-xml-parser. Can you update package fast-xml-parser to version 4.2.4?

Fields that contain over ~1893 characters are getting cut very short, returning a string that somewhat resembles what should have been returned. For example, if the long string started with "Version 1 ... " followed by large amounts of text, its returning "Versgr " and nothing else

I have a french MDB, using probably not an UTF-8 encoding. If I read the table 'tTour', i see:

{ NoTour: 1, NomTour: 'Engagement' }
{ NoTour: 2, NomTour: 'S�rie' }
{ NoTour: 3, NomTour: '16�me f.' }
{ NoTour: 4, NomTour: '8�me f.' }
{ NoTour: 5, NomTour: 'Quart f.' }
{ NoTour: 6, NomTour: 'Semi f.' }
{ NoTour: 7, NomTour: 'Finale'

I suspect an iso8859-15.

the mdb source:
https://u.pcloud.link/publink/show?code=XZnIlkVZLG3ffaEEef4XDRskx2F5qyBGbHYV

Is there an option to set the encoding source ?

Hi, I am trying to use this awesome package in Nestjs, but I receive this error, please help to solve this issue, thank you in advance.

const mdb_reader_1 = require("mdb-reader").default; ^ Error [ERR_REQUIRE_ESM]: require() of ES Module E:\api\node_modules\mdb-reader\lib\node\index.js from E:\api\dist\workshop\workshop.service.js not supported. Instead change the require of index.js in E:\api\dist\workshop\workshop.service.js to a dynamic import() which is available in all CommonJS modules. at Object.<anonymous> (E:\api\dist\workshop\workshop.service.js:12:22) at Object.<anonymous> (E:\api\dist\workshop\workshop.module.js:11:28) at Object.<anonymous> (E:\api\dist\app.module.js:17:27) at Object.<anonymous> (E:\api\dist\main.js:4:22)

I'm trying to read the table tRelais in this mdb:
https://u.pcloud.link/publink/show?code=XZu5EHVZVqot9BBBU7FMwQHIVSJ4zHIwB9i7

If I read this table with "sysinfotool MDB file viewer 20.0", I read a full table with no empty data. But, if I read with this library, I found many empty object:

{NoEngageRelais: 1487, NoEngagement: 44242, NoSerieTour: 8096, NoAthlete: 13310, NoOrdreRelais: null}
{NoEngageRelais: 1488, NoEngagement: 44242, NoSerieTour: 8096, NoAthlete: null, NoOrdreRelais: null} ...

The field NoOrdreRelais has the type octet in french language... It is a small int with 8bits.

My script:

function loadRelais (reader) {
  let cmtrelays = new Map()
  const table = reader.getTable('tRelais')
  let rows = table.getData()
  for (let row of rows) {
    console.log(row)
    let uid = row.NoEngageRelais
    let noEngagement = row.NoEngagement
    let serieTour = row.NoSerieTour
    let athUID = row.NoAthlete
    let ord = row.NoOrdreRelais
    cmtrelays.set(uid, { uid, noEngagement, serieTour, athUID, ord })
  }
  return cmtrelays
}

"typescript": "^3.1.6"

How to read password protected mdb files? Does your library support this or any clue to add this capability?

First of all, thank you for this great library. It is tremendously helpful for client-side exploration of MS Access databases.

I noticed that the browser-based example is using parcel to bundle the assets. You are probably aware of this, but it is possible to eliminate the build/bundling step by referencing a CDN. For example:

import { Buffer } from 'https://cdn.jsdelivr.net/npm/buffer@6/+esm'
import MDBReader from 'https://cdn.jsdelivr.net/npm/[email protected]/+esm' // @latest or @2 can be used instead

This could prove helpful for importing this library directly in the browser or other JavaScript runtimes that don't use a package manager.

• Jet
• MSISAM
• Office Agile
• Office Binary Doc RC4
• Office ECMA Standard Encryption
• Office RC4 Crypto API

I'm trying to use MDBReader in the browser for a project that I'm serving with Web Dev Server, but keep hitting the following error:

Error while handling server request.
PluginError: Could not resolve import "crypto".

I've also tried directly referencing the built browser version

import { MDBReader } from 'mdb-reader/lib/browser/index.js';

but this yields a different error in lib/browser/environment/index.js where browserifyAES fails to import because it doesn't provide a default export.

Am I importing something incorrectly in either scenario?

See #169

Hi, i am trying to run a script i wrote on a linux server but get an error (Windows works perfectly).

file:///root/mdb-reader/node_modules/mdb-reader/lib/node/MDBReader.js:14
        this.db = new Database(this.buffer, password ?? "");
                                                      ^

SyntaxError: Unexpected token '?'

Any idea how to fix this? Appeciate the help.

When I try to load this using the Browser example, I get:

Uncaught RangeError: offset is not uint
at checkOffset (index.js:1097:1)
at Uint8Array.readInt32LE (index.js:1220:1)
at readComplexOrLong (complexOrLong.js:2:1)
at readFieldValue (index.js:43:1)
at Table.getDataFromPage (Table.js:242:1)
at Table.getData (Table.js:132:1)
at new MDBReader (MDBReader.js:15:1)