Code Monkey home page Code Monkey logo

unpdf's Introduction

unpdf

A collection of utilities to work with PDFs. Designed specifically for Deno, workers and other nodeless environments.

unpdf ships with a serverless build/redistribution of Mozilla's PDF.js for serverless environments. Apart from some string replacements and mocks, unenv does the heavy lifting by converting Node.js specific code to be platform-agnostic. See pdfjs.rollup.config.ts for all the details.

This library is also intended as a modern alternative to the unmaintained but still popular pdf-parse.

Features

  • ๐Ÿ—๏ธ Works in Node.js, browser and workers
  • ๐Ÿชญ Includes serverless build of PDF.js (unpdf/pdfjs)
  • ๐Ÿ’ฌ Extract text and images from PDFs
  • ๐Ÿงฑ Opt-in to legacy PDF.js build
  • ๐Ÿ’จ Zero dependencies

PDF.js Compatibility

Note

This package is currently using PDF.js v4.0.189.

Installation

Run the following command to add unpdf to your project.

# pnpm
pnpm add unpdf

# npm
npm install unpdf

# yarn
yarn add unpdf

Usage

Extract Text From PDF

import { extractText, getDocumentProxy } from "unpdf";

// Fetch a PDF file from the web
const buffer = await fetch(
  "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
).then((res) => res.arrayBuffer());

// Or load it from the filesystem
const buffer = await readFile("./dummy.pdf");

// Load PDF from buffer
const pdf = await getDocumentProxy(new Uint8Array(buffer));
// Extract text from PDF
const { totalPages, text } = await extractText(pdf, { mergePages: true });

Access the PDF.js API

This will return the resolved PDF.js module and gives full access to the PDF.js API, like:

  • getDocument
  • version
  • โ€ฆ and all other methods

Especially useful for platforms like ๐Ÿฆ• Deno or if you want to use the PDF.js API directly. If no custom build was defined beforehand, the serverless build bundled with unpdf will be initialized.

import { getResolvedPDFJS } from "unpdf";

const { getDocument } = await getResolvedPDFJS();
const data = Deno.readFileSync("dummy.pdf");
const doc = await getDocument(data).promise;

console.log(await doc.getMetadata());

for (let i = 1; i <= doc.numPages; i++) {
  const page = await doc.getPage(i);
  const textContent = await page.getTextContent();
  const contents = textContent.items.map((item) => item.str).join(" ");
  console.log(contents);
}

Use Official or Legacy PDF.js Build

Generally speaking, you don't need to worry about the PDF.js build. unpdf ships with a serverless build of the latest PDF.js version. However, if you want to use the official PDF.js version or the legacy build, you can define a custom PDF.js module.

// Before using any other method, define the PDF.js module
// if you need another PDF.js build
import { configureUnPDF } from "unpdf";

configureUnPDF({
  // Use the official PDF.js build (make sure to install it first)
  pdfjs: () => import("pdfjs-dist"),
});

// Now, you can use the other methods
// โ€ฆ

Config

interface UnPDFConfiguration {
  /**
   * By default, UnPDF will use the latest version of PDF.js compiled for
   * serverless environments. If you want to use a different version, you can
   * provide a custom resolver function.
   *
   * @example
   * // Use the official PDF.js build (make sure to install it first)
   * () => import('pdfjs-dist')
   */
  pdfjs?: () => Promise<PDFJS>;
}

Methods

configureUnPDF

Define a custom PDF.js module, like the legacy build. Make sure to call this method before using any other methods.

function configureUnPDF(config: UnPDFConfiguration): Promise<void>;

getResolvedPDFJS

Returns the resolved PDF.js module. If no build is defined, the latest version will be initialized.

function getResolvedPDFJS(): Promise<PDFJS>;

getMeta

function getMeta(data: BinaryData | PDFDocumentProxy): Promise<{
  info: Record<string, any>;
  metadata: Record<string, any>;
}>;

extractText

Extracts all text from a PDF. If mergePages is set to true, the text of all pages will be merged into a single string. Otherwise, an array of strings for each page will be returned.

function extractText(
  data: BinaryData | PDFDocumentProxy,
  { mergePages }?: { mergePages?: boolean },
): Promise<{
  totalPages: number;
  text: string | string[];
}>;

renderPageAsImage

Note

This method will only work in Node.js and browser environments.

To render a PDF page as an image, you can use the renderPageAsImage method. This method will return an ArrayBuffer of the rendered image.

In order to use this method, you have to meet the following requirements:

  • Use the official PDF.js build
  • Install the canvas package in Node.js environments

Example

import { configureUnPDF, renderPageAsImage } from "unpdf";

configureUnPDF({
  // Use the official PDF.js build
  pdfjs: () => import("pdfjs-dist"),
});

const pdf = await readFile("./dummy.pdf");
const buffer = new Uint8Array(pdf);
const pageNumber = 1;

const result = await renderPageAsImage(buffer, pageNumber, {
  canvas: () => import("canvas"),
});
await writeFile("dummy-page-1.png", Buffer.from(result));

Type Declaration

declare function renderPageAsImage(
  data: BinaryData,
  pageNumber: number,
  options?: {
    canvas?: () => Promise<typeof import("canvas")>;
    /** @default 1 */
    scale?: number;
    width?: number;
    height?: number;
  },
): Promise<ArrayBuffer>;

FAQ

Why Is canvas An Optional Dependency?

The official PDF.js library depends on the canvas module for Node.js environments, which doesn't work inside worker threads. That's why unpdf ships with a serverless build of PDF.js that mocks the canvas module.

However, to render PDF pages as images in Node.js environments, you need to install the canvas module. That's why it is a peer dependency.

License

MIT License ยฉ 2023-PRESENT Johann Schopplich

unpdf's People

Contributors

eltigerchino avatar johannschopplich avatar moinulmoin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

unpdf's Issues

Missing `pdfjs-dist` types

Environment

  • unpdf v0.10.1
  • node v18.19.0

Reproduction

The types should be exported here so we can use them.

import * as PDFJS from './types/src/pdf'
declare function resolvePDFJS(): Promise<typeof PDFJS>
export { resolvePDFJS }

Hence, the currently generated declaration file looks like this:

import * as PDFJS from './types/src/pdf'
declare function resolvePDFJS(): Promise<typeof PDFJS>
export { resolvePDFJS } // no types are included

unpdf/pdfjs type not exported error

Describe the bug

The types are not exported together with unpdf/pdfjs. This prevents typing variables / function params when composing with the library.

Additional context

No response

Logs

No response

Struggling to get it to work in a Supabase Edge Function

Environment

Using via esm: https://esm.sh/[email protected]/
Deno version: 1.38.4 (I'm guessing this because it's not easy to see the supabase edge function environment, but they say they use the latest stable version.

Reproduction

Sorry, it's a bit hard to reproduce since it's only failing in the deployed Supabase edge function. When I run it locally in my docker container, it works fine. Here's the relevant code though of my edge function:

import { configureUnPDF, getResolvedPDFJS } from 'https://esm.sh/[email protected]';
import * as pdfjs from 'https://esm.sh/[email protected]/dist/pdfjs.mjs';

configureUnPDF({
  // deno-lint-ignore require-await
  pdfjs: async () => pdfjs,
});
const resolvedPdfJs = await getResolvedPDFJS();
const { getDocument } = resolvedPdfJs;

export async function convertPdfToText(
  arrayBuffer: ArrayBuffer
): Promise<string> {
  try {
    const data = new Uint8Array(arrayBuffer);

    // Get the document
    const doc = await getDocument(data).promise;
    let allText = '';

    // Iterate through each page of the document
    for (let i = 1; i <= doc.numPages; i++) {
      const page = await doc.getPage(i);
      const textContent = await page.getTextContent();

      // Combine the text items with a space (adjust as needed)
      const pageText = textContent.items
        .map((item) => {
          if ('str' in item) {
            return item.str;
          }
          return '';
        })
        .join(' ');
      allText += pageText + '\n'; // Add a newline after each page's text
    }

    return allText;
  } catch (error) {
    console.error('Error converting PDF to text', error);
    throw error;
  }
}

Describe the bug

In the supabase edge functions log, it consistently throws this error:

event loop error: Error: PDF.js is not available. Please add the package as a dependency.
    at f (https://esm.sh/v135/[email protected]/deno/unpdf.mjs:2:574)
    at async h (https://esm.sh/v135/[email protected]/deno/unpdf.mjs:2:230)
    at async file:///home/runner/work/tl.ai/tl.ai/supabase/functions/process/index.ts:12:23

Originally, I followed the base setup instructions. Then, I tried to use getResolvedPDFJS. Finally, I tried to first configureUnPDF and pointing pdfjs specifically to the one exported from your package. However, all still failed in the production environment.

I'm mainly wondering if I'm not following the instructions correctly for configuring pdfjs. Thanks in advance for your help!

Additional context

No response

Logs

No response

Unpdf can't render pages with images

Environment

Node.js v20.9.0
PNPM v8.10.0
UnPDF v0.10.0

Reproduction

Example Code: CodeSandBox

Describe the bug

When I am ready to render a page with unpdf, renderPageAsImage will report an error if there is an image embedded in the pdf.

Additional context

No response

Logs

TypeError: r.createCanvas is not a function
    at NodeCanvasFactory._createCanvas (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1316062)
    at NodeCanvasFactory.create (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1153693)
    at CachedCanvases.getCanvas (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1163909)
    at CanvasGraphics.paintInlineImageXObject (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1198408)
    at CanvasGraphics.paintImageXObject (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1197232)
    at CanvasGraphics.executeOperatorList (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1172983)
    at InternalRenderTask._next (file:///workspaces/workspace/node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/pdfjs.mjs:1:1152632)

Does not work in BGSW using Plasmo framework

Environment

Framework: Plasmo 0.84.0
Client side Chrome Browser Extension

Reproduction

Can be reproduced by creating a BGSW in the Plasmo framework and importing unpdf. Error message is:

๐Ÿ”ด ERROR | Build failed. To debug, run plasmo dev --verbose.
๐Ÿ”ด ERROR | Failed to resolve 'unpdf/pdfjs' from './node_modules/.pnpm/[email protected]/node_modules/unpdf/dist/index.mjs'

Describe the bug

Bug is as aforementioned: unpdf seems to be looking for a pdfjs dependency that is inaccessible.

Additional context

No response

Logs

No response

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.