generalpiston / typeorm-encrypted Goto Github PK

• Description
• Examples
• Usage details
• Installation details
• Usage details in JavaScript
• Automate test running
• Badges from: https://david-dm.org/, https://circleci.com/, and https://badge.fury.io/

Currently, using the AutoEncryptSubscriber automatically encrypts and decrypts fields. It would be nice to decompose these 2 and make encryption/decryption configurable.

Secure key generation is a topic of research. We can provide pseudo-secure methods in the README to help developers get started at least:

• https://www.openssl.org/docs/man1.0.2/man1/openssl-rand.html
• https://stackoverflow.com/questions/40277918/shell-script-to-generate-random-hex-numbers

We can also add resources for folks to learn more about it.

Getting this error when trying to update data inside table.

@Column({
    type: 'varchar',
    nullable: true,
    select: false,
    transformer: new EncryptionTransformer({
       key: process.env.ENCRYPTION_KEY,
       algorithm: 'aes-256-gcm',
       ivLength: 16,
    }),
  })
  secret: string;

repository.save(data)

When I comment out transformer inside column definition it passes without error.

I was trying to reproduce your examples using column transformer (just copy/paste your code) and got the error: Invalid IV length

I was able to create a new entity, but when I tried to get the data, I got that error. Any hint about this, please?

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch [email protected] for the project I'm working on.

Here is the diff that solved my problem:

diff --git a/node_modules/typeorm-encrypted/src/transformer.ts b/node_modules/typeorm-encrypted/src/transformer.ts
index ec5d1c8..f4e4d63 100644
--- a/node_modules/typeorm-encrypted/src/transformer.ts
+++ b/node_modules/typeorm-encrypted/src/transformer.ts
@@ -5,7 +5,7 @@ import { decryptData, encryptData } from './crypto';
 export class EncryptionTransformer implements ValueTransformer {
   constructor(private options: EncryptionOptions) {}
 
-  public from(value?: string | null): string | undefined {
+  public from(value?: string | null | object): string | undefined {
     if (!value) {
       return;
     }
@@ -55,3 +55,60 @@ export class EncryptionTransformer implements ValueTransformer {
     }
   }
 }
+export class JSONEncryptionTransformer implements ValueTransformer {
+  constructor(private options: EncryptionOptions) {}
+
+  public from(value?: string | null | object): string | undefined {
+    if (!value) {
+      return;
+    }
+
+    const decrypted = decryptData(
+      Buffer.from(value as string, 'base64'),
+      this.options
+    ).toString('utf8');
+
+    return JSON.parse(decrypted);
+  }
+
+  public to(value?: any | FindOperator<any> | null): string | FindOperator<any> | undefined{
+    if ((value ?? null) === null) {
+      return;
+    }
+
+    if (typeof value === 'object' && !value.type) {
+      return encryptData(
+        Buffer.from(JSON.stringify(value) as string, 'utf8'),
+        this.options
+      ).toString('base64');
+    }
+
+    if (!value) {
+      return;
+    }
+    // Support FindOperator.
+    // Just support "Equal", "In", "Not", and "IsNull".
+    // Other operators aren't work correctly, because values are encrypted on the db.
+    if (value.type === `in`) {
+      return In((value.value as string[]).map(s =>
+        encryptData(
+          Buffer.from(s, 'utf-8'),
+          this.options
+        ).toString('base64')
+      ));
+    } else if (value.type === 'equal') {
+      return Equal(encryptData(
+        Buffer.from(value.value as string, 'utf-8'),
+        this.options
+      ).toString('base64'));
+    } else if (value.type === 'not') {
+      return Not(
+        this.to(value.child ?? value.value)
+      );
+    } else if (value.type === 'isNull') {
+      return value
+    } else {
+      throw new Error('Only "Equal","In", "Not", and "IsNull" are supported for FindOperator');
+    }
+  }
+}

This issue body was partially generated by patch-package.

It doesn't work when i want to get value by QueryBuilder

this.userRepository
      .createQueryBuilder('user')
      .where('user.email = :email', { email })
      .getOne();

  @Column({
    type: 'varchar',
    nullable: false,
    transformer: new EncryptionTransformer({
      key: 'e41c966f21f9e1577802463f8924e6a3fe3e9751f201304213b2f845d8841d61',
      algorithm: 'aes-256-cbc',
      ivLength: 16,
      iv: 'ff5ac19190424b1d88f9419ef949ae56',
    }),
  })
  email: string;

Hello ,

i have database requirement is fully encrypted database to store i also requirement to filter all record so can it is possible to do a search operation with encrypted database using typeorm?

if it is really possible so give your throat how we have do it ?

To use 'where in' clause, I execute next code. But TypeError is thrown.
Are there any way to avoid this?

import { In } from "typeorm"

const repo = getRepository<XXX>()

await repo.find({
  where: {
    encryptedField: In(["aaa"])
  }
})

Error

TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received an instance of FindOperator

      at EncryptionTransformer.Object.<anonymous>.EncryptionTransformer.to (node_modules/typeorm-encrypted/src/transformer.ts:28:4)
      at Function.Object.<anonymous>.ApplyValueTransformers.transformTo (src/util/ApplyValueTransformers.ts:19:28)
      at ColumnMetadata.Object.<anonymous>.ColumnMetadata.getEntityValue (src/metadata/ColumnMetadata.ts:658:44)
      at SelectQueryBuilder.<anonymous> (src/query-builder/QueryBuilder.ts:1088:51)
      at step (node_modules/typeorm/node_modules/tslib/tslib.js:143:27)
      at Object.next (node_modules/typeorm/node_modules/tslib/tslib.js:124:57)
      at SelectQueryBuilder.Object.<anonymous>.QueryBuilder.getWhereCondition (src/query-builder/QueryBuilder.ts:1208:80)
      at SelectQueryBuilder.Object.<anonymous>.SelectQueryBuilder.where (src/query-builder/SelectQueryBuilder.ts:721:32)

Hey there!

First of all, wanna say thanks for all your hard work in maintaining this project! it is awesome 👏🏻

I am wondering if there is a way to manage encryption keys retention overtime?
Let's say I have both "old"-"new" encryption keys pair and want to update my database data,
eg encrypt data with a new key. Is there a (relatively) simple way how to do keys retention properly?

Thanks

Issue type:
[x] bug report
[x] documentation issue

Database system/driver:
[x] mysql / mariadb

TypeORM version:
[x] latest - 0.2.22

Steps to reproduce or a small repository showing the problem:

 @Override("createOneBase")
  async createOne(@ParsedRequest() req: CrudRequest, @ParsedBody() dto: Resource) {
    const location = new Location();
    location.name = "locationone";
    await getConnection().manager.save(location);
    const resource = new Resource();
    resource.color = "dogs";
    resource.locations = [location];
    await getConnection().manager.save(resource);

  }

I added a console log in the encrypt method of typeorm-encrypted/lib/transformers/index.js and here's the results

// Line 8 and 9 of index.js
//function encrypt(entity) {
//    console.log('entity', entity);


entity Location { name: 'locationname' }

entity Resource {
  color: 'dogs',
  locations: [
    Location {
      name: 'locationname',
      address: null,
      id: 15,
      createdAt: 2020-02-25T18:52:48.358Z,
      updatedAt: 2020-02-25T18:52:48.358Z
    }
  ]
}
entity undefined

// end of log for typeorm-encrypted/lib/transformers/index.js

Here's the error I get:

TypeError: Cannot read property 'constructor' of undefined
    at Object.encrypt (/Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/typeorm-encrypted/lib/transformers/index.js:15:83)
    at AutoEncryptSubscriber.beforeInsert (/Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/typeorm-encrypted/lib/subscribers/AutoEncryptSubscriber.js:18:24)
    at /Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/typeorm/subscriber/Broadcaster.js:39:54
    at Array.forEach (<anonymous>)
    at Broadcaster.broadcastBeforeInsertEvent (/Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/typeorm/subscriber/Broadcaster.js:37:53)
    at /Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/typeorm/persistence/SubjectExecutor.js:162:99
    at Array.forEach (<anonymous>)
    at SubjectExecutor.broadcastBeforeEventsForAll (/Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/typeorm/persistence/SubjectExecutor.js:162:33)
    at SubjectExecutor.<anonymous> (/Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/typeorm/persistence/SubjectExecutor.js:68:50)
    at step (/Users/natedeazy/projects/MedOneScheduling/scheduling-backend/node_modules/tslib/tslib.js:136:27)

Hello,

I do :

export const MyEncryptionTransformerConfig = {
    key: 'e41c966f21f9e1577802463f8924e6a3fe3e9751f201304213b2f845d8841d61',
    algorithm: 'aes-256-gcm',
    ivLength: 16,
};

It encrypt but when I would display if failed.....
Error : Unsupported state or unable to authenticate data

Thanks

When attempting to use this library with the Data Mapper pattern, the encryption options are ignored. This is because of the following line in the transformer: entity.constructor === target && mode == "regular". When an object is passed into the save/create function, constructor does not exist.

Example:

@Entity()
class User {
    @Column(<ExtendedColumnOptions>{ type: 'varchar', name: 'name_first',
    encrypt: {
      key: "e41c966f21f9e1577802463f8924e6a3fe3e9751f201304213b2f845d8841d61",
      algorithm: "aes-256-cbc",
      ivLength: 16,
      iv: "ff5ac19190424b1d88f9419ef949ae56"
    }}) 
    firstName: string

    @Column('text', {name: 'name_last'}) 
    lastName: string
}

const userRepository = connection.getRepository(User);

await userRepository.save({ firstName: 'John', lastName: 'Smith'}) 

console.log(await userRepository.find()) //No results

Use declaration merging instead of extending @ColumnOptions.

See https://www.typescriptlang.org/docs/handbook/declaration-merging.html for more info.

I am trying to use as transformer, its not working.

Hello,

When I generate key I see error : "Invalid key length"

I do for exemple :

export const MyEncryptionTransformerConfig = {
    key: Buffer.from(env.app.crypto.key, 'utf8').toString('base64'),
    algorithm: env.app.crypto.algorithm,
    ivLength: 16,
};

In env I add :
env.app.crypto.key = nCM*R$izc*G3vqg7jo086mbc9Rg^7NYl

Thanks

Hello,

In a scenario where I have 2 applications (let's call them A and B) needing to share the same encrypted data - would it be possible to have application B decrypt something encrypted by application A?

I assume that the IV/Auth tag generated during encryption are stored in the cipher text. Looking at the source code I can sort of identify where they are located in the cipher text, but in practice I'm not sure how to go about actually extracting them to use them for decryption.

I know this would be a highly questionable practice and something not recommended from a security perspective. I'm trying to weigh options and was wondering if this one is at all a possibility.

Thank you.

I have encrypted a column email but I can't find user that have the same email because it gives a different encryption. Is there a way to bypass that ?

I tried something like this

But without a chance.

Hi! 👋

Firstly, thanks for your work on this project! 🙂

Today I used patch-package to patch [email protected] for the project I'm working on.

Here is the diff that solved my problem:

diff --git a/node_modules/typeorm-encrypted/src/transformer.ts b/node_modules/typeorm-encrypted/src/transformer.ts
index ec5d1c8..f5f3035 100644
--- a/node_modules/typeorm-encrypted/src/transformer.ts
+++ b/node_modules/typeorm-encrypted/src/transformer.ts
@@ -2,30 +2,88 @@ import { ValueTransformer, FindOperator, In, Equal, Not } from 'typeorm';
 import { EncryptionOptions } from './options';
 import { decryptData, encryptData } from './crypto';
 
-export class EncryptionTransformer implements ValueTransformer {
+// export class EncryptionTransformer implements ValueTransformer {
+//   constructor(private options: EncryptionOptions) {}
+
+//   public from(value?: string | null | object): string | undefined {
+//     if (!value) {
+//       return;
+//     }
+
+//     return decryptData(
+//       Buffer.from(value as string, 'base64'),
+//       this.options
+//     ).toString('utf8');
+//   }
+
+//   public to(value?: string | FindOperator<any> | null): string | FindOperator<any> | undefined {
+//     if ((value ?? null) === null) {
+//       return;
+//     }
+//     if (typeof value === 'string') {
+//       return encryptData(
+//         Buffer.from(value as string, 'utf8'),
+//         this.options
+//       ).toString('base64');
+//     }
+//     if (!value) {
+//       return;
+//     }
+//     // Support FindOperator.
+//     // Just support "Equal", "In", "Not", and "IsNull".
+//     // Other operators aren't work correctly, because values are encrypted on the db.
+//     if (value.type === `in`) {
+//       return In((value.value as string[]).map(s =>
+//         encryptData(
+//           Buffer.from(s, 'utf-8'),
+//           this.options
+//         ).toString('base64')
+//       ));
+//     } else if (value.type === 'equal') {
+//       return Equal(encryptData(
+//         Buffer.from(value.value as string, 'utf-8'),
+//         this.options
+//       ).toString('base64'));
+//     } else if (value.type === 'not') {
+//       return Not(
+//         this.to(value.child ?? value.value)
+//       );
+//     } else if (value.type === 'isNull') {
+//       return value
+//     } else {
+//       throw new Error('Only "Equal","In", "Not", and "IsNull" are supported for FindOperator');
+//     }
+//   }
+// }
+
+export class JSONEncryptionTransformer implements ValueTransformer {
   constructor(private options: EncryptionOptions) {}
 
-  public from(value?: string | null): string | undefined {
+  public from(value?: string | null | object): string | undefined {
     if (!value) {
       return;
     }
 
-    return decryptData(
+    const decrypted = decryptData(
       Buffer.from(value as string, 'base64'),
       this.options
     ).toString('utf8');
+
+    return JSON.parse(decrypted);
   }
 
-  public to(value?: string | FindOperator<any> | null): string | FindOperator<any> | undefined {
+  public to(value?: any | FindOperator<any> | null): string | FindOperator<any> | undefined{
     if ((value ?? null) === null) {
       return;
     }
-    if (typeof value === 'string') {
+
+    if (typeof value === 'object' && !value.type) {
       return encryptData(
-        Buffer.from(value as string, 'utf8'),
+        Buffer.from(JSON.stringify(value) as string, 'utf8'),
         this.options
       ).toString('base64');
     }
+
     if (!value) {
       return;
     }

This issue body was partially generated by patch-package.

The example in readme for transformers doesnot work. getting invalid iv length

Hello

When I do like this :

this.query->createQuery('table')
.select('SUBSTRING(table.cp, 0, 2)', 'cp')
.andSelect('COUNT(table.id)', 'nb')
....

That return good number (nb) but the result is returned (cp) the two letters but not decrypred ...
Exemple have in CP value 75011 (crypted: start by pM) and return pM not 75.

Thanks

The current implementation of the transformer returns undefined instead of null when the column is null.
Will it be better if it stick to the same return type as Typeorm.

public from(value?: string | null): string | undefined {
    if (!value) {
      return;     // Should it return value instead?
    }
    return decryptData(
      Buffer.from(value as string, 'base64'),
      this.options
    ).toString('utf8');
  }

Hello. I'm having problem with typeorm version 0.3 as typeorm-encrypted has peer dependencies. Is it possible to adjust this?

See for more information #25.

As per example:

import { EncryptionTransformer } from 'typeorm-encrypted'

Is not possible, and one must do:

import { EncryptionTransformer } from 'typeorm-encrypted/lib/transformer'

I just ran into a problem that I've spent a little while diagnosing. I've forked the repo and gone to make my edits for a PR and found that the current master branch contains the fix I need.

Any chance of getting the master branch published to npm?

typeorm has embedded entities (http://typeorm.io/#/embedded-entities) which allows for custom type definitions. If listeners can operate on embedded entities, this would negate the need for decorators and complicated field search code.

Example defintion:

import { Column, AfterLoad, BeforeInsert, BeforeUpdate } from "typeorm";

import { encryptData, decryptData } from "../transformers";
import { IEncryptedEntityOptions } from "./IEncryptedEntityOptions";

export class EncryptedEntity {
  options: IEncryptedEntityOptions;

  @Column({
    type: "varchar",
    nullable: false
  })
  data: string;

  EncryptedEntity(options: IEncryptedEntityOptions) {
    this.options = options;
  }

  @AfterLoad()
  decryptFields(): void {
    if (this.options.autoencrypt) {
      this.data = decryptData(this.data);
    }
  }

  @BeforeInsert()
  encryptFieldsBeforeInsert(): void {
    if (this.options.autoencrypt) {
      this.data = encryptData(this.data);
    }
  }

  @BeforeUpdate()
  encryptFieldsBeforeUpdate(): Promise<this> {
    if (this.options.autoencrypt) {
      this.data = encryptData(this.data);
    }
  }
}

See openssl list-cipher-algorithms for a complete list. Here's the abbreviated shortlist:

• aes
• des
• bf

Why does this library use subscribers rather than plain transformers in typeorm? Was it because this came out before transformers were introduced into typeorm?

• I would suggest you to avoid forcing users to extend EncryptedBaseEntity class if possible. Usually users don't like to extend third party classes or change their model inheritance tree. (just imagine if there is another libs who force to extend their entities too and user uses them all)
• @EncryptedColumn looks fine, its not a valid from typeorm point of view because this decorator is not creating a new column (like all Column decorators in typeorm), it just makes it encrypted. I would name it simply @Encrypted instead.
• Also, its no a suggestion, just a hint most users don't know. You can override ColumnOptions properties and do it alternatively this way (it will be type-safe):

@Column({
    type: "varchar",
    nullable: false,
    encrypt: {
        key: "d85117047fd06d3afa79b6e44ee3a52eb426fc24c3a2e3667732e8da0342b4da",
        algorithm: "aes-256-cbc",
        ivLength: 16
    }
  })

When you specify column like this:

@Column({
    default: 'TEST_DATA',
    transformer: new EncryptionTransformer({
      key: '<key>',
      algorithm: 'aes-128-cbc',
      ivLength: 16,
    }),
  })
  encryptedColumn: string;

It inserts exact value that is specified in the default field instead of encrypted one.

I'm testing it with autoLoadEntities === true

Is there a way to query based on partial words in encrypted data?

user.find({
  where: { email: ILike('%' + search + '%') }
})

Circle CI tests are failing, but tests run locally. Figure out why.

There are 3 configurations to figure:

1. Auto encrypt/decrypt
2. encrypt/decrypt from utilities

Noticed the recent dependabot PR merges but your NPM repo was updated 8 months ago. Has this project been abandoned? Are you planning to push these changes to NPM any time soon?

Just wondering because it looks like a fantastic project.

Hey, if one is using key rotation for improved security, how does that affect the use of this package?

@entity()
export class UserEntity extends BaseEntity {
@PrimaryGeneratedColumn()
id: number;
@column()
name: string;
@column()
age: string;
@column({
type: "varchar",
nullable: false,
unique: true,
transformer: new EncryptionTransformer({
key: 'e41c966f21f9e1577802463f8924e6a3fe3e9751f201304213b2f845d8841d61',
algorithm: 'aes-256-gcm',
ivLength: 16
})
})
email: string;
} in this case it always generate new cyper even if the email is same expected it need to throw error of duplicate please take a look at this issue

When adding the transformer to an existing table, a data migration is needed. See #19 (comment) for more detail.

• Add a key to an unencrypted column that needs to be encrypted.
• Swap keys on an existing encrypted column

https://github.com/typeorm/typeorm/blob/master/CHANGELOG.md

Several users have asked for full database encryption via typeorm-encrypted. Let's point it out in the FAQ that we don't do that and explain where they can find more info on it.

• Remove unnecessary files from package
• Ensure versions
• Verify tests pass
• Tag release

I currently have a problem when I run the npm run start, I'm not quite sure if it's a problem with the typescript setting

error TS2307: Cannot find module 'typeorm-encrypted' or its corresponding type declarations.

tsconfig.json

{
  "compilerOptions": {
    "module": "commonjs",
    "declaration": true,
    "removeComments": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "allowSyntheticDefaultImports": true,
    "target": "es2017",
    "sourceMap": true,
    "outDir": "./dist",
    "baseUrl": "./",
    "incremental": true
  }
}                                            

I would like a way to conditionally encrypt columns

specifically I have a key value table (used to store in this case some sort of preferences). Dependant on the value of the key the value colums should or shouldn't be encrypted.

I image that a callback with the entity as argument could be added to the options that enables or disabled the encryption without much trouble?

Would such a feature be appreciated?

encrypt: {
      key: "d85117047fd06d3afa79b6e44ee3a52eb426fc24c3a2e3667732e8da0342b4da",
      algorithm: "aes-256-gcm",
      ivLength: 16,
      enabled: (entity: MyEntity) => entity.key === 'apiKey',
    }