Adding data type to encrypted column in AcraServer encryption configuration about acra HOT 15 CLOSED

just for future reference - fix to this (fully transparent mode without all those type concerns) is coming in 0.92-0.93, with limited scope first (not all types will be supported, but most). I will update this issue when that happens.

from acra.

@kumar1202 work in progress. That's not the easiest thing to do considering all the features of SQL protocol. We have progress with PostgreSQL, working on MySQL now. Type awareness is scheduled to the nearest releases.

from acra.

Tadam! 🍉

Acra 0.93 release brings Type Awareness!

This release brings type awareness which improves transparent encryption on AcraServer. Type awareness means that it's possible to tell AcraServer what are the original data types for fields. During decryption, AcraServer will convert decrypted fields to their original data types. No need to change client application code to work with "binary data".

It's also possible to choose a default value for each data field if its decryption failed. AcraServer can send a a default value like "<encrypted data>" instead of decryption errors, making developers' and users' life easier.
https://github.com/cossacklabs/acra/releases/tag/0.93.0

Usage examples are in the acra engineering demo repository
https://github.com/cossacklabs/acra-engineering-demo/

Here is how configuration files looks like:
https://github.com/cossacklabs/acra-engineering-demo/blob/master/django-transparent/acra-server-configs/encryptor_config.yaml#L20

Docs:
https://docs.cossacklabs.com/acra/configuring-maintaining/general-configuration/acra-server/encryptor-config/
https://docs.cossacklabs.com/acra/guides/integrating-acra-server-into-infrastructure/

It might look like a simple feature, but under the hood it required to significantly improve SQL parser and handle edge cases that modern ORMs bring to the table (when they hide SQL from end developers, making their life easier, but our life more complicated).

cc @wdesplas @pvleap @kumar1202

Kudos to Acra maintainers: @Lagovas @ZhmakaAS @G1gg1L3s @iamnotacake @shadinua @9gunpi and @vixentael :)

from acra.

@wdesplas totally makes sense, thank you for your request.

Adding data type to the AcraServer's encryption configuration file – that's a feature in our backlog.

If your concern is related to wording, will changing this line to "without much altering the application code" help?

from acra.

@vixentael :
Thansk a lot for you reply.
Your sentence make sense however, In my case, as a telecom compagny Engineer, I'm not able to change any code line in the applications.
This is the only feature that does not allow us to make acra working in our environement to move forward to adopt this solution.

regards,

from acra.

I am facing the same issue trying to transparently encrypt 2 columns of a postgres table and these columns are of postgres datatype text and hold simple text values. The result returned for my select queries is the hex encoded string. Please let me know if there is any other config file where I can specify the data type of these columns and also if i can specify the dialect of my postgres db?

Also I keep seeing the error "ignoring error of non parsed sql statement" in the acra server logs though there is no one connecting to my acra server. Is it continously polling my encryptor config file and framing a select query against the postgres db with the columns configured in there?

from acra.

@wdesplas , @pvleap, recently we have released a new version of Acra with new features like transparent tokenization that supports string/text types in databases. You can find out more details on our documentation site about configuration AcraServer's encryptor config and about tokenization

from acra.

Hey @9gunpi, any updates on the progress of full transparent mode with types support?

from acra.

Hey @vixentael, is this feature supported in Enterprise edition? As I was reading through the comparisons table, and found that format preserving encryption is supported in EE.

from acra.

@kumar1202 format preserving encryption is different from type awareness.

Type awareness means that it's possible to "tell" AcraServer the original data types of the columns. During encryption, AcraServer will encrypt fields to binary, the database will store fields as binary. During decryption, AcraServer will decrypt fields and cast them to the original data type, so that the application receives decrypted data fields as expected in most cases. Transparent for app, minimum code changes.

Type awareness is a big feature for the upcoming 0.93 release which is scheduled before end of May. It might happen next week, if we are lucky and all edge cases are found, or later in May, if we find more ORMs that behave weird. The feature is ready, we are hunting edge cases and updating playgrounds.

This is what this issue is about.

Type awareness will be available for both Acra CE and Acra EE.

Acra provides different security controls that suit best for different use cases, security guarantees and tech limitations. For some cases, encryption is enough, for others tokenization + encryption, or even masking + tokenization + encryption. Our goal here to give understandable tools with high security guarantees without requiring companies to actually send their sensitive data to a "magic saas".

from acra.

Actually, you can see how type awareness will work if you look at these PRs:

https://github.com/cossacklabs/acra-engineering-demo/pull/47/files
https://github.com/cossacklabs/acra-engineering-demo/pull/50/files

These PRs are updating Acra's engineering examples, and we are waiting for 0.93 release before merging them. As I mentioned, type awareness is working for 90% use cases right now, while we are fighting edge cases and interesting SQL protocol caveats trying to release as stable build as possible.

from acra.

hey, @vixentael
I have few doubts regarding tokenization and type awareness.

1. Is tokenization not available in community edition?
2. How does the de-tokenization works? How do we store token's mapping to the original data? documentation mentions the following statement: Both tokenization types require deploying an additional database, Redis by default, to store pairs: token <-> encrypted data..
   Would Acra encrypt the token <-> encrypted data mapping in redis as well?
3. With type awareness in place, the only change that we need to do on the application side is the change of data type of the column to be encrypted to bytea via db migration?

from acra.

hey, @vixentael I have few doubts regarding tokenization and type awareness.

1. Is tokenization not available in community edition?
2. How does the de-tokenization works? How do we store token's mapping to the original data? documentation mentions the following statement: Both tokenization types require deploying an additional database, Redis by default, to store pairs: token <-> encrypted data..
   Would Acra encrypt the token <-> encrypted data mapping in redis as well?
3. With type awareness in place, the only change that we need to do on the application side is the change of data type of the column to be encrypted to bytea via db migration?

Let me answer:

1. It is available in community edition
2. Acra encrypts data, generates new value and store them in the token db as <hash(token)>: <encrypted_data>. So when it receive token, it can find source encrypted value, decrypt and return decrypted
3. Yes, you are right.

from acra.

@Lagovas Thanks for the reply.
As you mentioned, tokenization is available in community edition, does this implies that format preserving encryption feature is available in community edition as well? CMIIW, preserving the format of the data column is made possible using tokenisation feature, right?

from acra.

@kpsinghlubana let's step back from the definitions and discuss your use case. what exactly are you trying to do?

yes, tokenization allows to leave the format of original data column, put a non-sensitive token there, but place a real encrypted data into separate column/database.

from acra.