Code Monkey home page Code Monkey logo

dbanon's Introduction

dbanon

Release Status codecov

A run-anywhere, dependency-less database anonymizer.

Installation

Download the latest release from GitHub.

Usage

dbanon reads from stdin and writes to stdout.

mysqldump mydb | dbanon -config=myconfig.yml | gzip > mydb.sql.gz

The -config flag can use bundled configurations or point to the path of a custom configuration file.

Rules

Static

These are basic faking rules that don't accept any arguments or know/care about the current value of a cell.

first_name
last_name
full_name
company_name
username
password
ipv4
url
linkedin_url
md5
uuid
json
query_params
datetime
state
city
postcode
street
country_code
money
money_decimal
email
telephone

Dynamic

These are more advanced rules that accept arguments and can be impacted by the current value of the cell.

dynamic.email

arguments

  • ignoreEmail - Any emails with this suffix will not be faked.

example

rule = dyanmic.email(@test.com)
current cell value = [email protected]
after dbanon = [email protected]

rule = dyanmic.email(@test.com)
current cell value = [email protected]
after dbanon = [email protected]

Configuration

Magento 2

dbanon bundles a default Magento 2 configuration file. However you almost certainly won't use it directly.

At minimum, you'll first need to run the map-eav subcommand. This translates EAV attribute codes to their respective attribute ids.

You must feed it a mysqldump of eav_entity_type and eav_attribute (in that order).

mysqldump mydb eav_entity_type eav_attribute | dbanon -config=magento2 map-eav > ~/magento2-mapped.yml

map-eav will replace the attribute codes in the config file with attribute ids and print an updated config to stdout.

Next you'd run dbanon with the config generated by map-eav.

mysqldump mydb | dbanon -config=~/magento2-mapped.yml | gzip > mydb.sql.gz

Most Magento 2 databases, however, will have additional data that needs to be anonymized beyond the default bundled file.

For this you'll first want to create a new configuration file based off the bundled configuration. Instructions on customizing the configuration file are included in the "Custom Configuration" section.

Custom Configuration

Specify the path to your config file via the -config flag

mysqldump mydb | dbanon -config=myconfig.yml | gzip > mydb.sql.gz

See the etc directory for examples.

Columns are specified as key / value pairs. The value string winds up getting passed to this function, which gets random values from dmgk/faker.

It is also possible to pass direct Faker function calls for supported "raw providers"

Logging

dbanon records messages about anything notable (e.g. invalid configuration) to the file dbanon.log in the directory from which you run it.

-log-file

The -log-file flag can be used to have dbanon log to a different location.

mysqldump mydb | dbanon -config=myconfig.yml -log-file=var/dbanon.log

-log-level

The -log-level flag can be used to control the verbosity of logs. Supported values can be found here.

mysqldump mydb | dbanon -config=myconfig.yml -log-level=debug | gzip > mydb.sql.gz

The default log level is info.

-silent

Logging can be disabled entirely by passing the -silent flag to dbanon

mysqldump mydb | dbanon -config=myconfig.yml -silent | gzip > mydb.sql.gz

Profiling

dbanon will generate a CPU profile to the file dbanon.prof when passed the -profile flag.

mysqldump mydb | dbanon -profile -config=myconfig.yml >/dev/null

Limitations

  • Currently only supports MySQL

Updating

dbanon will self-update when passed the -update flag

dbanon -update

Development

Publishing releases

Releases are published with GoReleaser.

  • Create and push a tag (e.g. git tag -a v0.6.2 -m "v0.6.2 && git push origin v0.6.2)
  • Build the go-bindata (e.g. go-bindata -pkg bindata -o bindata/bindata.go etc/*)
  • Run goreleaser (e.g. goreleaser release --rm-dist)

May need to create a new GitHub token if the current one expires

https://goreleaser.com/scm/github/

dbanon's People

Contributors

github-actions[bot] avatar mdshack avatar mpchadwick avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.