Comments (8)
ppomes
commented on June 8, 2024
1
Thanks for the screenshot, I think I understand, and I have enough data for now ;)
from myanon.
ppomes
commented on June 8, 2024
I just added a warning on stderr, for each field declared in config file, but not found in dump.
Example:
ppomes@ubvm:~/src/myanon$ main/myanon -f tests/test1.conf < tests/test1.sql > /dev/null
WARNING! Field `lottypes`:`blob4` from config file has not been found in dump. Maybe a config file error?
Can you pull and try again? Thanks!
from myanon.
Mat-Moo
commented on June 8, 2024
Ok run this through - looks like we are getting some false positives
- table doesn't exist (maybe a different warning as table name could be wrong?)
- NULL values in fields
- table empty
All show errors when they probably shouldn't - but did correctly show the one I knew was wrong :)
from myanon.
ppomes
commented on June 8, 2024
Thanks for the feedback.
Currently, myanon stores internally all fields to anonymize in a hash with a key 'table:field': so for a non existing table, you should get a warning for all 'table:key' combinations matching the missing table. This explains your first point.
For the two other ones, is it possible to upload a small scenario? I will then have a look.
Thanks.
from myanon.
Mat-Moo
commented on June 8, 2024
2022-12-20 15:22:55 WARNING! Field `addresses`:`region` ## NO FIELD IN TABLE
2022-12-20 15:22:55 WARNING! Field `agencies`:`name` ## NO TABLE
2022-12-20 15:22:55 WARNING! Field `agencies`:`notes` ## NO TABLE
2022-12-20 15:22:55 WARNING! Field `communications`:`communication` ## NO TABLE
2022-12-20 15:22:55 WARNING! Field `company_telephone_number`:`number` ## TABLE EMPTY ??
2022-12-20 15:22:55 WARNING! Field `contacts`:`phone` ## FIELD NULL ??
2022-12-20 15:22:55 WARNING! Field `courier_jobs`:`job_number` ## TABLE EMPTY ??
2022-12-20 15:22:55 WARNING! Field `courier_jobs`:`notes` ## TABLE EMPTY ??
2022-12-20 15:22:55 WARNING! Field `paper_source_documentations`:`sent_notes` ## TABLE EMPTY ??
2022-12-20 15:22:55 WARNING! Field `participant_protocol_comment`:`comment` ## TABLE EMPTY ??
2022-12-20 15:22:55 WARNING! Field `participant_visit_comments`:`comment` ## TABLE EMPTY ??
2022-12-20 15:22:55 WARNING! Field `site_comment`:`comment` ## TABLE EMPTY ??
2022-12-20 15:22:55 WARNING! Field `stipends`:`name` ## NO TABLE
2022-12-20 15:22:55 WARNING! Field `stipends`:`note` ## NO TABLE
2022-12-20 15:22:55 WARNING! Field `stipends`:`cancellation_reason` ## NO TABLE
2022-12-20 15:22:55 WARNING! Field `studies`:`notificaton_email` ## TYPO notification_email
2022-12-20 15:22:55 WARNING! Field `training_attendees`:`name` ## TABLE EMPTY ??
2022-12-20 15:22:58 WARNING! Field `banks`:`sort_code` ## FIELD NULL ??
2022-12-20 15:22:58 WARNING! Field `banks`:`iban_number` ## FIELD NULL ??
So that's results and thinking of why errors may of occurred
from myanon.
Mat-Moo
commented on June 8, 2024
Hard to upload examples as all hidden in s3 buckets and automation - but I guess creating a table with no data, and config file to anon a column in it should be easy enough to create a test for?
from myanon.
ppomes
commented on June 8, 2024
Partial fix: warning should not be reported anymore on NULL fields. Fix is in "main" branch. Can you try again? Thanks!
from myanon.
Mat-Moo
commented on June 8, 2024
Looking good!
from myanon.
Related Issues (17)
- NULL values are not maintained for rows with both NULL and non-NULL values HOT 9
- Documentation HOT 1
- Feature request ability to use faker to generate more robust data HOT 4
- Generate fields with data from other fields HOT 19
- Blob values are not quoted HOT 3
- Using mysqldump with `--hex-blob` flag breaks myanon HOT 3
- Adding column names results in syntax error HOT 3
- Can't parse "set" HOT 1
- Fixed text truncated HOT 10
- Support for data-only dumps (--no-create-info) HOT 2
- Randomise the seed HOT 3
- Does fixed even replace if null? HOT 5
- Official Docker image HOT 3
- Support for --insert-ignore HOT 3
- Support for complex fields HOT 11
- README - Additional support for MacOS HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from myanon.