davidmreed / amaxa Goto Github PK
View Code? Open in Web Editor NEWA multi-object ETL tool for Salesforce.
Home Page: https://amaxa.readthedocs.io
License: BSD 3-Clause "New" or "Revised" License
A multi-object ETL tool for Salesforce.
Home Page: https://amaxa.readthedocs.io
License: BSD 3-Clause "New" or "Revised" License
๐ Hi, I learned about your project via your tweet.
This looks interesting. I have a use case where I need to copy some parent-child relationships from one org to another and perform a transform step to append a suffix to some text fields.
I didn't readily see how in the docs to perform a transform other than uppercase, lowercase, or strip.
Are there other transform capabilities to further change values?
Thanks
May be subsumed into #8
fields
should take precedence over field-group
, allowing tailoring of options at the level of the individual field.
Behavior if fields
and field-group
are disjoint to be defined.
We'd like to make it possible to extend Amaxa's transform system with simple pluggable modules.
Initial design sketch:
Genesis of this proposal is discussion between David and Kyle Crouse about the infrastructure needed to support 3rd-party data obfuscation capabilities.
See also #29.
I've tried an extraction where a base object is defined in the beginning of the op file with the extract: ids:
attribute. However, I also need to extract this object down the line a a child object. Listing the same object further down in the op file, this time with the extract: descendents: true
attribute did not work as expected - the only extracted record of that object was the anchor record. Is this a limitation or a bug?
Thanks!
Eran
On Windows 10,
When try to extract the data (Case object) using smart, get the following error:
Case: starting extraction..
Unexpected exception 'charmap' codec can't encode character '\u0308' in position 3736: character maps to <undefined> occurred.
Please suggest solution for this issue.
[Submitted by Kyle Crouse on GitLab]
This may be out of scope for this tool, but it could be useful to be able to obfuscate sensitive data prior to loading (or even during the export so that the sensitive data is never even saved to disk). Something like Faker coupled with the SF field metadata would make it fairly easy to generate realistic fake data.
If you do think it's out of scope for the tool, maybe some sort of plugin system for transforms could allow it to live on its own? Obviously neither the obfuscation transform nor the plugin system is a small undertaking, but I think they could make this tool even more useful.
The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.
Amaxa should allow CumulusCI authentication, as it does with Salesforce DX.
When running on Windows the following error is received when using sfdx credentials:
[DBG]: PS C:\Users\Will> amaxa .\data\data-move.yml --credentials .\data\cred.yml
amaxa : Traceback (most recent call last):
At line:1 char:1
+ amaxa .\data\data-move.yml --credentials .\data\cred.yml
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (Traceback (most recent call last)::String) [], RemoteException
+ FullyQualifiedErrorId : NativeCommandError
File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\runpy.py", line 193, in _run_module_as_main
File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\runpy.py", line 85, in _run_code
File "C:\portapps\Python\Scripts\amaxa.exe\__main__.py", line 9, in <module>
File "c:\portapps\python\lib\site-packages\amaxa\__main__.py", line 55, in main
credential_loader.load()
File "c:\portapps\python\lib\site-packages\amaxa\loader\core.py", line 67, in load
step()
File "c:\portapps\python\lib\site-packages\amaxa\loader\credentials.py", line 20, in _load
self._load_v2()
File "c:\portapps\python\lib\site-packages\amaxa\loader\credentials.py", line 146, in _load_v2
capture_output=True,
File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\subprocess.py", line 472, in run
File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\subprocess.py", line 775, in __init__
File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\subprocess.py", line 1178, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified
adding "shell=True" to the subprocess call in _load_v2 in credentials.py seems to resolve:
["sfdx", "force:org:display", "--json", "-u", sfdx_username],
encoding="utf-8",
capture_output=True,
shell=True,
Depends on #7.
This will obviate the need for explicit type checking, and make possible sophisticated, type-aware transforms like maskers and fakers.
Hello @davidmreed , i don't know if it is an issue but how do we LIMIT the number or records?
Example: i would like to have 100 Accounts with associated Asset, Case, Contract, Contact, etc.
Thx.
[Submitted by Brianna Brown on GitLab]
Hi there,
Thank you for your responsiveness as well as for this awesome tool.
I was wondering if you have any advice about how to handle the Product, Pricebook and PricebookEntry trifecta? I haven't determined a way to import this data using Amaxa. Because a pricebookentry needs to be first added to the Default Pricebook - i'm struggling to figure out how to import pricebookentry because of the nature of the standard object.
Without the upsert function, it's not possible to add the pricebookentries to an existing pricebook (ie - the standard pricebook which is a salesforce default record).
I was wondering if you had any tips for dealing with this scenario. I do realize it's a specific case, and one of the more difficult relationships to navigate.
Thanks for your help!
-Brianna
When extracting in reverse order, the import fails when trying to load a child of a master-parent relationship, where the field was defined reparenting not allowed.
Example: We have an object called Employee, which has child File records, tied in a master detail. The Employee__c field on the File object is defined as reparenting not allowed.
If I try to extract just a specific File using a specific Id, followed by extracting the parent Employee record (using extract: descendents: true - reverse import), the operation fails: "Employee__c is a dependent lookup, but not updateable".
I'm assuming this happens because the operation is reverse and that identifies as an update to the Employee__c field, not insert. Which isn't allowed when reparenting is disabled for the relationship. Is there any possible solution or workaround for this?
Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.