Code Monkey home page Code Monkey logo

docs's Introduction

Stitch black and white logo

Stitch Documentation

Creative Commons License

Welcome to the Stitch Docs! This is the repository for stitchdata.com/docs, a Jekyll-based documentation site.

The master branch is continuously deployed to production using Netlify.

If you need support for Stitch, contact Stitch Support by emailing [email protected] or logging into your Stitch account and clicking the pink chat bubble in the bottom-right corner.


Contributing

"It takes a village (to have excellent documentation)." We love listening to our customers and learning about how they're using Stitch. Your contributions can help us ensure we're providing you with the documentation you need to get the most out of Stitch.

Found a broken link? Want to share something awesome you learned while using Stitch? Fantastic! Check out the Contribution guide to learn more about our code of conduct and pull request process.

Take a look at the open list of issues to get an idea of what's in the queue. If you find something that interests you, leave a note that you've started working on it to prevent conflicts.

CLA

Before you get started, you'll need to submit a completed Contributor License Agreement (CLA) to Stitch.

Our reason for having a CLA is to define the rights of the contributors — Stitches and non-Stitches alike - and avoid potential confusion and misunderstandings. We believe in DTRT, or Doing The Right Thing. For this project, that means:

  • Ensuring all contributors are equal, and
  • The rights to contributions used by the community cannot be withheld or revoked

Shortly after you open a pull request, you should receive an email from Stitch containing a link to the CLA. Click the link, complete the form, read the CLA, then click I agree. You won't be asked to sign the CLA again unless we make a change.

Stitch will not merge a pull request made against the Stitch Docs project until it is associated with a signed CLA.


Built With


Authors

Anaëlle (@Anaelle1) - Stitch - Technical Documentation Engineer

Check out the other contributors to the Stitch Docs.


Inspiration and Thanks

  • SendGrid for inspiring us to overhaul and then open source our docs
  • Tom Johnson for the theme that got us started

Resources


Licensing & Legal

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License, meaning you are free to use it for any purpose, commericial or non-commerical, as long as you attribute Stitch. If you revise, transform, or build upon this content, you must also distribute your contributions under this license.

Copyright © 2017 Stitch

docs's People

Contributors

ajmers avatar anaelle1 avatar briansloane avatar ca-robles avatar caseyalvarez avatar cosimon avatar dbohan-33 avatar dependabot[bot] avatar dmosorast avatar dsprayberry avatar erinkcochran87 avatar erins-docs avatar fwalsh-talend avatar github-actions[bot] avatar harrisonmarcrose avatar ichensvmk avatar jasondyjohnson avatar jazsmi avatar khalkyer avatar laurenhallden avatar lmilan avatar luandy64 avatar matabouti avatar mwfrantz avatar nick-mccoy avatar pcreux avatar remnantkevin avatar shedd avatar stitchthanadams avatar toblea avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docs's Issues

Destination Google cloud storage

Hi,

Does stitch support Google cloud storage (GCS) as a destination? If not, is it on your roadmap?

While incase of BigQuery destination, I see it uses GCS temporarily to hold the data. Can we configure time/quantity to hold the data before moving to Big query?

Ringcentral API call fail

Tried setting up Ringecntral production credentials via stitch, but I get an error log. The Ringcentral team asked if we could get access to a more comprehensive log from your end. Not sure of the cause of the error.

LOG currently from stitch after trying to configure:

2021-11-12 17:15:53,975Z main - INFO Running tap-ringcentral version 1.0.1 and target-stitch version 3.2.0
2021-11-12 17:15:55,136Z main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2021-11-12 17:15:55,139Z main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2021-11-12 17:15:55,140Z main - INFO [smart-services] building smart service producer: topic( com.stitchdata.streamRecordCount ), use_ssl( true )
2021-11-12 17:15:56,151Z main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [23] at offset None
2021-11-12 17:15:56,153Z main - INFO Starting tap to discover schemas: tap-env/bin/tap-ringcentral --config /tmp/tap_discover_config.json --discover
2021-11-12 17:15:56,521Z tap - CRITICAL 400 Client Error: Bad Request for url: https://platform.ringcentral.com/restapi/oauth/token
2021-11-12 17:15:56,521Z tap - Traceback (most recent call last):
2021-11-12 17:15:56,521Z tap - File "tap-env/bin/tap-ringcentral", line 33, in
2021-11-12 17:15:56,521Z tap - sys.exit(load_entry_point('tap-ringcentral==1.0.1', 'console_scripts', 'tap-ringcentral')())
2021-11-12 17:15:56,522Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 192, in wrapped
2021-11-12 17:15:56,522Z tap - return fnc(*args, **kwargs)
2021-11-12 17:15:56,522Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_ringcentral/init.py", line 58, in main
2021-11-12 17:15:56,522Z tap - client = RingCentralClient(args.config)
2021-11-12 17:15:56,522Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_ringcentral/client.py", line 24, in init
2021-11-12 17:15:56,522Z tap - self.refresh_token, self.access_token = self.get_authorization()
2021-11-12 17:15:56,522Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_ringcentral/client.py", line 48, in get_authorization
2021-11-12 17:15:56,522Z tap - response.raise_for_status()
2021-11-12 17:15:56,522Z tap - File "/code/orchestrator/tap-env/lib/python3.5/site-packages/requests/models.py", line 935, in raise_for_status
2021-11-12 17:15:56,522Z tap - raise HTTPError(http_error_msg, response=self)
2021-11-12 17:15:56,522Z tap - requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://platform.ringcentral.com/restapi/oauth/token
2021-11-12 17:15:56,553Z main - INFO Tap exited abnormally with status 1
2021-11-12 17:15:57,721Z main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [5] at offset None
2021-11-12 17:15:57,728Z main - INFO No tunnel subprocess to tear down
2021-11-12 17:15:57,728Z main - INFO Exit status is: Discovery failed with code 1 and error message: "400 Client Error: Bad Request for url: https://platform.ringcentral.com/restapi/oauth/token".

Support large transactions for Postgres

We are having some problems with our Postgres integrations with a transaction that generates more than 1 GB on our replication_slot. In these kinds of problems, we are needed to stop the integration and restart it again.

As I can understand this is because of the limitations on Postgres implementation, if there is a transaction that generates more than 1 GB of data wal2json plugin couldn't handle this. Because of this issue, there are some configuration parameters on the wal2json side like format-version and write-in-chunks

It seems using format-version=2 and write-in-chunks=true look promising.

Is there a way to use these parameters for wal2json or any other way to handle large transactions for Postgres implementation?

wrong record in _sdc_batched_at

Hello,
I found some weird record in my DWH. Result of my research i found an error in the _sdc_batched_at column. in that my case there isn't problem in _sdc_sequence column but in _sdc_batched_at column i think it might be an issue. I added to below an image about my table result. You can see last sdc_sequence number true but last sdc_batched_at row is wrong!

Could you help me with this issue?

image

Refactor function

Seems like this function should assert that type looks a certain way and that the object contains properties otherwise throw. I'm not sure the exception handling is actually hit, but rather the first error statement gets printed.

(defn schema-file?
[input-json-schema-file]
(try
(let [json-schema (read-schema input-json-schema-file)]
(or (and (or (= "object" (json-schema "type"))
(= ["null" "object"] (json-schema "type")))
(contains? json-schema "properties"))
(println (format "%s is not a valid schema file" input-json-schema-file))))
(catch Exception e
(println (format "%s is not valid schema file" input-json-schema-file)))))

Asana integration has no way to track deleted tasks

Hi!

I'm super new to this world, but one thing I realized when connecting Asana to our warehouse via Stitch was that we have no way to know if a task has been deleted. So when trying to show the "lifecycle" of individual tasks, we see tasks that seemingly stop and get "stuck" in place for a long time. But that is because those tasks were deleted and so there are never any more updates to those tasks.

It'd be awesome if we had a way to see a boolean field for if a task was deleted and if possible a timestamp showing when the deletion happened.

Looks like @erinkcochran87 was the last one to update the Asana integration, so I don't know if you are the right person to ask about this, but figured I'd @ you just in case!

Thanks!

Bad data replicating Recurly

In order to have the correct data sync for line item invoices from recurly, "X-Api-Version: 2.22" must be specified as an HTTP header. Without this, recurly returns the data in a kind of "quirks" mode, where it returns incorrect data.

Could this be corrected in stitch? or specified as an additional parameter in the recurly/stitch integration?

Adwords - Duplicate rows

I am running a Stitch based Adwords extraction to Redshift.
I noticed that on my Call Metrics table one of the source rows is replicated every run, thereby inflating the real value.
The attached file has all instances of this duplicate row. The only differentiating fields for these rows is the default sdc rows that Stitch uses.
duplicate_stitch.xlsx

I haven't made any settings changes to the report extract so there might be something that I am missing. I haven't checked this with other tables yet.
Please help fix this.

Campaign Manager 360 Integration - Connection Test Timeout

Hi,

Trying to set up a new Stitch integration for Campaign Manager 360 I am getting a "Connection test timed out" error message after successful authorization.

From Stitch documentation, it is not clear how to troubleshoot this error. Could you please help with that?

Kind regards,
Dmytro

BigQuery has reported an error, but the docs are not helpful with what to do next

Google Cloud is currently suffering from a number of issues:

https://status.cloud.google.com/

And we're loading some data via Stitch into BigQuery.

We've had the following error:

Stitch_-_Notifications

And the links send us to this page:

https://www.stitchdata.com/docs/troubleshooting/destinations/destination-data-loading-errors

Which doesn't have any context about BigQuery.

My suspicion is that this issue will resolve itself when BigQuery comes back up, but until then, the Stitchdata status dashboard (https://status.stitchdata.com/) says everything is ok, and I'd love some reassurance that either everything is ok, or what to do to verify if it is / isn't ok.

Can the docs be improved to explain what to do in this scenario?

Thanks!

(p.s. your documentation is generally fantastic and an inspiration. Thank you).

Stitch with postgresql 9.6

Hi,

According postgresql manual said you only need

https://www.stitchdata.com/docs/integrations/databases/postgresql/v1

host replication [account] [IP]   md5

to add into stitch but actually you do need

  1. host your_database_name for tapping permission and
  2. host replication for replication

I don't know what going with your support but this small problem used us 2 month to troubleshooting and support only answer please check the manual ( but which is total wrong from beginning. )

Please fix manual.

Connection refused [Postgresql]

I'm trying to integrate postgresql with stich data and I'm having issues establishing a connection.

I'm currently encountering this error "2020-08-19 11:39:46,651Z main - INFO Exit status is: Discovery failed with code 1 and error message: "could not connect to server: Connection refused"."

Within my postgresql.conf file the listen_addresses is set to '*' by default. Is this correct or do I need to explicitly allow access to these IP addresses:

  • 52.23.137.21/32
  • 52.204.223.208/32
  • 52.204.228.32/32
  • 52.204.230.227/32

And will I still need the CREATEROLE or SUPERUSER privilege?

Add date as a dimension to Google E-Commerce integration

It's hard to derive useful insights from the Google E-Commerce integration (e.g. from which sources are the majority of our transactions coming?) without a time dimension. Each transaction has a date associated with it, so it would be valuable to add the transaction date to the schema.

Stitchdata not syncing all pipedrive columns to data.world

Hi - I've added new custom fields into pipedrive persons (People); however it doesn't seems that stitchdata is grabbing these fields because I don't see them show up in data.world. for example, the field 7acf707bd7373a4a9badc0b425adc62efd96ff9a is not coming through.

Google Analytics integration data discrepancy

Hi there,

I created an integration from Google analytics to PostgreSQL and noticed a discrepancy in the "sessions" data.

When I use Google's Query Explorer to get the exactly same metrics I'm using in my integrations the number is different from what I have in my database. In my database is always the number is always higher which leads to different numbers viewing from Google's dashboard and from my Grafana's dashboards.

For example on 11/22 Google show me 259 against 275 sessions in the database.

metrics:

dimensions = ga:sessionCount
metrics = ga:sessions

dimensions = ga:sessionDurationBucket
metrics = ga:avgSessionDuration

dimensions = ga:pagePath
metrics = ga:pageviews

Google Metrics.xlsx

Thx for your help.

Question about Postgres UUID datatype and log-based replication

Hi, I created a PostgreSQL integration on a database having many tables with a uuid primary key. All of the tables are set up to replicate via the log-based incremental strategy.

When I run an extraction, I run into the following error. This happens on the same table, every time.

2021-11-02 22:49:18,069Z    tap - INFO  [main] tap-hp-postgres.sync-strategies.full - Starting FULL TABLE sync of stream: companion-public-variant
2021-11-02 22:49:18,099Z    tap - INFO  [main] tap-hp-postgres.sync-strategies.full - Querying: SELECT "product_id", "sku", "option_values", "index", "weight_units", "id", "name", "updated_at", "attributes", "created_at", "shopify_id", "barcode", "shopify_images", "price", "attachments", "weight" FROM "public"."variant" WHERE ("id" > ? OR "id" IS NULL) ORDER BY "id" (ec3e91a2-2362-45fc-92f7-ce24944d3611) with itersize: 20000
2021-11-02 22:49:18,111Z    tap - FATAL [main] tap-hp-postgres.main - Fatal Error Occured - ERROR: operator does not exist: uuid > character varying
2021-11-02 22:49:18,111Z    tap - FATAL [main] tap-hp-postgres.main - Fatal Error Occured -   Hint: No operator matches the given name and argument types. You might need to add explicit type casts.
2021-11-02 22:49:18,111Z    tap - FATAL [main] tap-hp-postgres.main - Fatal Error Occured -   Position: 239
2021-11-02 22:49:18,114Z    tap - ERROR [main] #error {
2021-11-02 22:49:18,114Z    tap -  :cause ERROR: operator does not exist: uuid > character varying
2021-11-02 22:49:18,114Z    tap -   Hint: No operator matches the given name and argument types. You might need to add explicit type casts.
2021-11-02 22:49:18,114Z    tap -   Position: 239
2021-11-02 22:49:18,114Z    tap -  :via
2021-11-02 22:49:18,114Z    tap -  [{:type org.postgresql.util.PSQLException
2021-11-02 22:49:18,115Z    tap -    :message ERROR: operator does not exist: uuid > character varying
2021-11-02 22:49:18,115Z    tap -   Hint: No operator matches the given name and argument types. You might need to add explicit type casts.
2021-11-02 22:49:18,115Z    tap -   Position: 239
2021-11-02 22:49:18,115Z    tap -    :at [org.postgresql.core.v3.QueryExecutorImpl receiveErrorResponse QueryExecutorImpl.java 2553]}]
2021-11-02 22:49:18,115Z    tap -  :trace
2021-11-02 22:49:18,115Z    tap -  [[org.postgresql.core.v3.QueryExecutorImpl receiveErrorResponse QueryExecutorImpl.java 2553]
2021-11-02 22:49:18,115Z    tap -   [org.postgresql.core.v3.QueryExecutorImpl processResults QueryExecutorImpl.java 2285]
2021-11-02 22:49:18,115Z    tap -   [org.postgresql.core.v3.QueryExecutorImpl execute QueryExecutorImpl.java 323]
2021-11-02 22:49:18,115Z    tap -   [org.postgresql.jdbc.PgStatement executeInternal PgStatement.java 473]
2021-11-02 22:49:18,115Z    tap -   [org.postgresql.jdbc.PgStatement execute PgStatement.java 393]
2021-11-02 22:49:18,115Z    tap -   [org.postgresql.jdbc.PgPreparedStatement executeWithFlags PgPreparedStatement.java 164]
2021-11-02 22:49:18,115Z    tap -   [org.postgresql.jdbc.PgPreparedStatement executeQuery PgPreparedStatement.java 114]
2021-11-02 22:49:18,115Z    tap -   [tap_hp_postgres.sync_strategies.full$query_and_write_messages_BANG_ invokeStatic full.clj 161]
2021-11-02 22:49:18,115Z    tap -   [tap_hp_postgres.sync_strategies.full$query_and_write_messages_BANG_ invoke full.clj 153]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.sync_strategies.full$sync_BANG_ invokeStatic full.clj 205]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.sync_strategies.full$sync_BANG_ invoke full.clj 200]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.core$dispatch_traditional_sync_by_strategy invokeStatic core.clj 59]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.core$dispatch_traditional_sync_by_strategy invoke core.clj 36]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.core$sync_traditional_stream_BANG_ invokeStatic core.clj 94]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.core$sync_traditional_stream_BANG_ invoke core.clj 83]
2021-11-02 22:49:18,116Z    tap -   [clojure.core$partial$fn__5841 invoke core.clj 2632]
2021-11-02 22:49:18,116Z    tap -   [clojure.lang.PersistentVector reduce PersistentVector.java 343]
2021-11-02 22:49:18,116Z    tap -   [clojure.core$reduce invokeStatic core.clj 6827]
2021-11-02 22:49:18,116Z    tap -   [clojure.core$reduce invoke core.clj 6810]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.core$do_sync invokeStatic core.clj 196]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.core$do_sync invoke core.clj 188]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.main$main_impl invokeStatic main.clj 114]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.main$main_impl invoke main.clj 101]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.main$_main invokeStatic main.clj 130]
2021-11-02 22:49:18,116Z    tap -   [tap_hp_postgres.main$_main doInvoke main.clj 125]
2021-11-02 22:49:18,116Z    tap -   [clojure.lang.RestFn applyTo RestFn.java 137]
2021-11-02 22:49:18,117Z    tap -   [tap_hp_postgres.main main nil -1]]}
2021-11-02 22:49:18,117Z    tap - INFO  [main] com.stitchdata.target-stitch-avro.core - wait-for-target
2021-11-02 22:49:18,117Z    tap - INFO  [main] com.stitchdata.target-stitch-avro.core - waiting for accumulator thread to complete
2021-11-02 22:49:18,118Z    tap - INFO  [accumulator] com.stitchdata.target-stitch-avro.accumulator-thread - process fin

Is there something I can do to fix or work around this problem? For the time being I have disabled Stitch extraction on the table in question.

Freshdesk API - Contacts Table missing

Hello everyone,

I'm having some issues in Stitch to identify the table 'Contacts' from in Freshdesk integration. All other tables are being recognized normally, except for the 'contacts' one and this seems to be related to the Freshdesk v1 version, because this table only became available in v2. I also noticed that the integration correctly loads data from v2 tables, however I think it is using metadata table names from v1.

Anyone could help me to clarify this point and have thoughts on why this is happening?

Kind regards,
Bernardo

Connection to AWS S3 Failing

When I initially configured S3 has my destination everything was working fine in late August. Now since yesterday I am receiving the following error when trying to connect to my previously configured S3 Bucket.

2018-10-02 20:23:18,949Z main - INFO Attempting to connect to Amazon S3 destination
2018-10-02 20:23:19,036Z main - INFO Verifying access to bucket xxxxxxxxxxxx
2018-10-02 20:23:19,116Z main - INFO Failed to check OCSP response cache file. No worry. It will validate with OCSP server: file: /root/.cache/snowflake/ocsp_response_cache.json, lock file: /root/.cache/snowflake/ocsp_response_cache.json.lck, error: [Errno 2] No such file or directory: '/root/.cache/snowflake/ocsp_response_cache.json'
2018-10-02 20:23:19,118Z main - INFO Failed to locate OCSP response cache file. No worry. It will validate with OCSP server: /root/.cache/snowflake/ocsp_response_cache.json
2018-10-02 20:23:19,474Z main - CRITICAL Error interacting with S3
2018-10-02 20:23:19,474Z main - CRITICAL An error occurred (404) when calling the HeadObject operation: Not Found
2018-10-02 20:23:19,481Z main - INFO Exit status is: Check failed with code -1.

I saved again the correct permissions policy for the S3 Bucket and still getting the same error.

"ERROR: permission denied for database x" for AWS Postgres destination

Hi, I am trying to import Harvest data into a Postgres db destination on AWS and I get permission errors (connection to the db works fine).

When trying to follow the setup permissions I was unable to run this command GRANT SELECT ON ALL TABLES IN SCHEMA pg_catalog TO stitch; because it generated the following error: "permission denied for table pg_statistic". The documentation mentions that Stitch first tries to read the pg_catalog schema before uploading, so I imagine this is the source of the original error. But I'm not sure what else to try get this to work.

TINYINT(4) considered as BIT

MySQL Version: 5.6.33
Redshift Version: PostgreSQL 8.0.2

TINYINT(4) is still converting into Binary Data (True/False)

Include all _sdc_ meta columns on all synced tables

Stitch only adds the _sdc_deleted_at meta column when a deleted record is first discovered.
Is there any reason all meta columns cannot be added at the time of the first sync when the destination tables are created by Stitch, regardless of whether or not hard deletes ever occur in the source db? This seems consistent regardless of which database integration is being used.

Summary: Add _sdc_deleted_at column to all destination tables regardless of the existence of a delete event.

Actions from automations?

Do opens/clicks from automations appear in reports_email_activity? I don't think I see this click data in my table.

Question about Surveymonkey integration

I'm using Stitch to replicate data from surveys that live in Survey Monkey (about 10), the problem is, I don't want to have to add an integration every time a new survey needs to be replicated; so I tried putting all of my surveyIDs in the homonymous field at the time an integration is created like:

surveyID1,surveyID2,surveyID3

but when the historical replication starts it throws an error saying Resource not found which leads me to think that Stitch is looking for that exact string in the surveyIDs

is there a way to replicate more than one survey per integration?

Google Analytics Grain of Data Pull

Hello!

My organization has a custom report in Stitch configured to pull ga:users data by year/month. However, Stitch automatically pulls ga:users data by day. When GA aggregates to a monthly level, it reports different numbers than the sum of the daily totals.

My organization would like to know why we are forced to take the daily grain of data. My org is taking from the same google API as Stitch, but cannot replicate the same numbers.

My Client ID in Stitch is 182174.

Thank you very much!

Jira issue status history

Hello,
I'm doing a POC exploring Jira datasets.

At the moment I'm synchronizing the following tables:
changelogs
issue_comments
issue_transitions
issues
projects
resolutions
versions

I'm looking for all the status changes of issues and when they happened. The changelogs are cleaned up quite frequent so I can't find information for all of them.

Is there something I'm missing? Is there a table I should import that will have the data I'm looking for?

Deletions in Forecast Not updating in the target DB

Hi there, I really like your Forecast integration on Stitch but I have an issue. When I add a task in Forecast its appended to the target DB. When I delete a task it not deleted from the DB when the integration next syncs. I'm using PostgreSQL in AWS.

Stitch JSONB field sync is randomly sending null

StR:

  • Source DB is hosted in Postgresql 11.12, changes detection is WAL based
  • Stitch driver version is v2
  • Source table contains JSONB field entries
  • Destination DB is in MS SQL
  • Destination entries for the same table sometimes contain NULL in string field entries mapped from Postgresql field entries in source tables.

The issue does not seem to be related to data, as it reproduces randomly, and could load (or not, it seems that it's randomized) the same entry correctly if doing full table sync instead of WAL-based CDC.

The issue is critical as it's affecting our production users.
Please assist.

Force append only incremental update on Postgres destination

Hi there,

the docs for a Postgres destination state, when syncing a table with a single Primary Key:

If using Incremental Replication, data will be de-duped based on the Primary Key column and upserted into the table.

The upsert prevents me from reporting incremental changes to a data object/record over time, as a single record just gets updated instead of adding additional entries for any changes occurring between sync intervals. E.g. image I would want to report the changes to a 'project' record in the table 'Projects.

How can I force the append only incremental updates as described here.

Thanks,

Links to Netsuite repo are broken

Hello,

i was trying to find the Netsuite integration code, but have struggled to find it. The following pages that make reference to the Netsuite integration repo don't link to a repo:

The only valid link to a repo took me to this docs repo https://github.com/stitchdata/docs/blob/master/_saas-integrations/netsuite/v1/netsuite-v1.md

Does the netsuite integration exist?

object of type 'NoneType' has no len()

I recently tried to set up my Kustomer Stitch integration to Redshift but I continue to get the following error: "object of type 'NoneType' has no len()". I'm pretty sure I have the right API Keys and starting date -- but what is strange here is that when I kick off a manual backfill -- a bunch of data still populates in Redshift -- whereas when I try to schedule a 30-min job to pull new fields -- I get this error. Help would be much appreciated!

Harvest V1 API is Deprecated

The V1 API used for this integration was deprecated in 2017 and Harvest is now in the process of removing this outdated API. The V2 API is available for continued use. A migration guide has been created to help Developers transition to using the current V2 API: https://help.getharvest.com/api-v1/introduction/overview/migration-guide/

It appears there's a V2 option for this integration already. Could you remove the V1 integration option from https://www.stitchdata.com/docs/integrations/saas/#community-saas-integrations?

Screen Shot 2022-03-14 at 9 46 48 AM

Tables not replicating

Hi - I have a table checked to get tracked (email_events) but I'm not seeing it in my Loading Reports and it's been 2 days. Can you help me?

SFTP Integrations Suggestion Treat empty string as null

I'm looking to connect csv resources using SFTP integrations (or other integrations with csv files) to redshift. I feature that would be helpful here is to have an option in settings to "Treat empty strings as nulls". When extracting a csv which contains a mix of empty strings and double values. Example:

id,Column1, Column2, Column 3,...
1,,1.23,4.56,...
2,1.23,,4.56,...
3,1.23,4.56,,...

it loads duplicate columns for both the double value (actual doubles) and string values (empty strings). It would be very helpful to have a setting or just an assumption to treat these empty values as nulls to prevent a duplication of columns. In my specific case these csv files are used by other programs which expect null values to be empty so the work around in setting these values as null doesn't work for other uses entirely and I'm trying to avoid creating a separate file solely for the use of stitch to extract to redshift.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.