pipedreamhq / pipedream Goto Github PK

I'd like to trigger a workflow from another workflow via a built-in action.

Workaround: I can create an HTTP trigger for the second workflow and send an HTTP request from the first, but this isn't as clean.

When a workflow needs a long debugging stage, it is highly probable that it will accumulate a lot of log entries. At the moment such entries can be deleted only one by one.
It would be nice to have the option to delete all entries realtive to a day (an [x] button near the labels days ago)

See attached screenshot

I'd like to author TypeScript code steps on Pipedream, similar to how I can choose to "Run Node.js code" today.

I star repos on Github to 1) give kudos to the author and 2) bookmark the repo so I can find it later. I'd like a similar feature on Pipedream.

Because my workflows are getting bigger and bigger, I ended up with lot of environment variables (API keys, external services end-points urls, and so on).

I would suggest, within "Settings" section, to have such env variables sorted by name (ascending and/or descending). Because it is getting really hard to find the right env variable while I'm writing new code, it would be really nice to have an arrow icon (up/down) that implements the sorting, like in the workflows list.

Many thanks!

For HTTP or Email triggers, anyone who knows the destination URL or address of the workflow can send events to it. I can manually add validation in a code step to ensure workflow events come from some source (e.g. only run the workflow when a header is set to a specific value), but I'd like a built-in way to restrict how a workflow is triggered, via API key or some other mechanism.

I'd like to send AMQP messages to a workflow.

I want to run a workflow every time a new row is added to a Google Spreadsheet

Puppeteer provides a Node API to drive a Chrome headless browser. This allows you to programmatically visit sites, take screenshots, and more.

PIpedream currently doesn't support Puppeteer, but I'd like to run workflows that use it.

I'd like a way to configure automatic retries for a workflow so that, if it throws an error, I can run it again N number of times with a specified delay.

I'd like a way to group related workflows in the Pipedream UI, for example by adding a tag to workflows or putting them into folders.

I'd like to get a regular email with a summary of my workflow executions, similar to Zapier or other services.

Like the Global Error Workflow, it'd be great if this were a workflow I could modify, so that I could send the summary to Slack or some other destination, or modify the data sent in the summary.

It might make sense to separate the streams of output, for example in { stdout, stderr }.

Currenty, I can require() an npm package within a code step, but I cannot specify the version of the package I'd like to use. I'd like support for this.

Some possible game APIs to integrate against:

• Dota2
• Minecraft
• League of Legends
• Pokemon
• runescape

We'd love feedback on others that you'd find helpful.

Is your feature request related to a problem? Please describe.
I want to rename a workflow during the forking process.

Describe the solution you'd like
After hitting "fork this workflow" I'd expect to see a modal with a text field that allows me to either enter a new name or modify the existing workflow name (potentially with "copy" appended to the end or something similar.

Do you have a workaround?
Forking then renaming afterward.

Comparable features in other tools?
Yea probably somewhere, but I don't have a reference handy : )

Additional context
Add any other context or screenshots about the feature request here.

I'd like to access a log of all workflow executions (my event history) or errors so I can run analysis on them and observe patterns. The log would need to contain metadata like:

timestamp of execution
duration of workflow execution
success / failure state
workflow id
deployment id
(any other relevant data from steps.trigger.context)
timestamp of error (if error)
error object (if present)

Similar to how I can "Run Node.js code" in a step, I'd like to "Run Golang code" so I can write Go on Pipedream.

I want to connect to the SQL Service and run queries using a JDBC interface.

I'd like a way to search through and discover all public workflows on Pipedream.

I'd like to run Gmail actions as workflow steps, e.g. I'd like to send a new email or attach a label to an email using Pipedream.

Currently, workflows are a linear sequence of steps. I'd like a way to spawn N instances of a step, each of which operates on a subset of the input data.

For example, if I have 100 URLs and want to make HTTP requests to each, I have to write a Node.js code step using something like Promise.all to make those requests in parallel. I'd prefer to use the Send HTTP Request action, feeding in each of the 100 URLs to that action, and run them in parallel.

I also need a way to process the results of these parallel operations once each of the tasks has completed.

Think: map / reduce semantics, or something like JavaScripts Promise.all.

Currently, Pipedream requests broad OAuth scopes for many apps so that, upon connecting your account, you're able to perform any of our listed actions.

But sometimes I just want to perform a very narrow task, and allow Pipedream access to a subset of the listed scopes.

Other times, I might want to authorize one step of a workflow with access to one scope, and another step access to another, so I can tightly control what steps have access to what auth. This is especially important if I'm running a third party npm package that interacts with a service, and I want to carefully control the access that code has to my account.

Overall, some mechanism for selecting my own scopes when I authorize a connected account in Pipedream feels like it would solve these use cases, but I'm open to any other ideas.

I need a way to store data in Pipedream in one workflow and read it in another. Currently, data saved using $checkpoint is scoped to a workflow, so I don't have an easy way to share that data across workflows.

In code steps, I'm presented with an auto-complete menu that gives me the property at the next "level" in a given object, e.g. I type steps. and get the properties of steps that I can select from:

Selecting one of these properties presents yet another "level" of properties, if there are more to show, and I can continue until I select the value I want.

I like this form of auto-complete and would like it in actions forms, as well. I still like the sample values presented next to the values in the "action-style" autocomplete, so ideally I'd like to keep those, too.

I'd like a way to upload a file that I can reference in a workflow, for example a machine learning model I've trained offline and want to load into Pipedream. Or I might have a simple text file I need to reference as metadata in my workflow.

When my workflow yields an error, sometimes it's because my source data was messy. I want to modify the event payload slightly before replaying that event through my workflow.

Other tools — for example integrator.io — have an optimized flow for this use case.

As a workaround, I can edit the test event for my workflow, add the modified payload there, and send the event through the workflow. But this is time consuming when I get a lot of errors in my workflow.

I want to write and test workflows on my local machine, storing the workflow's code in my existing version control system.

I also want to programmatically deploy workflows using a CLI, for example using a command like pipedream deploy. This way, I can deploy workflows locally or as part of my CI/CD pipeline.

Related: #1139

I'd like to run a set of steps or actions within a loop construct. For example, I might receive an array of 100 events in my HTTP payload, and I need to run an action on each. I can do this with code, but I have no way to loop through the elements of this array using Pipedream's actions. I'd like a "loop" action that can handle this use case.

"If I change the name of a step, I need to manually change it if I used it in other steps"

This doc articulates what's public and what's private in a workflow, but as a new user this isn't clear to me when I'm using the product. I'd like more visual clarity on this.

I'd like to see common Slack events as triggers, for example I want to trigger a workflow every time a new message is posted to a channel.

I'd like to programmatically create a workflow via API, passing the relevant data (title, description, trigger, steps, etc.) as parameters.

Right now the list of actions can be quite long for some apps, and I don't have a way to open that list in fullscreen mode like I do for editing code. I'd love to expand the app / action search list so it's easier to see everything in one place.

Currently, workflows are a linear sequence of steps. I'd like support to branch a workflow in a conditional manner (if condition is true, run this step, else run this other step).

I'd like to send data to Amplitude using the API documented here: https://help.amplitude.com/hc/en-us/articles/360032842391-HTTP-API-V2

Some workflows trigger errors with a high frequency, since they hit unreliable APIs. Or I might receive messy source data, which triggers errors because it's not of the expected format.

I already get notified of these errors, but when I visit my workflow, I need to quickly filter on errors so I can see them in one place. This lets me easily replay them one by one.

Is your feature request related to a problem? Please describe.

No

Describe the solution you'd like

Just like Pipedream exposes a way to "Run Node.js code", I'd like a way to "Run Python code". I'd expect support for Python 3 only since Python 2 will be EOL in 2020.

Do you have a workaround?

I need to use specific packages like pandas for which there is no npm equivalent, so even though I can write Node.js for certain things, I need Python.

That is: if I'm pre-processing data in one workflow, then sending the data to another workflow, I'd like to see how they're logically connected (data flows from A -> B).

I'd like to run workflows on Gmail events, e.g. when a new email arrives in my Inbox.

Instead of making HTTP requests to https://myendpoint.m.pipedream.net, I'd like to couple a custom domain to a workflow and make requests to https://my.domain.com, instead.

I'd like a status page for pipedream.com services so I can monitor the health of the service and understand if there's a system issue affecting my workflow.

To make this useful, the delay couldn't count against the execution time limit. The workflow would have to suspend during the delay, and resume when the delay is done.

Describe the bug

When executing :w to save in Vim mode, the app loses focus on the step I'm currently editing. I have to click into the step again to start editing.

To Reproduce
Steps to reproduce the behavior:

1. Enable Vim mode in Settings
2. Add a new Node.js code step
3. Write some code, press :w
4. Editor loses focus

Expected behavior

Ideally, I'd like to keep the browser's focus in the editor for the current step.

Desktop (please complete the following information):

• OS: macOS
• Browser: Chrome
• Version 78.0.3904.70

I might enable a large stream of events via webhook, for example, that hit my workflow. But I might only need to process a subset of the event types. I'd like a way to filter those events before my workflow runs via some set of rules.

I can add a code step to perform this filtering today, but my workflow still executes for every event. I'd like my workflow to execute only on the events I care about.

For example, I'd like to query a Postgres DB from a workflow, and I need to add a network ACL to expose port 5432 to a limited set of IPs. So I'd like to know the IP address(es) from which the request will come.

Ideally, this would be provided in a publicly-accessible data exchange format (for example, AWS provides theirs as a JSON doc), so I can update firewall or network rules programmatically.

e.g. via the CLI, if I have a .env file in a local project, I'd like Pipedream to read and apply the values of those environment variables, using them for the relevant workflow.