samkirkland / ftp-deploy Goto Github PK

View Code? Open in Web Editor NEW

86.0 86.0 40.0 181 KB

Deploy websites over FTP with one command line

License: MIT License

JavaScript 2.58% TypeScript 97.42%

ftp-deploy's People

Contributors

Stargazers

Watchers

Forkers

treetrum johngeoffrey adripo kaerimichi iuwqyir cruonline maciejkorsan penguibird udik gryphondev1 shguevara rufilboss aozora0000 javirpo timthelion chientrm croatialu shunta27 unverbraucht rcosta02 danigta dipak131999 loduis u1e2k olzie-12 sitedata rmarin-sot tommasomeli michel-degroot supbose kanaksarkar91 hugoj-goncalves lei-yt zaghadon kkmurerwa im-machakata evolkmann berdy-vivien mrczarny

ftp-deploy's Issues

Cannot find module multimatch

When I try to run the example I get the following error:

➜  example node deploy.js
internal/modules/cjs/loader.js:613
    throw err;
    ^

Error: Cannot find module 'multiMatch'
Require stack:
- /home/antonio/example/node_modules/@samkirkland/ftp-deploy/dist/module.js
- /home/antonio/example/deploy.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:610:15)
    at Function.Module._load (internal/modules/cjs/loader.js:526:27)
    at Module.require (internal/modules/cjs/loader.js:666:19)
    at require (internal/modules/cjs/helpers.js:16:16)
    at Object.<anonymous> (/home/antonio/example/node_modules/@samkirkland/ftp-deploy/dist/module.js:39:38)
    at Module._compile (internal/modules/cjs/loader.js:759:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:770:10)
    at Module.load (internal/modules/cjs/loader.js:628:32)
    at Function.Module._load (internal/modules/cjs/loader.js:555:12)
    at Module.require (internal/modules/cjs/loader.js:666:19)

`.*` glob matches files in subfolders too

Expected:

.* should only match (dot) files in the root folder (local-dir) (glob tool example)

Actual:

.* matches all dot-files in subfolders as well.
This should happen only for **/.* glob (glob tool example)

I assume this affects all patterns, not tested yet.

Output:

After removing the exclude glob .*:

uploading ".htaccess"
uploading "includes/.htaccess"
uploading "languages/.htaccess"
uploading "maintenance/.htaccess"
uploading "maintenance/archives/.htaccess"

Version:

'@samkirkland/ftp-deploy': 1.1.0

Feature request: Limit of simultaneous connections

When using shared hosts some limit the amount of simultaneous connections to the server. For example, with the German hoster All-Inkl.com you can get an uncaught error:

FTPError: 530 Sorry, the maximum number of connections (10) for your host are already connected.
    at FTPContext._onControlSocketData (/Users/mauricestuffer/WebDev/astro-boilerplate/node_modules/basic-ftp/dist/FtpContext.js:282:39)
    at Socket.<anonymous> (/Users/mauricestuffer/WebDev/astro-boilerplate/node_modules/basic-ftp/dist/FtpContext.js:123:44)
    at Socket.emit (node:events:527:28)
    at addChunk (node:internal/streams/readable:324:12)
    at readableAddChunk (node:internal/streams/readable:293:11)
    at Readable.push (node:internal/streams/readable:234:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23) {
  code: 530
}

A way to limit the connections / queue the uploads would fix this.

FTPError: 451-Error during read from data connection

Hi,

I'm trying to add an action to deploy my ReactJS website to my FTPS online from Github Actions.
All running smoothly except when the upload should start, I've got this error :

My FTP drive space is OK (I've got more than 50GB for few MB to upload). I can read and write in my remote folder.
I wonder if the error come from basic-ftp module or from my side with a CA not correct?
Any idea?
My version of FTP Deploy Action is : 4.3.3

read ECONNRESET on Node 16

---------------------- full error below ----------------------

Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:220:20) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}
node:internal/process/esm_loader:94
    internalBinding('errors').triggerUncaughtException(
                              ^

Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:220:20) {
  errno: -4077,
  code: 'ECONNRESET',
  syscall: 'read'
}

Might be related to https://stackoverflow.com/a/64139528/1870799

patrickjuchli/basic-ftp#205

ECONNRESET: Connection is closed on empty server-dir

Hi,
I am basically using the github action.

I noticed that, wenn the remote directory is empty, the FTP connection is closed by QUIT.

Reproduction:

Have an empty directory on the FTP server
Sync your files to that server
- it tries to download the file list (which is empty)
- then closed the FTP connection
- then collect local files
- tries to make the first directory on the ftp
- crashes as the connection was closed on the remote side, because of the QUIT.

If you found this project helpful, please support it
by giving it a ⭐ on Github --> https://github.com/SamKirkland/FTP-Deploy-Action
or add a badge 🏷️ to your projects readme --> https://github.com/SamKirkland/FTP-Deploy-Action#badge
Using the following excludes filters: ["**/.git*","**/.git*/**","**/node_modules/**"]
Creating local state at ./htdocs/.ftp-deploy-sync-state.json
Local state created
Connected to 202.x.x.x:21 (No encryption)
< 220 ProFTPD Server (ProFTPD) [202.x.x.x]

> AUTH TLS
< 234 AUTH TLS successful

Control socket is using: TLSv1.3
Login security: TLSv1.3
> USER xxx
< 331 Password required for xxx

> PASS ###
< 230 User xxx logged in

> FEAT
< 211-Features:

...

> TYPE I
< 200 Type set to I

> STRU F
< 200 Structure set to F

> OPTS UTF8 ON
< 200 UTF8 set to on

> OPTS MLST type;size;modify;unique;unix.mode;unix.owner;unix.group;unix.ownername;unix.groupname;
< 200 MLST OPTS type;size;modify;unique;UNIX.mode;UNIX.owner;UNIX.group;UNIX.ownername;UNIX.groupname;

> PBSZ 0
< 200 PBSZ 0 successful

> PROT P
< 200 Protection set to Private

  changing dir to ./httpdocs/
> MKD .
< 550 .: File exists

> CWD .
< 250 CWD command successful

> MKD httpdocs
< 550 httpdocs: File exists

> CWD httpdocs
< 250 CWD command successful

  dir changed
Trying to find optimal transfer strategy...
> EPSV
< 229 Entering Extended Passive Mode (|||64293|)

Optimal transfer strategy found.
> RETR .ftp-deploy-sync-state.json
> QUIT
----------------------------------------------------------------
No file exists on the server "./httpdocs/.ftp-deploy-sync-state.json" - this must be your first publish! 🎉
The first publish will take a while... but once the initial sync is done only differences are published!
If you get this message and its NOT your first publish, something is wrong.
----------------------------------------------------------------
Local Files:	1,175
Server Files:	0
----------------------------------------------------------------
Calculating differences between client & server
----------------------------------------------------------------
📁 Create: admin
....
----------------------------------------------------------------
Making changes to 1175 files/folders to sync server state
Uploading: 20.9 MB -- Deleting: 0 B -- Replacing: 0 B
----------------------------------------------------------------
creating folder "admin/"
  changing dir to admin

----------------------------------------------------------------
--------------  🔥🔥🔥 an error occurred  🔥🔥🔥  --------------
----------------------------------------------------------------

----------------------------------------------------------------
----------------------  full error below  ----------------------
----------------------------------------------------------------

Error: Client is closed because read ECONNRESET (data socket)
    at /home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:5197:29
    at new Promise (<anonymous>)
    at FTPContext.handle (/home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:5179:16)
    at Client.sendIgnoringError (/home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:4226:25)
    at Client._openDir (/home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:[47](https://github.com/xx#step:3:48)44:20)
    at Client.ensureDir (/home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:4735:24)
    at /home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:3555:126
    at Generator.next (<anonymous>)
    at /home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:3[54](https://github.com/xx)0:[71](https://github.com/xxx)
    at new Promise (<anonymous>)
Closing reason: Error: read ECONNRESET
    at TLSWrap.onStreamRead (node:internal/stream_base_commons:[217](https://github.com/xxx):20) {
  code: 'ECONNRESET'
}

The workaround would be to just put any file to the remote dir

Possible memory leak

I get memory leak warning when I have more than 10 usages of ftp-deploy

It looks like the problem comes somewhere from bluebird, that is used by promise-ftp

Any idea how to fix that?

Exclude files via script/deploy only dist directory

Hi,
is it possible to exclude files like package.json from deployment via script + CLI as well? Or is it perhaps possible to only sync a specific directory like "dist"?
Thanks and cheers

Error: FTPError: 534 Policy requires SSL.

i get this error , but i have no error with FileZilla
with this config

Cannot find module 'multiMatch'

STEPS TO REPRODUCE

create new project npm init -y
install library npm i @samkirkland/ftp-deploy
write the following script into the package.json -> "test": "ftp-deploy --server someftps.server.com --username *** --password ***"

Expected Result
Since this deployment would be to an ftps server, I would expect an error on the server type as I am not including --protocol ftps.

Actual Result
The following output is what occurs.

Error: Cannot find module 'multiMatch'
Require stack:
- /home/runner/work/trx-now-webapp/trx-now-webapp/node_modules/@samkirkland/ftp-deploy/dist/module.js
- /home/runner/work/trx-now-webapp/trx-now-webapp/node_modules/@samkirkland/ftp-deploy/dist/cli.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)
    at Function.Module._load (internal/modules/cjs/loader.js:667:27)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (/home/runner/work/trx-now-webapp/trx-now-webapp/node_modules/@samkirkland/ftp-deploy/dist/module.js:39:38)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/home/runner/work/trx-now-webapp/trx-now-webapp/node_modules/@samkirkland/ftp-deploy/dist/module.js',
    '/home/runner/work/trx-now-webapp/trx-now-webapp/node_modules/@samkirkland/ftp-deploy/dist/cli.js'
  ]
}

Possible Solution
I can't think of anything off the top of my head, but what I do know is that somehow to call for multiMatch doesn't occur as it should on my local machine after installing directly from npm.

Please Advise.

node 16 is not support anymore from from github need to update to version 20

Hello Everyone , since two day i am trying to upload my laravel project to live serve using FTP
but I got this error >
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, SamKirkland/[email protected]. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.

my ymal file is:
on:

push:

branches:

- main

pull_request:
branches:
- main
name: 🚀 Deploy website on pull_request
jobs:
web-deploy:
name: 🎉 Deploy
runs-on: ubuntu-latest
steps:
- name: 🚚 Get latest code
uses: actions/checkout@v3

  - name: 📂 Sync files
    uses: SamKirkland/[email protected]
    with:
      server: ftp.domain.com
      username: [email protected]
      password: ${{ secrets.ftp_password }}
      server-dir: /public_html/
      exclude: public_html/.env

Old files are not deleted during deployment if `local-dir` is specified

Problem
If you specify local-dir, old files are not deleted.

Steps to reproduce

Create a dist folder and put some files there (e.g. delete.this.txt)
Set local-dir: dist/
Run deploy. The file is uploaded correctly to the remote server
Change the file's name. (e.g delete.this.txt -> new.txt)
Run deploy again

Expected
delete.this.txt is deleted and new.txt is uploaded

Actual result
new.txt is uploaded, but delete.this.txt is not deleted
Log output:

removing delete.this.txt...
> DELE dist/delete.this.txt
< 550 Could not delete dist/delete.this.txt: No such file or directory

Possible solution
This happens because there is no dist folder on the remote. The problem is that here the basePath is passed to the removeFile function. However, basePath exists on the local server, not the remote server.

basePath value should be replaced with the value from server-dir setting

Better timeout support: `.ftp-deploy-sync-state.json` model / early generation, number of files / folders to sync...

I'm having an issue where after some time creating folders / files, the action stop working (seems to freeze) and finally resolve to this: Error: Timeout (control socket)

My config:

- name: 📁 Sync files
      uses: SamKirkland/[email protected]
      with:
        server: XXX
        username: XXX
        password: ${{ secrets.PREPROD_FTP_PASS }}
        local-dir: ./wp-content/themes/val/
        server-dir: ./autodeploytest/
        timeout: 150000

Log representation:

----------------------------------------------------------------
🚀 Thanks for using ftp-deploy. Let's deploy some stuff!   
----------------------------------------------------------------
If you found this project helpful, please support it
by giving it a ⭐ on Github --> https://github.com/SamKirkland/FTP-Deploy-Action
or add a badge 🏷️ to your projects readme --> https://github.com/SamKirkland/FTP-Deploy-Action#badge
----------------------------------------------------------------
No file exists on the server "./autodeploytest/.ftp-deploy-sync-state.json" - this must be your first publish! 🎉
The first publish will take a while... but once the initial sync is done only differences are published!
If you get this message and its NOT your first publish, something is wrong.
----------------------------------------------------------------
Local Files:	340
Server Files:	0
----------------------------------------------------------------
Calculating differences between client & server
----------------------------------------------------------------

📁 Create: ... (a lot)
📄 Upload: ... (a lot)

----------------------------------------------------------------
Making changes to 340 files/folders to sync server state
Uploading: 4.97 MB -- Deleting: 0 B -- Replacing: 0 B
----------------------------------------------------------------

creating folder "..." (some)
uploading "..." (some)

...then at one point, it freeze here i don't know why...
and after the specific timeout, it finally resolve to this:

----------------------------------------------------------------
--------------  🔥🔥🔥 an error occurred  🔥🔥🔥  --------------
----------------------------------------------------------------

----------------------------------------------------------------
----------------------  full error below  ----------------------
----------------------------------------------------------------

Error: Timeout (control socket)
    at Socket.<anonymous> (/home/runner/work/_actions/SamKirkland/FTP-Deploy-Action/v4.3.4/dist/index.js:5288:33)
    at Object.onceWrapper (node:events:627:28)
    at Socket.emit (node:events:513:28)
    at Socket._onTimeout (node:net:550:8)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
Error: Error: Timeout (control socket)

The problem is that it starts over from the beginning if i trigger it again, so zero progress.
I tried changing default timeout value from 120000 to 15000, but it always result in timeout error.

It would be nice to have an option to create .ftp-deploy-sync-state.json before a timeout occurs so it can actually make progress (like one second before set timeout)
Or to be able to set a number of files / folders to sync so it stops after this number and generate .ftp-deploy-sync-state.json to store progress.
Or even have a .ftp-deploy-sync-state.json model / template, so we can see how it is structured and create it ourselves on the server so it can resume where it stopped previously

After 10 minutes Server sent FIN packet unexpectedly, closing connection

Dear Developer Team!

After 10 minutes I always got this error, if I run with --dangerous-clean-slate
Error: Server sent FIN packet unexpectedly, closing connection. at Socket.<anonymous> (/var/www/myproject/node_modules/basic-ftp/dist/FtpContext.js:125:56) at Socket.emit (events.js:326:22) at endReadableNT (_stream_readable.js:1241:12) at processTicksAndRejections (internal/process/task_queues.js:84:21) (node:5131) UnhandledPromiseRejectionWarning: Error: Server sent FIN packet unexpectedly, closing connection. at Socket.<anonymous> (/var/www/myproject/node_modules/basic-ftp/dist/FtpContext.js:125:56) at Socket.emit (events.js:326:22) at endReadableNT (_stream_readable.js:1241:12) at processTicksAndRejections (internal/process/task_queues.js:84:21)
Many directory creation.
It would be great if there will be an option for generate the ftp-deploy-sync-state.json somehow from the actual state. I think this option better then delete everything and copy everything again. Maybe the dry-run could do that.

So what is my thought for users who has already a production code on ftp server:
Given, the repository master code equal the production ftp code
When, run firstly the deploy with some kind of generete option just like dry-run
Then, ftp-deploy-sync-state.json generated without any file deployed

I think this would very helpful for every symfony project becouse in case of a symfony project we have two thousand folders and ten thousand files.

Thank you very much!

How to include only some subdirectories in local dir?

I need to include only some paths from my local dir and exclude everything else.
Like only the custom theme subfolder.

themes/custom_theme
themes/other_theme

The idea would be like this or similar:
--local-dir ./public/wp-content/themes/ --include custom_theme/**/*

However, it looks like there's no --include flag. I was trying to use --exclude with negative globs but no luck.
--local-dir ./public/wp-content/themes/ --exclude !custom_theme/**/*

Is this even possible with this tool?

Thanks.

Deploy only Modfied files (part of the PR)

Is there a way to deploy only modified file that are part of the PR (of my PR contains 5 files, I want only these 4 files to be uploaded to the server).

Thanks for your help!

This yaml download everything, even if the pr contains only 4 files.

-->

name: 🚀 Deployer sur site FTP 'francois'

on:
pull_request:
branches:
- FTP_dev

jobs:
web-deploy:
name: 🎉 Deploy
runs-on: ubuntu-latest
steps:
- name: 🚚 Récupère le code
uses: actions/checkout@v4

- name: 📂 Synchronise les fichiers(check deploy4)
  uses: SamKirkland/[email protected]
  with:
    server: ${{ secrets.FTP }}
    username: ${{ secrets.FTP_USERNAME }}
    password: ${{ secrets.FTP_PASSWORD }}

Update `.ftp-deploy-sync-state.json` if settings are changed

Hey,

I was using ftp-deploy through SamKirkland/FTP-Deploy-Action and had the situation that ftp-deploy-sync-state.json was created before I got my settings right. I missed to initiate a submodule directory so the action was thinking that the folder doesn't exist anymore and tried to remove it. I canceled the action but it appears that the state was captured in the sync state. After I found out about it and I removed it from the remote end my exclude property was acknowledged and everything was working fine. Due to SamKirkland/FTP-Deploy-Action#205 I was thinking the whole time it was something wrong on my end.

Not sure what would be the best solution here. Maybe the exclude check needs to be implemented at the FTP operation level and not sometime before. Or changes to the state file should be applied after all operations successful ran.

What do you think?

Implementing selective FTP deploy to deploy only changed files

Currently, our FTP deploy process uploads all files to the server every time we deploy, even if only a few files have been modified. This can be time-consuming and inefficient, especially when dealing with large projects with many files.

To improve the deploy process, we would like to request a feature that allows us to selectively upload only the files that have been modified since the last deployment. This feature would greatly reduce the deployment time and save bandwidth usage on both our end and the server's end.

We understand that there may be some challenges in implementing this feature, such as tracking the changes made to files since the last deployment and identifying which files need to be uploaded or by simply comparing the last write time of the two files. However, we believe that this feature would greatly improve the overall efficiency of our deploy process and make it easier for our team to manage deployments.

We welcome any feedback or suggestions on how this feature can be implemented, and we are happy to collaborate with the community to make this feature a reality. Thank you for considering our feature request.

import typo

looks like case-sensitive typo in some of the imports, i get this error.

internal/modules/cjs/loader.js:985
  throw err;
  ^
Error: Cannot find module 'multiMatch'
Require stack

ftp-deploy/src/module.ts

Line 5 in d5d6af7

import multiMatch from "multiMatch";

FTPError: 421 No transfer timeout

It occurs with "Server sent FIN packet error." (of basic-ftp)

I found it occurs when this package creates folders for a long time.

Usually, when combined with GitHub actions, it takes over 10 minutes to create 200 ~ 300+ folders.