ankitpokhrel / tus-php Goto Github PK

Upload key is incorrect if we use multi level api path like /tus/files/<upload-key>

I've been trying to use tus-php to upload files to CloudFlare Stream. Eventually got it to work after much fiddling. Core to the problem seems to be that the tus-php client specifies its own upload filepath using the $key parameter, whereas this does not appear to be according to the protocol spec.

As per the Tus spec, on file creation, CloudFlare returns a Location header, which specifies the upload URL for the file. tus-php ignores this, and uses its own value of $key. This causes CloudFlare to fail the upload and return an internal server error.

When creating a new file, the server response is as follows:

HTTP/1.1 201 Created
Location: https://tus.example.org/files/24e533e02ec3bc40c387f1a0e460e216
Tus-Resumable: 1.0.0

The Location header should be the URL that is used for the PATCH request.

Reference: https://tus.io/protocols/resumable-upload.html

With Laravel 5.6.29 , I got following error info when I installed this package

./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for ankitpokhrel/tus-php dev-master -> satisfiable by ankitpokhrel/tus-php[dev-master].
    - Conclusion: remove nesbot/carbon 1.25.0
    - Conclusion: don't install nesbot/carbon 1.25.0
    - ankitpokhrel/tus-php dev-master requires nesbot/carbon ^1.26 -> satisfiable by nesbot/carbon[1.26.0, 1.26.1, 1.26.2, 1.26.3, 1.26.4, 1.27.0, 1.28.0, 1.29.0, 1.29.1, 1.29.2, 1.30.0, 1.31.0, 1.31.1, 1.32.0].
    - Can only install one of: nesbot/carbon[1.26.0, 1.25.0].
    - Can only install one of: nesbot/carbon[1.26.1, 1.25.0].
    - Can only install one of: nesbot/carbon[1.26.2, 1.25.0].
    - Can only install one of: nesbot/carbon[1.26.3, 1.25.0].
    - Can only install one of: nesbot/carbon[1.26.4, 1.25.0].
    - Can only install one of: nesbot/carbon[1.27.0, 1.25.0].
    - Can only install one of: nesbot/carbon[1.28.0, 1.25.0].
    - Can only install one of: nesbot/carbon[1.29.0, 1.25.0].
    - Can only install one of: nesbot/carbon[1.29.1, 1.25.0].
    - Can only install one of: nesbot/carbon[1.29.2, 1.25.0].
    - Can only install one of: nesbot/carbon[1.30.0, 1.25.0].
    - Can only install one of: nesbot/carbon[1.31.0, 1.25.0].
    - Can only install one of: nesbot/carbon[1.31.1, 1.25.0].
    - Can only install one of: nesbot/carbon[1.32.0, 1.25.0].
    - Installation request for nesbot/carbon (locked at 1.25.0) -> satisfiable by nesbot/carbon[1.25.0].


Installation failed, reverting ./composer.json to its original content.

It seems that laravel/framework v5.6.29 requires nesbot/carbon 1.25.* while this package requires netbot/carbon ^1.26.

Some checks are missing in patch request

• All PATCH requests MUST use Content-Type: application/offset+octet-stream, otherwise the server SHOULD return a 415 Unsupported Media Type status.

• Invalid now If a PATCH request does not include a Content-Length header containing an integer value larger than 0, the server SHOULD return a 400 Bad Request status.

Link to Tus Protocol Documentation on PATCH

Checked the code and found that Client can set the $apiPath and which is in the abstract scope of two classes. i.e Server and Client both.

From Abstract Class :

/** @var string */
    protected $apiPath = '/files';

/**
     * Set API path.
     *
     * @param string $path
     *
     * @return self
     */
    public function setApiPath(string $path) : self
    {
        $this->apiPath = $path;

        return $this;
    }

So, this means that php Client can set the server API path . Is this the right way? Isn't it vulnerable?

Hello,

I was trying to run a simple test with this implementation however I must be doing something wrong because I can't get it to work. Here is my local web server root:

-rw-r--r--@  1 TCB13  staff    10244 21 Nov 20:04 .DS_Store
-rw-r--r--   1 TCB13  staff      104 21 Nov 20:06 .htaccess
drwxr-xr-x   8 TCB13  staff      256 21 Nov 20:44 .idea
-rw-------   1 TCB13  staff  2097152 21 Nov 19:49 2mb.file
-rw-r--r--   1 TCB13  staff      293 21 Nov 20:08 client.php
-rw-r--r--   1 TCB13  staff       68 21 Nov 19:43 composer.json
-rw-r--r--   1 TCB13  staff    29900 21 Nov 19:43 composer.lock
drwxr-xr-x   2 TCB13  staff       64 21 Nov 20:02 files
-rw-r--r--   1 TCB13  staff      195 21 Nov 20:03 server.php
drwxr-xr-x   2 TCB13  staff       64 21 Nov 19:51 uploads
drwxr-xr-x  14 TCB13  staff      448 21 Nov 19:53 vendor

As you can see I've a server.php file:

<?php
include "vendor/autoload.php";
$server = new \TusPhp\Tus\Server();
$server->setUploadDir(__DIR__ . DIRECTORY_SEPARATOR . "uploads");
$response = $server->serve();
$response->send();

And a client.php:

<?php
include "vendor/autoload.php";
$client = new \TusPhp\Tus\Client("http://tus.test/");
$key = "upload-key";
$client->setKey($key)->file(__DIR__ . DIRECTORY_SEPARATOR . "2mb.file", "2mb.file")->upload();

Also . htaccess :

RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^files/?(.*)?$ /server.php/$1 [QSA,L]

Now, when I run the client, I get:

Fatal error: Uncaught GuzzleHttp\Exception\ClientException: Client error: `POST http://tus.test/files` resulted in a `404 Not Found` response in /Users/TCB13/MAMP/tus/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113

I'm just using the code on the example, shouldn't it work?

Thank you.

PHP Version: 7.2.1
tus-php: 0.1.0

Just a small feature request to add the ability to specify a location for the file based cache.

Currently it is at vendor/ankitpokhrel/tus-php/.cache/tus_php.cache however when using code deploy or deploying into a load balancer setup that location will be replaced or not shared between instances.

Putting the tus_php.cache file in the directory specified by setUploadDir() would be a good alternative.

Using tus client for uploading a file and getting following error in http error log

PHP Fatal error: Uncaught TypeError: Argument 2 passed to TusPhp\File::seek() must be of the type integer, null given, called in /var/www/html-tus/vendor/ankitpokhrel/tus-php/src/Tus/Client.php on line 527 and defined in /var/www/html-tus/vendor/ankitpokhrel/tus-php/src/File.php:387

While debugging, found that issue can be in the following lines of code of 'getData' function

$file   = new File;
$handle = $file->open($this->getFilePath(), $file::READ_BINARY);
$offset = $this->partialOffset;

if ($offset < 0) {
    $fileMeta = $this->getCache()->get($key);
    $offset   = $fileMeta['offset'];
}

$file->seek($handle, $offset);

As $offset value is -1 and when it tries to get information from cache by key, it returns null.

Using file as cache and below are my configurations for cache from /vendor/ankitphokhrel/tus-php/src/Config/default.php

/**
 * File cache configs.
*/
'file' => [
     'dir' => '/var/www/html-tus/cache/',
     'name' => 'tus_php.cache',
],

Please also note that cache directory have write privileges and I have created 'tus_php.cache' by myself. However, no content get written in this during file upload

Progress bar instantly reaches 100% usually in local setup when using uppy with tus-php-server. There is a long pause afterwards while the file is being uploaded.

Hi

Any plans to support this under slim3 framework

When using tus-php in client mode, there is no way to set headers using middleware. It appears that it's only possible to set headers in server mode.

I meet permission problem so I have to define the cache dir for my app, however I found setCacheDir is defined in FileStore.php, is it possible to add setCacheDir method in AbstractTus.php?

Thanks.

I'm currently trying to use tus-php to upload to CloudFlare Stream. It looks like tus-php doesn't include all the standard required headers that the Tus protocol requires, and CloudFlare ends up giving an obscure error, and doesn't upload the file successfully.

Is there any way to output debug information so that I can see the contents of the HTTP request (including all headers) so that I can debug further?

I've considered intercepting the traffic using Wireshark, but CloudFlare uses HTTPS, which makes this rather difficult.

In my laravel case, REDIS_HOST is not 127.0.0.1, it config in laravel .env file, if i use the clear command bin like reademe : ./vendor/bin/tus tus:expired redis, it will get an error

In AbstractConnection.php line 155:

  Connection refused [tcp://127.0.0.1:6379]

https://github.com/ankitpokhrel/tus-php/blob/master/src/Cache/CacheFactory.php#L20
because there can't get the REDIS_HOST env value, ./vendor/bin/tus tus:expired redis don't run via laravel framework, so laravel config environment does't exist, my solution is make it run in laravel framework， add to laravel schedule job, like this：

app/Console/Kernel.php

protected function schedule(Schedule $schedule)
{
    $schedule->exec('./vendor/bin/tus tus:expired redis')->dailyAt('01:00');
}

I think you can update readme to help other people or do a better solution. 😎

What about adding a middleware for filtering HTTP requests. Application like authenticated, CORS, CSFR, IP Restriction Can be done.

ankitpokhrel/tus-php dev-master requires symfony/console ^4.1@dev -> satisfiable by symfony/console[4.1.x-dev] but these conflict with your requirements or minimum-stability

Hi Ankit,

When I want to upload a file twice, the second time the upload is not sent since the HEAD request (200 OK) consider the file is fully uploaded (Eg : Upload-Length: 156354 && Upload-Offset: 156354) since the Server find in cache an 'offset' key ...

So, would not it be better to remove the cache for files which has been successfully uploaded ?

Thank,
Aurélien

glancing through the code there is nothing that's used from the illuminate/response that the symfony/foundation that the illuminate/response wraps doesn't support out of the box.
This makes the whole package code size and vendor code usage to minimum.

I know the endpoint is set for Tus server.

const uppy = Uppy.Core({debug: true, autoProceed: false})
    .use(Uppy.Dashboard, {target: '#uppyUploader', inline: true})
    .use(Uppy.Tus, {endpoint: 'http://0.0.0.0:8081/files/'})
    .run()
  uppy.on('success', (fileCount) => {
    console.log(`${fileCount} files uploaded`)
  })

But I do not know how to call client, any tips will be appreciated.

I would like to do some stuff when the upload is finished. (transcode uploaded file for example and uploaded it to S3).

You mentioned in #65 that it can be done by middleware, but I honestly can't understand how.
Could you please give at least a general idea of how to go about it? I honestly tried, but can't grasp.
Or instead of middleware should I do something like described here #57 ?

I just get an error like show on the picture below:

CLICK HERE TO SHOW SCREEN SHOOT

I use Macbook Pro for development, and using MAMP.

Hi
Want to have an example in php where in want to save the files into the subfolder based on key generated

Hi guys,
I am using Uppy with tus-php on Laravel. My understanding is that the tus protocol besides being more redundant would allow multiple parallel upload streams for a single file, thereby speeding it up.

I cannot for the life of me find any help on this .. I can only get a single path stream going, well that's what it looks like in chrome dev tools networking tab.

Any help is much appreciated.

I need to do resumable file uploading using REST API (Laravel),
API will be consumed by Android APP.
Please Suggest.

Is it possible to upload files directly to an S3 bucket?

Hi there

Thank you for your package.

Can you please update it for using
"symfony/console": "^4.0"
?

Thanks in advance.

Currently using 1.26.3. The latest version is 2.6, maybe an upgrade?

I believe the current version is locked on 1.26 because of Laravel, however there are other frameworks and contexts where running Carbon 2.6 is needed.

The server should check the validity of Upload-Offset header during PATCH request.

The docker solution of the project not working seamlessly.
I execute the command: docker-compose up -d

And the terminal say me to:

Pulling tus-base (tus-php-base:latest)...
ERROR: pull access denied for tus-php-base, repository does not exist or may require 'docker login'

Can you give access of the docker hub repository?

Posted originally as something else, but I figured it out. I do however have an issue in Symfony. I'm getting the Too Many Redirects issue (max 5) when is ran on verify.php

$offset = $client->setKey($uploadKey)->file($fileMeta['tmp_name'])->getOffset();

Is it possible for this to run on request within a framework, like Laravel?

I want to test this out, however with deployment it would be ideal to have requests handled by our app instead of running a separate server.

Hi,

I was wondering after reading the doc and implementing tus-php with your help, how would it be possible to upload the same file several times in dev environment.

I'm running your example file I adapted under laravel.
Each time I try to upload a file that has already been sent, verify request is the only hit and returns Uploaded: 100.00% in js console and i would like it to be uploaded anyway.

I implemented it using default cache config and not dis.
Browser cache has been emptied, and tus:expired has already been run.

Thanks in advance !

Hi, I'm investigating this library and have been able to successfully set it up.
Thanks for the work.

I have a few questions about how it works though.

Does it emit hooks or events so that I can for instance perform post-upload operations on the server?
I'd also like to be able to handle failed uploads, upload progress etc on the server.
I'm using it with Laravel.

Would it just be a matter of intercepting and inspecting the response object before return $response->send();?

Also, how is the performance of this library? Do you know if there have been any issues related to that?

Thanks,

Implement basic file locking to prevent reading from file when another instance of php script writes to the same file.

Related #97 (comment)

The Client SHOULD use Upload-Expires header to determine if an upload is still valid before attempting to resume the upload.

https://tus.io/protocols/resumable-upload.html#upload-expires

Hi,

I'm thinking of using this package as a TUS client to interact with Cloudflare Stream that uses TUS as well. Cloudflare requires that I send some authentication headers together with my request.

Since Guzzle already supports custom headers, would you consider adding it too?

Thanks.

Your API seems great ! But i've been scratching my head for a few days now to integrate it. I'm still a beginner in Laravel and in php in general but it seems that after following all your integration process (I found it very clear), I guess it throws 405 error at the upload stage.

I already included csrf in the header request.

Error message.txt

Ever encountered this issue before ?
Help would be much appreciated ! :)

Routes

//Tus routes
Route::any('/tus/{any?}', function () {
    $response = app('tus-server')->serve();
    return $response->send();
})->where('any', '.*');


Route::any('/files/{any?}', function () {
    $response = app('tus-server')->serve();

    return $response->send();
})->where('any', '.*');

Route::get('test', function (){
	return view('test');
});
Route::post('verify', 'TusController@TusFile');
Route::post('upload', 'TusController@UploadTus');

Controller

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use TusPhp\Exception\FileException;
use TusPhp\Exception\ConnectionException;
use GuzzleHttp\Exception\ConnectException;
use TusPhp\Exception\Exception as TusException;
use GuzzleHttp\Client;
require '/Users/fabrique/code/dancenote/vendor/autoload.php';




class TusController extends Controller
{

/**
 * Uploads files in 5mb chunk.
 */
public function TusFile () {


$client = new \TusPhp\Tus\Client('http://dancenote.test/tus');

// Alert: Sanitize all inputs properly in production code
if ( ! empty($_FILES)) {
    $status    = 'new';
    $fileMeta  = $_FILES['tus_file'];
    $uploadKey = hash_file('md5', $fileMeta['tmp_name']);

    try {
        $offset = $client->setKey($uploadKey)->file($fileMeta['tmp_name'])->getOffset();

        if (false !== $offset) {
            $status = $offset >= $fileMeta['size'] ? 'uploaded' : 'resume';

        } else {
            $offset = 0;
        }
        echo json_encode([
            'status' => $status,
            'bytes_uploaded' => $offset,
            'upload_key' => $uploadKey,
        ]);
    } catch (ConnectException $e) {
        echo json_encode([
            'status' => 'errorconnect',
            'bytes_uploaded' => -1,
        ]);
    } catch (FileException $e) {
        echo json_encode([
            'status' => 'resume',
            'bytes_uploaded' => 0,
            'upload_key' => '',
        ]);
    }
} else {
    echo json_encode([
        'status' => 'error',
        'bytes_uploaded' => -1,
    ]);
}

}

public function UploadTus() {
	$client = new \GuzzleHttp\Client(['base_uri'=>'http://dancenote.test/tus', 'debug'=> true);
	// Alert: Sanitize all inputs properly in production code
	if ( ! empty($_FILES)) {
	    $fileMeta  = $_FILES['tus_file'];
	    $uploadKey = hash_file('md5', $fileMeta['tmp_name']);
	    try {
	        $client->setKey($uploadKey)->file($fileMeta['tmp_name'], time() . '_' . $fileMeta['name']);
	        $bytesUploaded = $client->upload(5000000); // Chunk of 5 mb
	        echo json_encode([
	            'status' => 'uploading',
	            'bytes_uploaded' => $bytesUploaded,
	            'upload_key' => $uploadKey
	        ]);
	    } catch (ConnectionException | FileException | TusException $e) {
	        echo json_encode([
	            'status' => 'error',
	            'bytes_uploaded' => -1,
	            'upload_key' => '',
	            'error' => $e->getMessage(),
	        ]);
	    }
	} else {
	    echo json_encode([
	        'status' => 'error',
	        'bytes_uploaded' => -1,
	        'error' => 'No input!',
	    ]);

}
}
}

The Symfony2 PHP framework silently transforms all HEAD requests into GET.
symfony/symfony#22028

This affects Drupal 8, as well as Laravel I'm sure:
https://www.drupal.org/project/drupal/issues/2752325

I don't know the stats, but between Drupal, Laravel, and Symfony itself.. this affects a substantial group of PHP projects.

What this causes is if you try to resume an upload, it attempts a few times, and then gives up. This is because the HEAD is getting transformed into a GET request in the background (you can't see this in the request/response traffic, it still says HEAD). Then the response to your TUS client (e.g. Uppy) is tus-php trying to provide a file download, instead of the proper HEAD headers.

If upload two diffrent files with same filename or same file with same filename upload from two browsers files append together

Could you provide a tutorial on how to implement?

It would be very nice to tell the server not send the response and instead return the Response object. Even though the sever has the method getResponse(), is imposible to get it because the script ends due to calling exit().

It would be better to allow serve() to be silent and then grab the response. Here are some ideas:

$response = $server->serve(false); // $send = false

$response = $server->handle(); or $response = $server->process();

$server->serve(false); // $send = false
$response = $server->getResponse();

Thank you for the great work.

The request url becomes http://example.com/tus/tus instead of http://example.com/tus

Great library,i have been able to get it up and running,though i need more functionality on the server ie
1.Returns/extract the size of file uploaded ie for each partial request
2.Returns/Extract the location of the uploaded file and the file name
The above two must be able to be done on the server side.
Thanks in advance.