Code Monkey home page Code Monkey logo

dospaces's Introduction

DigitalOcean Spaces Filesystem

This plugin provides an DigitalOcean Spaces integration for Craft CMS.

Requirements

This plugin requires Craft CMS 4.0.0 or later.

Installation

To install the plugin, follow these instructions.

  1. Open your terminal and go to your Craft project:

     cd /path/to/project
    
  2. Then tell Composer to load the plugin:

     composer require vaersaagod/dospaces
    
  3. In the Control Panel, go to Settings → Plugins and click the “Install” button for "DigitalOcean Filesystem".

Upgrading from Craft 3.x

A migration will automatically be run on installation to upgrade the old Volume type to the new Filesystem type.

Usage

To create a new asset filesystem for your Spaces bucket, go to Settings → Filesystems, create a new filesystem, and set the Filesystem Type setting to “DigitalOcean Spaces”. When configuring your filesystem, make sure you use env-variables, since some of the settings contain secrets that should not be exposed through your project config. Read the following docs, and create variables as needed.

Please note: If you want to use the Spaces CDN functionality, you only need to change the Base URL setting to your .cdn. URL. The endpoint URL should still be the one without .cdn..

Price, license and support

The plugin is released under the MIT license, meaning you can do whatever you want with it as long as you don't blame us. It's free, which means there is absolutely no support included, but you might get it anyway. Just post an issue here on github if you have one, and we'll see what we can do.

dospaces's People

Contributors

aelvan avatar j-greig avatar saboteur777 avatar tam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

dospaces's Issues

Error message upload failed

Hi, we encounter this error message when we try to upload a new file.
Hope you can help.

Upload failed. The error message was: “Call to undefined method GuzzleHttp\Psr7\Uri::composeComponents()”

settings-drop-spaces

Reconnecting to a DO Space

I had to create a fresh install but want to pull the assets I already uploaded to the CDN via this Spaces plugin. Is there any way to restore them in the CMS?

Volume exception when changing asset slug

Getting an error when changing the slug of an entry whose images are stored in a subfolder named after that slug.

Eg Couldn’t rename folder-1/image.jpg to folder/image.jpg

Is this possible? Or should I just remove {slug} from the upload location on the asset field settings?

Region param is unused in generated endpoint requests?

It appears to me that the region param required in defining a volume is never actually used anywhere in generating the endpoint URL for interacting with my space. On the other hand, if I don't include the region domain segment directly in my endpoint URL, then it won't work.

For better illustration, this is what I'm using in my .env file which the volume definition then reads from...

Works:

ASSETS_ENDPOINT="https://fra1.digitaloceanspaces.com"
ASSETS_REGION="random_string"

Doesn't work:

ASSETS_ENDPOINT="https://digitaloceanspaces.com"
ASSETS_REGION="fra1"

Cache Control not saving

Hey,

The Cache Control values don't seem to be saving to the database or project config.

Thanks!

Issue with Craft 3.6.0 > 3.6.10?

I think this was an issue with Craft rather than the dospaces plugin, but this may help someone.

Discovered a strange bug when running this plugin on a site using Craft version 3.6.10. All of our volumes were broken, so we couldn't access or upload assets. Trying to upload a file gave us this error -

Upload failed. The error message was: “Error executing "ListObjects" on "mutualcdn.https://ams3.digitaloceanspaces.com"; AWS HTTP error: cURL error 1: Protocol "mutualcdn.https" not supported or disabled in libcurl (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for mutualcdn.https://ams3.digitaloceanspaces.com”

The plugin settings were still the same. It was somehow getting confused about the Endpoint to use, and prefixing it with the Bucket value.

After updating Craft to 3.6.11.1 (latest version) it fixed this issue.

Upload failed. The error message was: “Endpoints must be full URIs and include a scheme and host”

Trying to set up spaces but I can't seem to get it to work. I have the following .env variables set up:

SPACES_API_KEY="XXXXXXXXXXXXXXX"

SPACES_SECRET="XXXXXXXXXXXXXXX"

SPACES_ENDPOINT="https://sfo2.digitaloceanspaces.com"

SPACES_REGION="sfo2"

SPACES_BUCKET="test"

SPACES_URL="https://test.sfo2.digitaloceanspaces.com"

I then have the following alias' setup:

'aliases' => [

            '@spacesUrl' => getenv('SPACES_URL'),

            '@spacesKey' => getenv('SPACES_API_KEY'),

            '@spacesSecret' => getenv('SPACES_SECRET'),

            '@spacesEndpoint' => getenv('SPACES_ENDPOINT'),

            '@spacesRegion' => getenv('SPACES_REGION'),

            '@spacesBucket' => getenv('SPACES_BUCKET'),

        ],

and finally here is a screenshot of my volume set up:

image

Im new to using a CDN for assets so I've probably made an obvious mistake but any help would be really appreciated :)

Plugin/Craft Update: Error when generating bucket Url

After updating craft & plugin version, the bucket url is being wrongly generated. For some reason the BUCKET_NAME is being outputted before the HTTP protocol, which is generating an invalid url when uploading assets. I have tested the environment multiple times before and after upgrading the craft/plugin and the error happens as result of the update. All the configuration on the admin asset to be ok.

I've done a quick investigation/debug on the flysystem-aws-s3-v3 - as the plugin uses it under the hood but could not find the root cause it.

I will do further investigation/debug as we need to reach the function that generates the bucket url (should be located in flysystem library/aws s3 client) and check why the bucket name is being added to the HTTP protocol url.

Please check the screenshot below:

image

  • Plugin Version: "1.1.2",
  • Craft Version:: "3.6.10" - The error is also happening on 3.5x version.

Thanks.

Installation fails on Craft 3.7.6

The following error shows, and installation fails Installation request for guzzlehttp/psr7 (locked at 2.0.0) -> satisfiable by guzzlehttp/psr7[2.0.0].

Slow / Never Ending Spin on Uploading Images > 1MB

Craft Version 3.7.5
Latest DO SPaces Plugin

My max upload is set to 100MB on the server (tested + confirmed). We are using CDN on Digital Ocean Spaces and are using the CDN link. Images are set to public.

Uploading images over 1MB spins endlessly in the CMS panel.

Uploading portrait-oriented images loses their orientation

  • Uploading a portrait-oriented image to a DO Spaces volume results in a landcape-oriented image
  • The image is taken with a DSLR, so it may be EXIF related
  • Uploading the same image to a local asset volume respects the portrait orientation

Craft version 3.1.9

Can bucket_endpoint be set in the a config?

Not a digital ocean problem but we were trying to test with MINIO S3 (https://min.io/) on our setup it seems to not like adding the bucket name as a subdomain, from looking at other issues (#16) I saw that you can add the below to the plugin and this fixes the problem for us but can this be set via a config?

$config['bucket_endpoint'] = true;

In Volume.php:

protected function createAdapter()
{
    $config = $this->_getConfigArray();
    $config['bucket_endpoint'] = true;
    $client = static::client($config);
    return new AwsS3Adapter($client, $this->getBucket(), $this->getSubfolder());
}

Setting Up with CDN Support

Can I clarify how we set this up with the CDN feature? After I noticed images were loading in slowly I began using the following - which loads in images faster. BUT then I noticed that I can no longer upload files through front end forms into volumes set up with Digital Ocean, also some of my templates are not using the full path referencing the cdn...

'SPACES_API_KEY' => 'XXXXXXX',
'SPACES_SECRET' => 'XXXXXXX',
'SPACES_ENDPOINT' => 'https://ams3.cdn.digitaloceanspaces.com',
'SPACES_REGION' => 'ams3',
'SPACES_BUCKET' => 'test',
'SPACES_URL' => 'https://test.ams3.cdn.digitaloceanspaces.com',

This is the error I get when uploading files through front end forms (bucket name is changed to test).

Error executing "ListObjects" on "https://test.ams3.cdn.digitaloceanspaces.com/?prefix=content%2Fdocs%2F&delimiter=%2F&encoding-type=url"; AWS HTTP error: Client error: GET https://test.ams3.cdn.digitaloceanspaces.com/?prefix=content%2Fdocs%2F&delimiter=%2F&encoding-type=url` resulted in a 403 Forbidden response: AccessDeniedtesttx0000 (truncated...) AccessDenied (client): - AccessDeniedtesttx0000000000000199322ec-005cdc1c7b-10e1ef4-ams3a10e1ef4-ams3a-ams3`

yii\base\ErrorException: fseek(): stream does not support seeking

Trying to download an asset through the Control Panel with the download button on Asset edit page, getting an internal server error with the following stack trace:

2020-08-24 12:16:57 [-][1][sculjoq6uhh6cl0sdpk1bjhk55][error][yii\base\ErrorException:2] yii\base\ErrorException: fseek(): stream does not support seeking in /var/www/vendor/yiisoft/yii2/web/Response.php:586
Stack trace:
#0 /var/www/vendor/craftcms/cms/src/web/ErrorHandler.php(74): yii\base\ErrorHandler->handleError()
#1 [internal function]: craft\web\ErrorHandler->handleError()
#2 /var/www/vendor/yiisoft/yii2/web/Response.php(586): fseek()
#3 /var/www/vendor/craftcms/cms/src/controllers/AssetsController.php(1077): yii\web\Response->sendStreamAsFile()
#4 [internal function]: craft\controllers\AssetsController->actionDownloadAsset()
#5 /var/www/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array()
#6 /var/www/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams()
#7 /var/www/vendor/craftcms/cms/src/web/Controller.php(178): yii\base\Controller->runAction()
#8 /var/www/vendor/yiisoft/yii2/base/Module.php(528): craft\web\Controller->runAction()
#9 /var/www/vendor/craftcms/cms/src/web/Application.php(291): yii\base\Module->runAction()
#10 /var/www/vendor/craftcms/cms/src/web/Application.php(559): craft\web\Application->runAction()
#11 /var/www/vendor/craftcms/cms/src/web/Application.php(270): craft\web\Application->_processActionRequest()
#12 /var/www/vendor/yiisoft/yii2/base/Application.php(386): craft\web\Application->handleRequest()
#13 /var/www/web/index.php(21): yii\base\Application->run()
#14 {main}

Issue with Indexing Assets

I get the following dialog when attempting to index assets on my Digital Ocean volume, and I'm not sure what to make of it.

Screen.Recording.2021-01-09.at.23.40.01.mov

I am able to successfully upload/edit images in these locations on this volume, and I can see them successfully uploaded when browsing via the DO web UI. Any ideas? Presumably based on a problem with @ in folder names, but not sure that Digital Ocean sees it as a problem.

Perhaps an issue more broadly with Craft?

Automatic image CDN cache clearing feature

Description:
We need a way to automatically clear the cache for images in DigitalOcean Spaces whenever they're updated or replaced in our CMS. This will help ensure clients see changes immediately.

Suggestion:
Add an auto-cache-clear feature for images in the DO CDN, post-update/replacement in the CMS.

Modifying Image Volume

Running into issues modifying Image volumes, which results in an Internal Server Error. I've tested making new volumes and modifying a volume with just PDFs without issue. Please let me know if you need logs to help further diagnose the issue. I've tested this with two different server setups.

Update: I found the issue after looking at logs. I was unable to update database entries because of another issue related lack of foreign keys (Integrity constraint violation 1062 updating a Craft 2 database to Craft 3). Your plugin works extremely well. Thank you for developing it.

403 Forbidden

Hello guys
I'm using dospace in craft running in a pod in k8s, unfortunately I'm receiving 403 Forbidden, when the plugin try to HEAD the resource. Particularly the Digital Ocean space is returning this response.

This is not the case when is running locally, using the the same credentials everything works perfectly in local env.

The question is, why using the same configuration is running locally but now when is running on a pod in k8s?

The second question, is if is required maybe to do an extra configuration both in Digital Ocean side or in craft side to avoid this 403 response?

{main} {"memory":7258864,"exception":"[object] (craft\\errors\\FsException(code: 0): Unable to check if Screenshot-2023-12-13-at-15.57.30.png exists at /app/vendor/craftcms/flysystem/src/base/FlysystemFs.php:151)
[previous exception] [object] (League\\Flysystem\\UnableToCheckFileExistence(code: 0): Unable to check existence for: Screenshot-2023-12-13-at-15.57.30.png at /app/vendor/league/flysystem/src/UnableToCheckExistence.php:19)
[previous exception] [object] (Aws\\S3\\Exception\\S3Exception(code: 0): Error executing \"HeadObject\" on \"https://bucket-name.reg1.digitaloceanspaces.com/files-test/Screenshot-2023-12-13-at-15.57.30.png\"; AWS HTTP error: Client error: `HEAD https://bucket-name.reg1.digitaloceanspaces.com/files-test/Screenshot-2023-12-13-at-15.57.30.png` resulted in a `403 Forbidden` response  (client): 403 Forbidden (Request-ID: tx0000012356465789-006546cdf6565-rt6u77f00) -  at /app/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php:196)
[previous exception] [object] (GuzzleHttp\\Exception\\ClientException(code: 403): Client error: `HEAD https://bucket-name.reg1.digitaloceanspaces.com/files-test/Screenshot-2023-12-13-at-15.57.30.png` resulted in a `403 Forbidden` response at /app/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113)"}

[Bug] Duplicate Bucket Name in Endpoint

When using the default endpoint value provided by Digital Ocean (i.e. 'https://ecabling.nyc3.digitaloceanspaces.com') the S3 plugin duplicates the bucket name in the URL and operations fail (i.e. 'https://ecabling.ecabling.nyc3.digitaloceanspaces.com'). Using an alternate URL (i.e. https://nyc3.digitaloceanspaces.com/ecabling') resolves the upload issue but causes issues elsewhere (url generation).

As far as I can tell, 1 line1 of code fixes this (i just implemented this so not sure if any other side effects). In Volume.php I added the below lines to the createAdapter method:

$config['bucket_endpoint'] = true;
protected function createAdapter()
{
    $config = $this->_getConfigArray();

    $config['bucket_endpoint'] = true;

    $client = static::client($config);

    return new AwsS3Adapter($client, $this->getBucket(), $this->getSubfolder());
}

Upload Failed Message

PHP version | 7.4.9
Linux 4.15.0-112-generic
MySQL 5.5.5
Imagick 3.4.4 (ImageMagick 6.9.7-4)
Craft Pro 3.5.7

DigitalOcean Spaces Volume | 1.1.3

I have three volumes images, videos, resources. I can upload images with no problem but when I try to upload a pdf to resources I get the following error message. The file is ultimately saved but I have to refresh to see it. I've tried deleting the vendor directory and running composer, that seemed to work for one upload but now the error is back.

Upload failed. The error message was: “Getting unknown property: vaersaagod\dospaces\Volume::path”

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.