vaersaagod / dospaces Goto Github PK
View Code? Open in Web Editor NEWDigitalOcean Spaces integration for Craft CMS
License: MIT License
DigitalOcean Spaces integration for Craft CMS
License: MIT License
You might want to take a look at the following discussion (prompted by image transform time out issues) where a fix was made to the AWS S3 plugin which the DO Spaces plugin is based on.
Description:
We need a way to automatically clear the cache for images in DigitalOcean Spaces whenever they're updated or replaced in our CMS. This will help ensure clients see changes immediately.
Suggestion:
Add an auto-cache-clear feature for images in the DO CDN, post-update/replacement in the CMS.
Is anyone finding the when they delete an asset in craft, it's removed from the spaces folder but remains in the CDN?
I get the following dialog when attempting to index assets on my Digital Ocean volume, and I'm not sure what to make of it.
I am able to successfully upload/edit images in these locations on this volume, and I can see them successfully uploaded when browsing via the DO web UI. Any ideas? Presumably based on a problem with @ in folder names, but not sure that Digital Ocean sees it as a problem.
Perhaps an issue more broadly with Craft?
Is it possible to set a metadata for files uploaded into a volume automatically? I would like to enable gzip compression for files server from the bucket.
It appears to me that the region
param required in defining a volume is never actually used anywhere in generating the endpoint URL for interacting with my space. On the other hand, if I don't include the region domain segment directly in my endpoint URL, then it won't work.
For better illustration, this is what I'm using in my .env
file which the volume definition then reads from...
Works:
ASSETS_ENDPOINT="https://fra1.digitaloceanspaces.com"
ASSETS_REGION="random_string"
Doesn't work:
ASSETS_ENDPOINT="https://digitaloceanspaces.com"
ASSETS_REGION="fra1"
Craft Version 3.7.5
Latest DO SPaces Plugin
My max upload is set to 100MB on the server (tested + confirmed). We are using CDN on Digital Ocean Spaces and are using the CDN link. Images are set to public.
Uploading images over 1MB spins endlessly in the CMS panel.
Hey,
The Cache Control values don't seem to be saving to the database or project config.
Thanks!
Trying to set up spaces but I can't seem to get it to work. I have the following .env variables set up:
SPACES_API_KEY="XXXXXXXXXXXXXXX"
SPACES_SECRET="XXXXXXXXXXXXXXX"
SPACES_ENDPOINT="https://sfo2.digitaloceanspaces.com"
SPACES_REGION="sfo2"
SPACES_BUCKET="test"
SPACES_URL="https://test.sfo2.digitaloceanspaces.com"
I then have the following alias' setup:
'aliases' => [
'@spacesUrl' => getenv('SPACES_URL'),
'@spacesKey' => getenv('SPACES_API_KEY'),
'@spacesSecret' => getenv('SPACES_SECRET'),
'@spacesEndpoint' => getenv('SPACES_ENDPOINT'),
'@spacesRegion' => getenv('SPACES_REGION'),
'@spacesBucket' => getenv('SPACES_BUCKET'),
],
and finally here is a screenshot of my volume set up:
Im new to using a CDN for assets so I've probably made an obvious mistake but any help would be really appreciated :)
Hello guys
I'm using dospace in craft running in a pod in k8s, unfortunately I'm receiving 403 Forbidden, when the plugin try to HEAD the resource. Particularly the Digital Ocean space is returning this response.
This is not the case when is running locally, using the the same credentials everything works perfectly in local env.
The question is, why using the same configuration is running locally but now when is running on a pod in k8s?
The second question, is if is required maybe to do an extra configuration both in Digital Ocean side or in craft side to avoid this 403 response?
{main} {"memory":7258864,"exception":"[object] (craft\\errors\\FsException(code: 0): Unable to check if Screenshot-2023-12-13-at-15.57.30.png exists at /app/vendor/craftcms/flysystem/src/base/FlysystemFs.php:151)
[previous exception] [object] (League\\Flysystem\\UnableToCheckFileExistence(code: 0): Unable to check existence for: Screenshot-2023-12-13-at-15.57.30.png at /app/vendor/league/flysystem/src/UnableToCheckExistence.php:19)
[previous exception] [object] (Aws\\S3\\Exception\\S3Exception(code: 0): Error executing \"HeadObject\" on \"https://bucket-name.reg1.digitaloceanspaces.com/files-test/Screenshot-2023-12-13-at-15.57.30.png\"; AWS HTTP error: Client error: `HEAD https://bucket-name.reg1.digitaloceanspaces.com/files-test/Screenshot-2023-12-13-at-15.57.30.png` resulted in a `403 Forbidden` response (client): 403 Forbidden (Request-ID: tx0000012356465789-006546cdf6565-rt6u77f00) - at /app/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php:196)
[previous exception] [object] (GuzzleHttp\\Exception\\ClientException(code: 403): Client error: `HEAD https://bucket-name.reg1.digitaloceanspaces.com/files-test/Screenshot-2023-12-13-at-15.57.30.png` resulted in a `403 Forbidden` response at /app/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113)"}
Not a digital ocean problem but we were trying to test with MINIO S3 (https://min.io/) on our setup it seems to not like adding the bucket name as a subdomain, from looking at other issues (#16) I saw that you can add the below to the plugin and this fixes the problem for us but can this be set via a config?
$config['bucket_endpoint'] = true;
In Volume.php:
protected function createAdapter()
{
$config = $this->_getConfigArray();
$config['bucket_endpoint'] = true;
$client = static::client($config);
return new AwsS3Adapter($client, $this->getBucket(), $this->getSubfolder());
}
Can I clarify how we set this up with the CDN feature? After I noticed images were loading in slowly I began using the following - which loads in images faster. BUT then I noticed that I can no longer upload files through front end forms into volumes set up with Digital Ocean, also some of my templates are not using the full path referencing the cdn...
'SPACES_API_KEY' => 'XXXXXXX',
'SPACES_SECRET' => 'XXXXXXX',
'SPACES_ENDPOINT' => 'https://ams3.cdn.digitaloceanspaces.com',
'SPACES_REGION' => 'ams3',
'SPACES_BUCKET' => 'test',
'SPACES_URL' => 'https://test.ams3.cdn.digitaloceanspaces.com',
This is the error I get when uploading files through front end forms (bucket name is changed to test).
Error executing "ListObjects" on "https://test.ams3.cdn.digitaloceanspaces.com/?prefix=content%2Fdocs%2F&delimiter=%2F&encoding-type=url"; AWS HTTP error: Client error:
GET https://test.ams3.cdn.digitaloceanspaces.com/?prefix=content%2Fdocs%2F&delimiter=%2F&encoding-type=url` resulted in a 403 Forbidden
response: AccessDenied
testtx0000 (truncated...) AccessDenied (client): - AccessDenied
testtx0000000000000199322ec-005cdc1c7b-10e1ef4-ams3a10e1ef4-ams3a-ams3`
PHP version | 7.4.9
Linux 4.15.0-112-generic
MySQL 5.5.5
Imagick 3.4.4 (ImageMagick 6.9.7-4)
Craft Pro 3.5.7
DigitalOcean Spaces Volume | 1.1.3
I have three volumes images, videos, resources. I can upload images with no problem but when I try to upload a pdf to resources I get the following error message. The file is ultimately saved but I have to refresh to see it. I've tried deleting the vendor directory and running composer, that seemed to work for one upload but now the error is back.
Upload failed. The error message was: “Getting unknown property: vaersaagod\dospaces\Volume::path”
Trying to download an asset through the Control Panel with the download button on Asset edit page, getting an internal server error with the following stack trace:
2020-08-24 12:16:57 [-][1][sculjoq6uhh6cl0sdpk1bjhk55][error][yii\base\ErrorException:2] yii\base\ErrorException: fseek(): stream does not support seeking in /var/www/vendor/yiisoft/yii2/web/Response.php:586
Stack trace:
#0 /var/www/vendor/craftcms/cms/src/web/ErrorHandler.php(74): yii\base\ErrorHandler->handleError()
#1 [internal function]: craft\web\ErrorHandler->handleError()
#2 /var/www/vendor/yiisoft/yii2/web/Response.php(586): fseek()
#3 /var/www/vendor/craftcms/cms/src/controllers/AssetsController.php(1077): yii\web\Response->sendStreamAsFile()
#4 [internal function]: craft\controllers\AssetsController->actionDownloadAsset()
#5 /var/www/vendor/yiisoft/yii2/base/InlineAction.php(57): call_user_func_array()
#6 /var/www/vendor/yiisoft/yii2/base/Controller.php(157): yii\base\InlineAction->runWithParams()
#7 /var/www/vendor/craftcms/cms/src/web/Controller.php(178): yii\base\Controller->runAction()
#8 /var/www/vendor/yiisoft/yii2/base/Module.php(528): craft\web\Controller->runAction()
#9 /var/www/vendor/craftcms/cms/src/web/Application.php(291): yii\base\Module->runAction()
#10 /var/www/vendor/craftcms/cms/src/web/Application.php(559): craft\web\Application->runAction()
#11 /var/www/vendor/craftcms/cms/src/web/Application.php(270): craft\web\Application->_processActionRequest()
#12 /var/www/vendor/yiisoft/yii2/base/Application.php(386): craft\web\Application->handleRequest()
#13 /var/www/web/index.php(21): yii\base\Application->run()
#14 {main}
Craft version 3.1.9
Getting an error when changing the slug of an entry whose images are stored in a subfolder named after that slug.
Eg Couldn’t rename folder-1/image.jpg to folder/image.jpg
Is this possible? Or should I just remove {slug} from the upload location on the asset field settings?
I had to create a fresh install but want to pull the assets I already uploaded to the CDN via this Spaces plugin. Is there any way to restore them in the CMS?
The following error shows, and installation fails Installation request for guzzlehttp/psr7 (locked at 2.0.0) -> satisfiable by guzzlehttp/psr7[2.0.0].
Running into issues modifying Image volumes, which results in an Internal Server Error. I've tested making new volumes and modifying a volume with just PDFs without issue. Please let me know if you need logs to help further diagnose the issue. I've tested this with two different server setups.
Update: I found the issue after looking at logs. I was unable to update database entries because of another issue related lack of foreign keys (Integrity constraint violation 1062 updating a Craft 2 database to Craft 3). Your plugin works extremely well. Thank you for developing it.
Hi! Is there any planned support for controlling the visibility of files on a per-volume basis?
After updating craft & plugin version, the bucket url is being wrongly generated. For some reason the BUCKET_NAME is being outputted before the HTTP protocol, which is generating an invalid url when uploading assets. I have tested the environment multiple times before and after upgrading the craft/plugin and the error happens as result of the update. All the configuration on the admin asset to be ok.
I've done a quick investigation/debug on the flysystem-aws-s3-v3
- as the plugin uses it under the hood but could not find the root cause it.
I will do further investigation/debug as we need to reach the function that generates the bucket url (should be located in flysystem library/aws s3 client) and check why the bucket name is being added to the HTTP protocol url.
Please check the screenshot below:
Thanks.
When using the default endpoint value provided by Digital Ocean (i.e. 'https://ecabling.nyc3.digitaloceanspaces.com') the S3 plugin duplicates the bucket name in the URL and operations fail (i.e. 'https://ecabling.ecabling.nyc3.digitaloceanspaces.com'). Using an alternate URL (i.e. https://nyc3.digitaloceanspaces.com/ecabling') resolves the upload issue but causes issues elsewhere (url generation).
As far as I can tell, 1 line1 of code fixes this (i just implemented this so not sure if any other side effects). In Volume.php I added the below lines to the createAdapter method:
$config['bucket_endpoint'] = true;
protected function createAdapter()
{
$config = $this->_getConfigArray();
$config['bucket_endpoint'] = true;
$client = static::client($config);
return new AwsS3Adapter($client, $this->getBucket(), $this->getSubfolder());
}
I think this was an issue with Craft rather than the dospaces plugin, but this may help someone.
Discovered a strange bug when running this plugin on a site using Craft version 3.6.10
. All of our volumes were broken, so we couldn't access or upload assets. Trying to upload a file gave us this error -
Upload failed. The error message was: “Error executing "ListObjects" on "mutualcdn.https://ams3.digitaloceanspaces.com"; AWS HTTP error: cURL error 1: Protocol "mutualcdn.https" not supported or disabled in libcurl (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for mutualcdn.https://ams3.digitaloceanspaces.com”
The plugin settings were still the same. It was somehow getting confused about the Endpoint
to use, and prefixing it with the Bucket
value.
After updating Craft to 3.6.11.1
(latest version) it fixed this issue.
I was wondering if your plugin works with https://github.com/nystudio107/craft-imageoptimize and imgix. I haven't been able to get it working with imgix via imageOptimize
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.