govcms / scaffold-tooling Goto Github PK

View Code? Open in Web Editor NEW

6.0 11.0 16.0 746 KB

PHP 17.15% Shell 82.85%

scaffold-tooling's Introduction

GovCMS Scaffolding

A collection of standard config, scripts and packages to support GovCMS scaffold.

scaffold-tooling's People

Contributors

Stargazers

Watchers

Forkers

simesy larowlan timcoxopc grahamethompson drupal-spider pstampy pandaskii kepol rsau barbun fleetadmiralbutter tara-wij silverham sonnykt francav ruwanl

scaffold-tooling's Issues

Unit test or bats for testing Drupal settings

I have added a basic test that checks to scaffold-tooling can be built into a drupal site with composer, and that the settings values can be accessed with drush.

f41f142#diff-1d37e48f9ceff6d8030570cd36286a61R26-R27

I am looking for someone to suggest a way to check settings values thoroughly, and perhaps some framework around testing the ./scripts as well.

Redis can't be enabled locally (Lagoon not tested)

When starting a project using the govcms8-scaffold-paas template and enabling Redis (Setting ENABLE_REDIS='true' in docker-compose.yml &default-environment and uncommenting the service), there is a PHP error reported:

[govcms8-paas]cli-drupal:/app$ printenv | grep REDIS
ENABLE_REDIS=true
[govcms8-paas]cli-drupal:/app$ drush st
Error: Call to a member function addPsr4() on null in /app/vendor/govcms/scaffold-tooling/drupal/settings/lagoon.settings.php on line 89 #0 /app/web/sites/default/settings.php(54): include()
#1 /app/web/core/lib/Drupal/Core/Site/Settings.php(125): require('/app/web/sites/...')
#2 /app/web/core/lib/Drupal/Core/DrupalKernel.php(1073): Drupal\Core\Site\Settings::initialize('/app/web', 'sites/default', NULL)
#3 /app/web/core/lib/Drupal/Core/DrupalKernel.php(275): Drupal\Core\DrupalKernel->initializeSettings(Object(Symfony\Component\HttpFoundation\Request))
#4 /app/vendor/drush/drush/src/Boot/DrupalBoot8.php(220): Drupal\Core\DrupalKernel::createFromRequest(Object(Symfony\Component\HttpFoundation\Request), Object(Composer\Autoload\ClassLoader), 'prod', true)
#5 /app/vendor/drush/drush/src/Boot/BootstrapManager.php(293): Drush\Boot\DrupalBoot8->bootstrapDrupalConfiguration(Object(Drush\Boot\BootstrapManager), NULL)
#6 /app/vendor/drush/drush/src/Boot/BootstrapManager.php(493): Drush\Boot\BootstrapManager->doBootstrap(3, 6, NULL)
#7 /app/vendor/drush/drush/src/Boot/BootstrapManager.php(427): Drush\Boot\BootstrapManager->bootstrapMax(6)
#8 /app/vendor/drush/drush/src/Boot/BootstrapManager.php(396): Drush\Boot\BootstrapManager->bootstrapToPhaseIndex(-2, Object(Consolidation\AnnotatedCommand\AnnotationData))
#9 /app/vendor/drush/drush/src/Boot/BootstrapHook.php(28): Drush\Boot\BootstrapManager->bootstrapToPhase('max', Object(Consolidation\AnnotatedCommand\AnnotationData))
#10 /app/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php(34): Drush\Boot\BootstrapHook->initialize(Object(Drush\Symfony\DrushArgvInput), Object(Consolidation\AnnotatedCommand\AnnotationData))
#11 /app/vendor/consolidation/annotated-command/src/Hooks/Dispatchers/InitializeHookDispatcher.php(27): Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->callInitializeHook(Object(Drush\Boot\BootstrapHook), Object(Drush\Symfony\DrushArgvInput), Object(Consolidation\AnnotatedCommand\AnnotationData))
#12 /app/vendor/consolidation/annotated-command/src/CommandProcessor.php(145): Consolidation\AnnotatedCommand\Hooks\Dispatchers\InitializeHookDispatcher->initialize(Object(Drush\Symfony\DrushArgvInput), Object(Consolidation\AnnotatedCommand\AnnotationData))
#13 /app/vendor/consolidation/annotated-command/src/AnnotatedCommand.php(289): Consolidation\AnnotatedCommand\CommandProcessor->initializeHook(Object(Drush\Symfony\DrushArgvInput), Array, Object(Consolidation\AnnotatedCommand\AnnotationData))
#14 /app/vendor/symfony/console/Command/Command.php(221): Consolidation\AnnotatedCommand\AnnotatedCommand->initialize(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /app/vendor/symfony/console/Application.php(1000): Symfony\Component\Console\Command\Command->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /app/vendor/symfony/console/Application.php(255): Symfony\Component\Console\Application->doRunCommand(Object(Consolidation\AnnotatedCommand\AnnotatedCommand), Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /app/vendor/symfony/console/Application.php(148): Symfony\Component\Console\Application->doRun(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /app/vendor/drush/drush/src/Runtime/Runtime.php(118): Symfony\Component\Console\Application->run(Object(Drush\Symfony\DrushArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /app/vendor/drush/drush/src/Runtime/Runtime.php(49): Drush\Runtime\Runtime->doRun(Array, Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /app/vendor/drush/drush/drush.php(72): Drush\Runtime\Runtime->run(Array)
#21 /app/vendor/drush/drush/includes/preflight.inc(18): require('/app/vendor/dru...')
#22 phar:///usr/local/bin/drush/bin/drush.php(141): drush_main()
#23 /usr/local/bin/drush(10): require('phar:///usr/loc...')
#24 {main}
Error: Call to a member function addPsr4() on null in include() (line 89 of /app/vendor/govcms/scaffold-tooling/drupal/settings/lagoon.settings.php).
 [warning] Drush command terminated abnormally.

This points to the line adding the Redis classloader - https://github.com/govCMS/scaffold-tooling/blob/2.0.5/drupal/settings/lagoon.settings.php#L89

I've not dug too deep on this, but note that we've recently changed the setup for our drupal-example Redis failover that may provide a cleaner solution - amazeeio/drupal-example#70

Incorrect temporary path

The scaffold incorrectly sets the temporary path for Drupal in
https://github.com/govCMS/scaffold-tooling/blob/develop/drupal/settings/all.settings.php#L37

$settings['file_temporary_path'] = 'sites/default/files/private/tmp';

According to D8.8 changelog, the correct settings is file_temp_path, so the above line must be:

$settings['file_temp_path'] = 'sites/default/files/private/tmp';

Without setting the correct path, the temporary path fallbacks to /tmp it causes issues with exporting webform submissions.

govcms-deploy fallback to install profile

As per internal discussion, allow govcms-deploy script to have an "install this profile" fallback when there is no functional site, nor a way to sync from any database.

Closes govCMS/govcms8-scaffold-paas#25

Make functional tests command useable with PAAS

Currently the functional tests command just outputs to the console with echo

https://github.com/govCMS/scaffold-tooling/blob/8.x-develop/.gitlab-ci-main.yml#L149

This means PAAS projects can't override it without unlocking their own gitlab-ci.yml and parting ways with the scaffold template.

Provide a variable or similar or a way for the command to be overridden without forking the CI file

Database dump error after recent update

> mysqldump: Error: 'Access denied; you need (at least one of) the PROCESS privilege(s) for this operation' when trying to dump tablespaces

Appears when using Drush10, resolved with --extra-dump=--no-tablespaces. Should be noted even with the error the dump appears to be created and valid.

Delete /app/audit-site after running audit

In CI it didn't matter so much, but i've wirred it up to ahoy.yml and it leaves an errant folder.
https://github.com/govCMS/scaffold-tooling/blob/master/scripts/tests/govcms-audit

drupal-extension is a dev tool should live in require-dev

https://github.com/govCMS/scaffold-tooling/blob/master/composer.json#L10

Should be in
https://github.com/govCMS/require-dev as it's a behat tool.

Better output for 'ahoy ship-shape'

Running ahoy ship-shape results in a very long output that can be seen below

Executing /app/vendor/bin/govcms-validate-modules...
GovCMS Validate :: Verify enabled modules

GovCMS Prepare XML report
Successfully generated govcms-validate-modules.xml

[success]: All modules are in expected states
Executing /app/vendor/bin/govcms-validate-active-permissions...
GovCMS Validate :: Disallowed permissions on active site

GovCMS Prepare XML report
Successfully generated govcms-validate-active-permissions.xml

[success]: No elevated permissions detected in configuration.
Executing /app/vendor/bin/govcms-validate-tfa...
GovCMS Validate :: Validate TFA config
[info]: TFA is enabled
[info]: TFA is required for authenticated users

GovCMS Prepare XML report
Successfully generated govcms-validate-tfa.xml

[success]: TFA is enabled and properly configured
Executing /app/vendor/bin/govcms-validate-profile...
GovCMS Validate :: Validate install profile
[info]: config/default/core.extension.yml is using the 'govcms' profile

GovCMS Prepare XML report
Successfully generated govcms-validate-profile.xml

[success]: 'govcms' profile is in use
Executing /app/vendor/bin/govcms-validate-permissions...
GovCMS Validate :: Disallowed permissions
[info]: config/default/user.role.anonymous.yml is valid
[info]: config/default/user.role.authenticated.yml is valid
[info]: config/default/user.role.civictheme_content_approver.yml is valid
[info]: config/default/user.role.civictheme_content_author.yml is valid
[info]: config/default/user.role.civictheme_site_administrator.yml is valid

GovCMS Prepare XML report
Successfully generated govcms-validate-permissions.xml

[success]: No elevated permissions detected in configuration.
Executing /app/vendor/bin/govcms-validate-platform-yml...
GovCMS Validate :: Yaml lint platform files

                            
 [OK] All 1 YAML files contain valid syntax.                                    



 [OK] All 1 YAML files contain valid syntax.                                    



 [OK] All 1 YAML files contain valid syntax.                                    



 [OK] All 1 YAML files contain valid syntax.            
 
---------------------------------------------------
THIS REPEATS FOR 2000 LINES ON THE SITE WITH CONFIG                         
---------------------------------------------------

GovCMS Prepare XML report
Successfully generated govcms-validate-theme-yml.xml

[success]: No YAML issues in theme files.
Executing /app/vendor/bin/govcms-validate-active-tfa...
GovCMS Validate :: Validate TFA config on active site
[info]: TFA is enabled
[info]: TFA is required for authenticated users

GovCMS Prepare XML report
Successfully generated govcms-validate-active-tfa.xml

[success]: TFA is actively enabled and properly configured
Executing /app/vendor/bin/govcms-validate-active-profile...
GovCMS Validate :: Validate profile on active site
[success]: 'govcms' profile is in use
Executing /app/vendor/bin/govcms-validate-theme-modules...
GovCMS Validate :: Scan themes for modules
[info]: web/themes/contrib/adminimal_theme/adminimal_theme.info.yml is valid
[info]: web/themes/custom/custom/mytheme/mytheme.info.yml is valid
[info]: web/themes/custom/custom/myothertheme/myothertheme.info.yml is valid

GovCMS Prepare XML report
Successfully generated govcms-validate-prevent-theme-modules.xml

[success]: No modules detected.
Executing /app/vendor/bin/govcms-validate-illegal-files...
GovCMS Validate :: Illegal files

GovCMS Prepare XML report
Successfully generated govcms-illegal-files.xml

[success]: No illegal files.
Executing /app/vendor/bin/govcms-validate-active-modules...
GovCMS Validate :: Active modules validation

GovCMS Prepare XML report
Successfully generated govcms-validate-active-permissions.xml

[success]: All modules are in expected states
Executing /app/vendor/bin/govcms-validate-php-functions...
GovCMS Validate :: Banned PHP function list


 [OK] No errors                                                                 

Successfully generated govcms-validate-php-functions.xml
[success]: No banned function detected in code.
XML files have been generated at ./.ship-shape:
  .ship-shape//govcms-validate-platform.xml
  .ship-shape//govcms-validate-active-permissions.xml
  .ship-shape//govcms-validate-php-functions.xml
  .ship-shape//govcms-validate-modules.xml
  .ship-shape//govcms-validate-active-tfa.xml
  .ship-shape//govcms-illegal-files.xml
  .ship-shape//govcms-validate-profile.xml
  .ship-shape//govcms-validate-tfa.xml
  .ship-shape//govcms-validate-prevent-theme-modules.xml
  .ship-shape//govcms-validate-theme-yml.xml
  .ship-shape//govcms-validate-permissions.xml

During forklifts, this needs to be run over and over again, until the DB and config are in the good "shape".

Is it possible to:

Batch the checking of config yamls and just print success/error at the end
Add visual separation between completed and the next task - a new line or a separator.
Have an overall status SUCCESS/FAILURE add at the end.

Preflight fails if there is no config

Workaround in .gitlab-ci-inputs.yml

.job-preflight:
  when: on_success
  allow_failure: true

config override directories no longer work with D9

Probably due to The ability to support multiple configuration directories in the $config_directories is deprecated

Change the drush import command from:
drush config:import dev --partial
to:
drush config:import --source=../config/dev --partial

CLAMAV_MODE can never be set to 0

In settings.php:

$clam_mode = getenv('CLAMAV_MODE') ?: 1;

If CLAMAV_MODE has a value of 0 (MODE_DAEMON) - the $clam_mode will always be 1.

Allow to provide custom phpcs.xml file

When using SaaS, sometimes we need to use a base theme (placed into themes/THEMENAME) which does not follow Drupal coding standards OR there may be custom sources that should not be scanned by PHPCS at all.

Currently, there is no way to configure PHPCS and developers cannot properly use CI to check for coding standards since they fail for the base theme.

Is it possible to allow to provide a custom phpcs.xml file in the repo (through an env variable)?

Redis settings cleanup

The redis settings are substantial and a good candidate to move into a standalone file.

They also assume the env LAGOON is set, and that's not really necessary nor over-rideable if not in this context (testing or hosting elsewhere). include a redis.settings.php from all.settings.php if ENABLE_REDIS.

Setup CI

@tobybellwood @simesy
I would like to help to setup CI using CircleCI for this project.

As a minimum, the CI would contain the following:

composer.json validation
PHP code linting for php files using PHPCS
Bash code linting for php files using shellcheck

Once setup, other checks can be added later.

If this is something that you are interested, please setup CI build in CircleCI for this project (no config is required) and I will submit a PR with CI configuration. This is required to allow my fork to run CI in PR.

Config import script doesn't work with drush 11

Problem statement

composer.json lets you install drush 11, but the script for running config import passes an additional 'sync' argument to drush config-import which isn't supported in drush 11.

Proposed resolution

Determine if this is still needed or if sync is the default in versions of drush < 11

Support Drupal 9

I'm working on a paas project and was looking to retain the govcms/scaffold-tooling entry in our composer.json

But its not Drupal 9 compatible because of the following:

clamav - no D9 version (see https://www.drupal.org/project/clamav/issues/3131448)
chosen_lib hard-pinned to 2.6.0 (relaxing this to ~2 would allow Drupal 9)
drush 9 (suggest make this ~9 || ~10)
stage_file_proxy pinned to an alpha version (relax this to ~1)
fast_404 (see #57 and https://www.drupal.org/project/fast_404/issues/3042976)
redis hard-pinned to 1.1.0 (relaxing this to ~1 would allow Drupal 9)
drupal-console requires symfony/console 3, but core require 4+

DEV_MODE might turn off google analytics in prod

Adding tests for drupal settings, I notice that DEV_MODE may cause undesireable development settings to load in a production environment. This would turn off google analytics. Probably minor but I wanted capture it while i noticed it.

Add test: build govcms8-scaffold-paas for new changes.

Clone the scaffold
Add a local reference to scaffold-tooling
(Maybe remove packagist.org: false to allow scaffold-tooling dependencies to resolve)
composer update
Build with docker
Check version of scaffold
Basic connectivity tests with drush.

Evaluate use of dev packages in `require`

Are packages like drupal/console, phar-io/manifest, phar-io/version, webflo/drupal-finder and webmozart/path-util needed in the production auto-loader?

Could they be moved to require-dev?