codeigniter4 / tasks Goto Github PK

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

I can't install using composer as there are no tagged releases and there's not a chance in hell I'm putting "dev-develop" into production as there's no guarantee that every server involved will pull down the same version of the code. Is there a particular reason that there are no tagged releases that composer can make use of?

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

I wonder about the choice to use Closure for the "on-the-fly" task type. It seems unnecessarily restrictive over callable, however when I went to expand it this is the prohibitive piece:

        // Get a hash based on the action
        // Closures cannot be serialized so do it the hard way
        if ($this->getType() === 'closure') {
            $ref  = new \ReflectionFunction($this->getAction());
            $file = new \SplFileObject($ref->getFileName());
            $file->seek($ref->getStartLine() - 1);
            $content = '';

            while ($file->key() < $ref->getEndLine()) {
                $content .= $file->current();
                $file->next();
            }
            $actionString = json_encode([
                $content,
                $ref->getStaticVariables(),
            ]);

I assume this is just a convenient way of identifying the task - could we come up with a way to expand this to handle all callable type?

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

Defining tasks solely in the Config file's init() method requires a single entry point for all scheduling. It might be good to offer a way to extend this to libraries and modules, so (for example) an Auth library could "register" its own tasks for periodic cleanup of spam accounts.

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

Run git fetch --depth=1 origin $GITHUB_BASE_REF
  git fetch --depth=1 origin $GITHUB_BASE_REF
  infection --threads=2 --skip-initial-tests --coverage=build/phpunit --git-diff-base=origin/$GITHUB_BASE_REF --git-diff-filter=AM --logger-github --ignore-msi-with-no-mutations
  shell: /usr/bin/bash -e {0}
  env:
    COMPOSER_PROCESS_TIMEOUT: 0
    COMPOSER_NO_INTERACTION: 1
    COMPOSER_NO_AUDIT: 1
    COMPOSER_AUTH: {"github-oauth": {"github.com": "***"}}
    COMPOSER_CACHE_FILES_DIR: /home/runner/.cache/composer/files

In Process.php line 112:
                                                                               
  The command "'git' 'diff' 'origin/' '--diff-filter' 'AM' '--name-only' '--'  
   'src/'" failed.                                                             
                                                                               
  Exit Code: 128(Invalid exit argument)                                        
                                                                               
  Working directory: /home/runner/work/tasks/tasks                             
                                                                               
  Output:                                                                      
  ================                                                             
                                                                               
                                                                               
  Error Output:                                                                
  ================                                                             
  fatal: bad revision 'origin/'                                                
                                                                               

run [--test-framework TEST-FRAMEWORK] [--test-framework-options TEST-FRAMEWORK-OPTIONS] [-j|--threads THREADS] [--only-covered] [-s|--show-mutations] [--no-progress] [--force-progress] [-c|--configuration CONFIGURATION] [--coverage COVERAGE] [--mutators MUTATORS] [--filter FILTER] [--formatter FORMATTER] [--git-diff-filter GIT-DIFF-FILTER] [--git-diff-lines] [--git-diff-base GIT-DIFF-BASE] [--logger-github [LOGGER-GITHUB]] [--logger-html LOGGER-HTML] [--noop] [--only-covering-test-cases] [--min-msi MIN-MSI] [--min-covered-msi MIN-COVERED-MSI] [--log-verbosity LOG-VERBOSITY] [--initial-tests-php-options INITIAL-TESTS-PHP-OPTIONS] [--skip-initial-tests] [--ignore-msi-with-no-mutations] [--debug] [--dry-run]

Error: Process completed with exit code 1.

https://github.com/codeigniter4/tasks/actions/runs/5154794891/jobs/9283672396

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

I take a look at what been going on here and i wonder if one thing is possible with this.

I work on a project with CI4 which relays heavily on cron to fetch the data from APIs. We're fetching data form about 30 endpoints every minute. And each of the request takes a lot of time to complete the processing. So in other to achieve those I need to spawn a lot of processes to work at the same time. So I have a single command which executed the multiple instances of other command by using something like this

/**
 * Spark Exec
 *
 * @param string   $sparkCommand Command
 * @param int|null $timeout      Timeout
 */
function spark_exec(string $sparkCommand, ?int $timeout = null): void
{
    $command = '';

    if ($timeout) {
        $command .= 'timeout ' . $timeout . ' ';
    }

    $command .= '"' . PHP_BINARY . '" -c "' . (get_cfg_var('cfg_file_path') ?: PHP_CONFIG_FILE_PATH) . '" ' . ROOTPATH . 'spark ' . $sparkCommand . ' > /dev/null &';

    exec($command);
}

And the problem is that I cannot have more that one process working working with the same endpoint and writing the same data db so I have mechanism to put some locks in place so I use files to track the locking

/**
     * Crates the Lockfile with the current class name
     *
     * @param string|null $identifier String to append to the filename
     *
     * @return bool|resource Locked file if successful otherwise false
     */
    public static function lock(?string $identifier = null)
    {
        //Add leading dash to identifier if it is set
        if (isset($identifier)) {
            $identifier = '-' . $identifier;
        }

        //Get the locks file directory
        $lockDir = WRITEPATH . 'cron/locks/';

        //Create the locks file directory if it does not exist
        if (! file_exists($lockDir)) {
            mkdir($lockDir, 0755, true);
        }

        //Get the filename
        $filename = $lockDir . $identifier . '.lock';

        //Open the file for writing to lock it
        $lock = fopen($filename, 'wb');

        if ($lock === false) {
            // Unable to open file, check that you have permissions to open/write
            CLI::error('Unable to write the lock file');

            exit;
        }

        if (flock($lock, LOCK_EX | LOCK_NB) === false) {
            // Lock is already in use by another process
            CLI::error('Another instance is already running');

            exit;
        }

        //Return the locked file
        return $lock;
    }

    /**
     * Closes the file removing the lock
     *
     * @param resource $lock Lock
     */
    public static function unlock(&$lock): void
    {
        //Check if lock exist
        if ($lock && get_resource_type($lock) !== 'Unknown') {
            //Get the filename
            $metaData = stream_get_meta_data($lock);
            $filename = $metaData['uri'];

            //Close the file to remove the lock
            fclose($lock);

            //Remove the lock file
            unlink($filename);
        }
    }

    /**
     * Check if lock files with set prefix exist
     *
     * @param string $prefix filename prefix
     *
     * @return bool Ture if there are lock files otherwise false
     */
    public static function isWorking(string $prefix): bool
    {
        $result = false;

        //We read the locks directory if there are  lock files it means we're still working
        if ($openDir = opendir(WRITEPATH . 'cron/locks/')) {
            while (($file = readdir($openDir)) !== false) {
                if (substr($file, 0, strlen($prefix)) === $prefix) {
                    $result = true;
                    break;
                }
            }
        }

        return $result;
    }

I wonder if there are plans to be able to achieve something similar with the scheduler here. From looking at the code I believe all the scheduled actions run on a single thread. So for example if I have 2 tasks scheduled to run first runs every 5 minutes second task runs every minute. And if first task takes more that 1 minute to be completed. On the next cron run second task will be started but once the first process finishes with task 1 it will continue with the second task in the first process and the order of execution is completely broken.

And there are a lot of real case situations where long running tasks are needed. One example is generating sitemap from the database for the huge site

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

I tried to use this library using composer require codeigniter4/tasks just like the docs say, but it returned the error below:

./composer.json has been updated
Running composer update codeigniter4/tasks
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Root composer.json requires codeigniter4/tasks, it could not be found in any version, there may be a typo in the package name.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.
 - It's a private package and you forgot to add a custom repository to find it  

Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Installation failed, reverting ./composer.json and ./composer.lock to their original content.

Upon troubleshooting, I realised that the package doesn't exist in packagist.org:

Is this an oversight?

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

See https://github.com/lonnieezell/codeigniter4-tasks/blob/develop/docs/vision.md

If you want to work on this, assign the card to yourself and ask any questions here.

The CronExpression class should be able to:

• understand the standard cron format (* * * * *)
• be able to check if a task should be ran based on the cron expression and the current time
• should be able to run based upon a different timezone than the current system settings
• should be able to list future and previous run times.