Rework RUCSS retry strategy about wp-rocket HOT 7 CLOSED

Scope a solution

To solve this issue, we are planning to overhaul the retry system, and it could be interesting to consider adopting a Strategy design pattern approach to achieve this. Here's the proposed plan:

1. Strategy Design Pattern: We'll utilize the Strategy design pattern, which you can learn more about here. This pattern allows us to define various strategies for handling different API responses.

2. Namespace Organization: All the code related to these strategies will be placed within a dedicated Strategy namespace for better organization.

3. Interface Definition: An interface will be created to establish the contract for all these strategies. This interface serves as a blueprint for all future strategy implementations.

4. Strategy Classes: We will develop multiple classes, each implementing the interface, to represent different strategies for handling API responses. These classes will define what actions to take based on the API result. Which means we will be able to a different strategy depending on the job_result from the API.

5. Controller Integration: Within our application's controller (the "RUCSS Controller"), we will integrate these strategy classes to make decisions on how to handle the API response. Depending on the result received from the API, we will choose the appropriate strategy to execute.

6. Future-Proofing: By adopting the Strategy pattern, we are making our system flexible and ready for future changes. It will be easy to add new strategies or modify existing ones as needed without major code alterations.

While Mathieu says:

In case the job has been found but with no results, the plugin must retry for longer than 3 minutes after the job has been sent. This timing must be filterable, but a good target would be up to 30 minutes.
In case the job has not been found on the SaaS, the plugin must retry once, but not more.
In case of 408, creating a new job must also reset the retry strategy.

We can see here different strategies that will have to be implemented.

For point 2, the ideal approach would be an exponential backoff: Try after 1 minute, then 2 minutes, then 5, then 10, then 10 for instance. Another quick&dirty approach could be to increase the number of retries. Both should be groomed to decide.

I think it would be nice to go with the exponential approach. For this, we can create a new column in the database called not_proceed_before which would be a timestamp.
Then in the strategy, we could check the column retries to get the information to which level we are, and calculate a timestamp stored in not_proceed_before.
Then in the function that process the in-progress jobs here, we can add a check to make sure the not_proceed_before timestamp of the pending job is < now() .

Development steps:

• [] Implement the Strategy Interface,
• [] Implement the Strategies
• [] Use strategies within the RUCSS Controller

Effort estimation:

M

Is a refactor needed in that part of the codebase?

Yes, as explain in the solution

from wp-rocket.

@CrochetFeve0251 , I would say a S for me, but an XS for you and probably others :)

from wp-rocket.

@Miraeld could you provide an estimation of the effort for the exponential waiting time?

from wp-rocket.

I would like to complete the grooming: to switch between different strategies we gonna use a StrategyFactory which gonna take in parameter the response from the API and which return the right strategy based on it.
We could then set the strategy on the context.

Otherwise that seems good to me.

from wp-rocket.

Error code refinement discussed here: https://wp-media.slack.com/archives/CUT7FLHF1/p1698749275154029

from wp-rocket.

I updated the ticket based on the Slack discussion, and added Acceptance Criteria as well to better frame the behavior.

from wp-rocket.

Another quick update to simplify the 404 and 422 behavior (fail immediately).
I documented a recap of the expected behavior on Notion: https://www.notion.so/wpmedia/TDD-RUCSS-SaaS-f69dfa7c2a9f4687a7fe7c2b3920aaa0?pvs=4#b50c86a037754405b5c18ed58704832c

from wp-rocket.