geta / geta-optimizely-productfeed Goto Github PK
View Code? Open in Web Editor NEWProduct Feed for Optimizely Commerce. Supports custom CSV and Google Product Feed formats.
License: Apache License 2.0
Product Feed for Optimizely Commerce. Supports custom CSV and Google Product Feed formats.
License: Apache License 2.0
It was required to have like 3-4 different product feeds. The current package generates feed on the fly and affects the site performance quite a lot. If it had been done at least 4 times, the site would be down. Also, it does not cache things and the same feed can be requested / generated multiple times.
Implement differently:
The feed generation process often takes quite a long time and it is frustrating to wait for it to finish without any indication about its progress.
It would be great to have some progress indicator that updates the job status bar every now and then.
Should be quite an easy change since we have loaded things to process in a list already.
When using internationalized domain name, feed resolving by URL fails because UriHelper.GetDisplayUrl is used (suitable for display only):
https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.http.extensions.urihelper.getdisplayurl?view=aspnetcore-7.0
It should be replaced with UriHelper.GetEncodedUrl.
Got SQL Execution timeout when FeedEntity has ~40MB size.
This error is discoverable only in hosted environments and with 100% reproducibility.
Log:
Error occurred. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> Microsoft.EntityFrameworkCore.DbUpdateException: An error occurred while saving the entity changes. See the inner exception for details.
---> Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
The statement has been terminated.
---> System.ComponentModel.Win32Exception (258): Unknown error 258
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
at Microsoft.Data.SqlClient.SqlDataReader.get_MetaData()
at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)
at Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior)
at Microsoft.Data.SqlClient.SqlCommand.ExecuteDbDataReader(CommandBehavior behavior)
at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(RelationalCommandParameterObject parameterObject)
at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.Execute(IRelationalConnection connection)
ClientConnectionId:5967d717-c8d8-4be5-b5ff-3c1c21a1b6e9
Error Number:-2,State:0,Class:11
ClientConnectionId before routing:430c2ed5-d826-4e1b-b83b-4d862311cbe9
Routing Destination:d6914a419bd2.tr371.norwayeast1-a.worker.database.windows.net,11049
--- End of inner exception stack trace ---
at Microsoft.EntityFrameworkCore.Update.ReaderModificationCommandBatch.Execute(IRelationalConnection connection)
at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList`1 entriesToSave)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(StateManager stateManager, Boolean acceptAllChangesOnSuccess)
at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.<>c.b__104_0(DbContext _, ValueTuple`2 t)
at Microsoft.EntityFrameworkCore.SqlServer.Storage.Internal.SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
at Geta.Optimizely.ProductFeed.Repositories.FeedRepository.Save(ICollection`1 feedData)
at Geta.Optimizely.ProductFeed.ProcessingPipeline`1.Process(JobStatusLogger logger, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Geta.Optimizely.ProductFeed.FeedBuilderCreateJob.Execute()
Any suggestions to fix this?
As a temp solution, I think we can override the FeedDbContext and increase the command timeout.
public static class FeedApplicationDbContextFactory
{
public static FeedApplicationDbContext CreateDbContext(string connectionString)
{
var optionsBuilder = new DbContextOptionsBuilder<FeedApplicationDbContext>();
optionsBuilder.UseSqlServer(connectionString, b =>
{
b.CommandTimeout(int.MaxValue);
});
return new FeedApplicationDbContext(optionsBuilder.Options);
}
}
And replace the FeedApplicationDbContext service like this after the services.AddProductFeed() registration:
services.Replace(ServiceDescriptor.Transient(_ => FeedApplicationDbContextFactory.CreateDbContext(cmsOptions.ConnectionString)));
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.