vuestorefront / magento2-vsbridge-indexer Goto Github PK
View Code? Open in Web Editor NEWThis is official Vue Storefront, native, Magento2 indexer
Home Page: https://vuestorefront.io
License: MIT License
This is official Vue Storefront, native, Magento2 indexer
Home Page: https://vuestorefront.io
License: MIT License
The mage2vuestorefront
is importing the created_at
field for products, but the indexer is not. Is there a reason for this?
My colleague created a feature that adds a "New product" badge to products added in the last n days. I couldn't understand how she had been able to get the field to elasticsearch, until I realized she was using mage2vuestorefront and I was using the indexer.
TOD
bin/magento vsbridge:index vsbridge_product_indexer --id 100
Hey guys,
I'm on Magento 2.2.6 - I switched from the mage2vs data pump to the vsbridge-indexer and noticed the data is different for configurable products "configurable_children"
The indexer doesn't include extra text attributes that the data pump did. So like individual color and size text attributes for the child do not appear.
Magento have dimensions for price indexer ...
Add new indexer for exporting reviews to ES
Category Filtering on VSF home page didn't seem to work with the default indexer install using category.name.keyword
on default.json file in VSF
Seems like the product category needs to be changed to TYPE_KEYWORD
on magento2-vsbridge-indexer/src/module-vsbridge-indexer-catalog/Index/Mapping/Product.php
From lines 220:
'category' => [
'type' => 'nested',
'properties' => [
'category_id' => ['type' => FieldInterface::TYPE_LONG],
'position' => ['type' => FieldInterface::TYPE_LONG],
'name' => ['type' => FieldInterface::TYPE_TEXT],
],
],
To
'category' => [
// 'type' => 'nested',
'properties' => [
'category_id' => ['type' => FieldInterface::TYPE_LONG],
'position' => ['type' => FieldInterface::TYPE_LONG],
'name' => [
'type' => FieldInterface::TYPE_TEXT,
'fields' => [
'keyword' => [
'type' => FieldInterface::TYPE_KEYWORD,
'ignore_above' => 256
]
]
]
]
],
This worked for me for vsf 1.8-1.9
this version won't install with Magento 2.3.1 it has a problem with the Magento 2.3.1 ES 6.7.1 version.
The field for ElasticSearch password should be type obscure
and should be encrypted. It should also be excluded from the standard config dump (bin/magento app:config:dump
) as it is considered sensitive data.
Hi,
We are unable to run the magento installer if the plugin has been added by composer. It gives the following error
[Progress: 481 / 877]
Module 'Magento_CatalogSearch':
In PatchApplier.php line 167:
No alive nodes found in your cluster
Once the plugin is disabled in app/etc/config.php then the install process will finish.
More information: vuestorefront/vue-storefront#2577
Hi there! In mage2vuestorefront
we're setting up the product.configurable_options.label
by extrapolating the value_index
to exact attribute label
Can we apply this to magento-2-vsbridge
indexer as well?
Firstly, really great work on this, indexing is really quick.
I noticed that the content types of attribute options differ from the content type provided by mage2vs. This is not an issue per se, but their is a difference i think we should discuss about.
Magento Attribute boolean types are 'true' or 'false', and importing data via mage2vs these are '1' or '0'.
Multiselect attributes with multiple options for a products are imported as arrays, that's great, but the option_ids in this array should be integers but are now strings.
Attribute options that are not filled in Magento are imported as NULL value, while in mage2vs these options are stripped from the dataset
On Magento 2.3.0
version have reindex error:
With magento sample data
bin/magento indexer:reindex
Design Config Grid index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:01
Vsbridge Category Indexer index has been rebuilt successfully in 00:00:01
Vsbridge Attributes Indexer index has been rebuilt successfully in 00:00:03
PHP Fatal error: Uncaught TypeError: Argument 2 passed to Magento\Catalog\Block\Product\ImageFactory::getLabel() must be of the type string, null given, called in /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php on line 154 and defined in /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php:106
Stack trace:
#0 /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php(154): Magento\Catalog\Block\Product\ImageFactory->getLabel(Object(Magento\Catalog\Model\Product\Interceptor), NULL)
#1 /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageBuilder.php(135): Magento\Catalog\Block\Product\ImageFactory->create(Object(Magento\Catalog\Model\Product\Interceptor), 'new_products_co...', Array)
#2 /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/AbstractProduct.php(513): Magento\Catalog\Block\Product\ImageBuilder->create(Object(Magento\Catalog\Model\Product\Interceptor), 'new_products_co...', Array)
#3 /Users/di in /Users/dima/sites/m230/vendor/magento/module-catalog/Block/Product/ImageFactory.php on line 106
<div class="block widget block-products-list grid">
<div class="block-content">
<!-- new_products_content_widget_grid--> <div class="products-grid grid">
<ol class="product-items widget-product-grid">
<li class="product-item"> <div class="product-item-info">
<a href="http://m230.test/selene-yoga-hoodie.html" class="product-item-photo">
How to purge this module completely?
I have tried removing the folder, removing anything named magento2-vsbridge-indexer from composer.json and composer.lock
I have done module:disable.
Was a bit fast without reading that it wasn't tested with 2.3... Now there's a lot of issues.
Edit:
Errors during compilation:
Divante\VsbridgeIndexerCatalog\Model\ResourceModel\Product\AttributeDataProvider
Incompatible argument type: Required type: \Divante\VsbridgeIndexerCatalog\Model\ResourceModel\entityType. Actual type: string; File:
/opt/bitnami/apps/magento/htdocs/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-catalog/Model/ResourceModel/Product/AttributeDataProvider.php
Divante\VsbridgeIndexerCatalog\Model\ResourceModel\Category\AttributeDataProvider
Incompatible argument type: Required type: \Divante\VsbridgeIndexerCatalog\Model\ResourceModel\entityType. Actual type: string; File:
/opt/bitnami/apps/magento/htdocs/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-catalog/Model/ResourceModel/Category/AttributeDataProvider.php
Total Errors Count: 2
In Log.php line 92:
Error during compilation
Dublicated orders in admin html, by creation via REST
Magento 2.2.6
Out of the box Search Engine using Mysql.
Expected result:
See one order in admin html
Actual result:
Multiple orders
tail -f var/log/exception
[2019-03-25 05:47:44] main.CRITICAL: Report ID: webapi-5c986b806ec6e; Message: {"error":{"root_cause":[{"type":"parsing_exception","reason":"[terms] query does not support [5082] within lookup element","line":1,"col":89}],"type":"parsing_exception","reason":"[terms] query does not support [5082] within lookup element","line":1,"col":89},"status":400} {"exception":"[object] (Exception(code: 400): Report ID: webapi-5c986b806ec6e; Message: {\"error\":{\"root_cause\":[{\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89}],\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89},\"status\":400} at /home/vuestorefront/magento2/vendor/magento/framework/Webapi/ErrorProcessor.php:205, Elasticsearch\\Common\\Exceptions\\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89}],\"type\":\"parsing_exception\",\"reason\":\"[terms] query does not support [5082] within lookup element\",\"line\":1,\"col\":89},\"status\":400} at /home/vuestorefront/magento2/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:610)"} []
Hi,
I'm testing this module, to see if it is ready to replace mage2vuestorefront, which we are currently using. But after installing the module and running the Magento indexers, i'm having troubles with linking the elastic indexes generated by the Magento 2 indexer with the VueStorefront API.
Error when searching for data via the API with https://my-vue-storefront.nl/api/catalog/vue_storefront_catalog/_search
{
"error":{
"root_cause":[
{
"type":"index_not_found_exception",
"reason":"no such index",
"resource.type":"index_or_alias",
"resource.id":"vue_storefront_catalog",
"index_uuid":"_na_",
"index":"vue_storefront_catalog"
}
],
"type":"index_not_found_exception",
"reason":"no such index",
"resource.type":"index_or_alias",
"resource.id":"vue_storefront_catalog",
"index_uuid":"_na_",
"index":"vue_storefront_catalog"
},
"status":404
}
I've setup the index name prefix like this:
The indexes are generated with elastic:
My vue-storefront and vue-storefront-api settings:
Are these settings allright? Or have I misinterpreted the docs?
PHP Fatal error: Uncaught TypeError: Argument 2 passed to Divante\VsbridgeIndexerCms\Model\ContentProcessor::parse() must be of the type string, null given, called in /var/www/magento/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-cms/Model/Indexer/Action/CmsBlock.php on line 81 and defined in /var/www/magento/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-cms/Model/ContentProcessor.php:23
Stack trace:
#0 /var/www/magento/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-cms/Model/Indexer/Action/CmsBlock.php(81): Divante\VsbridgeIndexerCms\Model\ContentProcessor->parse(Object(Magento\Widget\Model\Template\FilterEmulate), NULL)
#1 /var/www/magento/vendor/magento/framework/Indexer/SaveHandler/Batch.php(20): Divante\VsbridgeIndexerCms\Model\Indexer\Action\CmsBlock->rebuild('1')
#2 /var/www/magento/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-core/Indexer/GenericIndexerHandler.php(174) in /var/www/magento/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-cms/Model/ContentProcessor.php on line 23
root@ds-sdb01:/var/www/magento# nano /var/www/magento/vendor/divante/magento2-vsbridge-indexer/src/module-vsbridge-indexer-catalog/Model/ResourceModel/Product/Inventory.php
Related to: vuestorefront/vue-storefront#3090
It will be great if we can add the parent_products
to the products (simple
one's) to mark up the parent products (it's probably n:m
relationship, correct?). Without this relationship it's very very hard to find the parent product when you get the simple one
Or maybe there is another way?
Make it compatible with mage2vuestorefront module.
I noticed when running bin/magento vsbridge:reindex --store=1
, the command only runs for the specified store in Magento's context, however it still reindexes all ElasticSearch indexes for all stores.
I discovered this when trying to delete and recreate the index for each store because I had an attribute change its type and the reindex was failing due to the schema mismatch. I tried to run the command above to reindex one store to start with, and noticed all of the other stores throwing errors like this in var/log/vsbridge-indexer/info.log
:
vsbridgeIndexerLogger.ERROR: Bulk index operation failed 1 times in index vue_storefront_catalog_en_us for type product. Error (mapper_parsing_exception) : failed to parse [my_attribute]. Failed doc ids sample : 1.
It does delete the index for only the specified store, as it is using the IndexOperations
to run that command which accepts the store as a param. So the result is each time I run this for a store, it re-runs the index in ES for all stores, but only deletes and recreates the specified store.
I understand the challenge of resolving this as the index action $indexer->reindexAll();
calls Magento's indexer which then calls each of the indexers in these modules. At the moment I don't have any solutions to recommend, but thought I'd post it here for further discussion.
Hopefully the explanation makes sense. I feel like I kind of wrote in circles here. Let me know if you need some clarity.
Indexer is awesome, thank you for it!
But I found special case about catalog promotion.
Steps to reproduce:
I see that product and price in VSF too
but in VSF I still see previous price
When I check Elasticsearch I see correct "final_price"
but VSF get price from "special_price" not from "final_price"
Unfortunately, we can't easily get the color swatches from Magento using API
@afirlejczyk can You please take a look at this and propose some solution? I mean: we should probably put the swatches into attribute metadata and use it in VS frontend (currently we're just mapping the attribute label not using any other metadata)
It would be nice if we could somehow customize indexed attributes. E.g. I would like to fetch configurable_children.media_gallery.
This is more of a question than an issue but it is still an issue if you look at it from my angle @afirlejczyk . The Indices created by this extension does it include the schema needed to work with VSF-API because I have to do that manually on VSF-API or I don't get to see products in vue-storefront. If so, how do I add mappings to this extension so that when the indices are created on this extension they are fully compatible and do not need index rebuilding before they can be used.?
p.s VSF-API is vue-storefront-api
You should be aware that Reviews are stored in the Elasticsearch. To display Reviews correctly, you need to update your mage2vuestorefront
and run the Reviews sync:
node --harmony cli.js reviews
If Divante\VsbridgeIndexerCatalog\Model::getProductUrlSuffix()
returns null
it will cause an issue in Divante\VsbridgeIndexerCatalog\Model::addUrlPath()
.
$rewrite = mb_substr($rewrite, 0, -strlen($urlSuffix));
strlen(null)
will return 0
which in turn leads to mb_substr()
returning an empty string which will be indexed as url_path
for the product.
Starting with VS 1.9 the url_path
is being used for the url routing (vuestorefront/vue-storefront#2446)
The new edition of M2VS is populating these fields by the customizable code snippet:
https://github.com/DivanteLtd/mage2vuestorefront/blob/349c19063264a3fd8dd68c62571b42e32a1e98e9/src/config.js#L5
We should set the product.url_path
to normalized $product->getUrl()
- with protocol, domain and leading /
removed (so just the path without the leading slash); for category - not sure how magento does it by default but it should be just the default magento2 url path for category
Customizable Options of Simple Products are currently not exported at all. They are available in mage2vs.
Mode "Update by Schedule" might not work correctly in Magento Commerce
This is a question, not an issue. Is there a planned release schedule for this module? I see there has been only one release so far, and multiple updates to the develop branch since then. I would like to use some of these updates but would rather not use dev-develop
to avoid unwanted updates.
The usual setup is that we have vueSF on a separate server than Magento.
BUT composer.json (in the state how it is now) requires a specific version of elasticsearch to be installed on the MAGENTO server, then, however, we're able to configure another server host
in Magento plugin settings (Magento admin)
Magento 2.3 uses ES 6.1 by default (https://devdocs.magento.com/guides/v2.3/config-guide/elasticsearch/es-overview.html) which will cause the installation to fail with:
Problem 1
- Installation request for divante/magento2-vsbridge-indexer dev-master -> satisfiable by divante/magento2-vsbridge-indexer[dev-master].
- Conclusion: remove elasticsearch/elasticsearch v6.1.0
- Conclusion: don't install elasticsearch/elasticsearch v6.1.0
- divante/magento2-vsbridge-indexer dev-master requires elasticsearch/elasticsearch ~5.1 -> satisfiable by elasticsearch/elasticsearch[v5.1.0, v5.1.1, v5.1.2, v5.1.3, v5.2.0, v5.3.0, v5.3.1, v5.3.2, v5.4.0].
- Can only install one of: elasticsearch/elasticsearch[v5.1.0, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.1.1, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.1.2, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.1.3, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.2.0, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.3.0, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.3.1, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.3.2, v6.1.0].
- Can only install one of: elasticsearch/elasticsearch[v5.4.0, v6.1.0].
- Installation request for elasticsearch/elasticsearch (locked at v6.1.0) -> satisfiable by elasticsearch/elasticsearch[v6.1.0].
Solution:
Field "final_price" of Configurable Producs (the parent) is always exported with value "0" and therefore sorting and filtering by price is not working as intended.
In mage2vs it is set to the price of the cheapest configurable child.
Currently you have the ability to set the index name prefix which is then appended with the Store ID for each store that is indexed. We were previously using the store code as the unique identifier for each index when working with mage2vs and we hoped to keep this functionality while using this module as well.
The advantage of using store code as opposed to ID is that it is much easier to immediately understand what that index is for. In our situation each store code is the underscore version of a locale code, so seeing vue_storefront_catalog_en_us
is much easier to understand than vue_storefront_catalog_1
. The more stores you have the more this becomes important, especially when working in VS configurations and such.
I have setup the vue-storefront/vue-storefront-api and magento2 server in the same instance in google cloud server. I have followed the steps written in the documentation for installing indexer in magento. I'm running vue-storefront-api and magento server in the docker container but still I'm always getting the "No alive nodes found in your cluster" error. Can anyone help me out here?
Hi there,
I have installed the magento2-vsbridge-indexer. When I run the indexer:reindex for vsbridge_product_indexer
php bin/magento indexer:reindex vsbridge_product_indexer
It show the message:
Vsbridge Product Indexer indexer process unknown error:
Product 9208 is not part of the current product collection..
In case either I disabled the product(xxx) or change its visibility then it show the same message for any other product next to the previous.
Also it does not import all the products to elastic. I have approx 12K product in megento store and it import only approx 5K.
Please guide me how to fix this problem?
Thanks!
According to Magento's technical guidelines, rule 2.5 states "Proxies and interceptors MUST never be explicitly requested in constructors." Now that the M2 coding standards has added this rule the phpcs
linter throws an error in the constructor of RebuildEsIndexCommand when passing in each of the following proxies in __construct
. This requirement is in place because Proxies are generated files and therefore should be lazy loaded.
These should be injected using di.xml
as mentioned in the PR linked above.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.