12

I tried to upgrade my Magento 2.3.5 install to Magento 2.4.0, but am unable to get category pages to display products. Everything is in Docker, so I am using Elasticsearch 7.6 and have specified elasticsearch7 for catalog/search/engine. The test from the Magento admin is successful. The category pages were working with 2.3.5 so I am assuming this is something related to the switch to Elasticsearch. I've looked at various upgrade guides without much luck as reindexing isn't fixing it for me.

The search works and returns products, as does loading a product detail page directly, and viewing the products in the Admin, however viewing any category page just shows:

We can't find products matching the selection.

Things I've tried:

  1. Updated to Magento 2.4 using composer (via Docker) and then ran magento setup:upgrade and cleared cache.

  2. Verified that products are enabled, visible, in stock, have quantity, and are assigned to category

  3. Disabled all non-Magento modules, deleted ./generated/code, ./var/view_preprocessed, changed theme to Magento Blank.

  4. Cleared Cache / Reindexed

bash-5.0$ magento indexer:reindex
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:00
Category Products index has been rebuilt successfully in 00:00:00
Product Categories index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:01
Product EAV index has been rebuilt successfully in 00:00:00
Stock index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Catalog Search index has been rebuilt successfully in 00:00:05

bash-5.0$ magento cache:clean && magento cache:flush
Cleaned cache types:
config
layout
block_html
collections
reflection
db_ddl
compiled_config
eav
customer_notification
config_integration
config_integration_api
full_page
config_webservice
translate
Flushed cache types:
config
layout
block_html
collections
reflection
db_ddl
compiled_config
eav
customer_notification
config_integration
config_integration_api
full_page
config_webservice
translate
  1. Deleted Elasticsearch indices, reindexed, and checked using curl
bash-5.0$ curl elasticsearch:9200/_cat/indices

bash-5.0$ curl elasticsearch:9200/
{
  "name" : "18af19145c92",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "-95jfv_yQNqZWqXk1Z3nBg",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}


bash-5.0$ magento indexer:reindex
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:00
Category Products index has been rebuilt successfully in 00:00:00
Product Categories index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:00
Product EAV index has been rebuilt successfully in 00:00:00
Stock index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Catalog Search index has been rebuilt successfully in 00:00:04


bash-5.0$ curl elasticsearch:9200/_cat/indices
yellow open magento2_product_1_v2 J4HvEAGeTFmxlovr5hl7zw 1 1 694 0 1.6mb 1.6mb

I noticed that there is no "category" index being created but not sure if there should be...

  1. Enabled and disabled Flat Categories

  2. Deleted and re-imported all Products

After deleting and re-importing the products I reindexed, cleared cache, regenerated thumbnails, etc. but still seeing the same behavior.

Edit - I deleted the database and reinstalled from scratch, imported products/categories, cleared cache, etc and the category pages are still not working. Can someone with a working install let me know how many indices are created in Elasticsearch? Can someone provide a same product import CSV that is working correctly for me to test?

doublesharp
  • 606
  • 1
  • 5
  • 13
  • Check \Magento\CatalogSearch\Model\ResourceModel\Fulltext\Collection::_renderFiltersBefore - maybe Elasticsearch throws an error and Magento replaces search result with empty data. – Roman Kabanov Sep 17 '20 at 19:26
  • @RomanKabanov I set a breakpoint on _renderFiltersBefore() and stepped through it, no exceptions there.. $this->_filters is empty as well. Nothing obvious going all the way through to the block. – doublesharp Sep 17 '20 at 19:39
  • Please first find the Elasticsearch status Elasticsearch is running or not. – Chirag Gondaliya Sep 18 '20 at 04:03
  • @ChiragGondaliya the service is running, I included responses from it in the question. – doublesharp Sep 18 '20 at 17:44
  • @doublesharp, there's only one index magento2_product_1_v50. Do you use multi source inventory? I had all empty categories after playing with sources and assigning default website to none-default stock. Stores > Inventory > Stocks. Additionally, how about switching "Display Out of Stock Products: yes" temporarily? Stores > Configuration > Catalog > Inventory > Stock Options – Roman Kabanov Sep 18 '20 at 18:21
  • @RomanKabanov I am just using a vanilla install now (with custom attributes created) and importing a CSV, so I don't think it's using multi-source. If I set "Display Out of Stock Products" to Yes then they show up in the search, but not the categories, setting to No there are no products in either. Makes it seem like it is related to the stock/inventory that I am importing. – doublesharp Sep 18 '20 at 19:12
  • Another weird thing is when I set "Display Products Availability in Stock on Storefront" to Yes it shows the products as in stock when I view them directly, but they only go into the search index if i tell it to show out of stock products, and the categories still aren't working. – doublesharp Sep 18 '20 at 19:38
  • @RomanKabanov I don't see "Manage Sources" anywhere in the code and I don't see it in the Admin so I don't think we have multi source inventory, I'm new to Magento so I'm not sure how to enable it. – doublesharp Sep 18 '20 at 19:48

6 Answers6

34

I located an exception in ./magento/var/log/exception.log that showed an issue with the configuration of a custom attribute. The attribute was set to a type of text for the attribute custom_attribute in the eav_attribute table, but is_filterable set to 1 in the catalog_eav_attribute table. Setting this value to to 0 and reindexing/clearing cache allowed the category pages to load as expected.

[2020-09-18 19:46:47] main.CRITICAL: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_product_1_v10","node":"fqGZNT31RSOuoKWU0r5GXQ","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} {"exception":"[object] (Elasticsearch\\Common\\Exceptions\\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"magento2_product_1_v10\",\"node\":\"fqGZNT31RSOuoKWU0r5GXQ\",\"reason\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}}],\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}}},\"status\":400} at /magento/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:632)"} []
[2020-09-19 17:58:37] main.CRITICAL: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_product_1_v2","node":"6KxafAupTSqGYWZUYyULWg","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400} {"exception":"[object] (Elasticsearch\\Common\\Exceptions\\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"magento2_product_1_v2\",\"node\":\"6KxafAupTSqGYWZUYyULWg\",\"reason\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}}],\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}}},\"status\":400} at /magento/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:632)"} []

Show misconfigured attributes:

SELECT *
FROM catalog_eav_attribute cea 
JOIN eav_attribute ea
  ON ea.attribute_id = cea.attribute_id 
  AND backend_type IN ('text')
WHERE 
  cea.is_filterable = 1 OR cea.is_filterable_in_search = 1

Set is_filterable to 0 to allow category pages to load.

UPDATE catalog_eav_attribute cea 
JOIN eav_attribute ea
  ON ea.attribute_id = cea.attribute_id 
  AND backend_type in ('text')
SET 
  cea.is_filterable = 0, cea.is_filterable_in_search = 0
WHERE 
  cea.is_filterable = 1 OR cea.is_filterable_in_search = 1

Reindex/Refresh Cache

magento indexer:reindex
magento cache:clean
magento cache:flush
doublesharp
  • 606
  • 1
  • 5
  • 13
  • 3
    Thanks a lot for this explanation and the solution here. We had the same issue, but in our case the backend_type was 'varchar'. So the issue could also happen in other cases. Important is the attribute code which is written down in the exception message – norgeindian Oct 05 '20 at 08:36
  • thanks @norgeindian - I updated the answer for handle varchar and text. – doublesharp Nov 12 '20 at 15:55
  • 2
    Just to add to this. is_filterable_in_search set to 1 on a text field also causes this issue – elfling Nov 16 '20 at 16:46
  • @elfling thanks, I added that column to the answer. – doublesharp Nov 16 '20 at 18:06
  • @doublesharp Thank, sql query mentioned by you helped me to identify the issue. – Mukesh Jan 08 '21 at 13:20
  • You should be able to have varchar attributes in filters, because that is what multi select attributes use.

    If you get an error for a varchar attribute it could be the data migration bug that put data in text table for varchar attributes see https://magento.stackexchange.com/questions/325917/elastic-search-illegal-argument-exception-error-text-fields-are-not-optimised

     – iphigenie Jan 25 '21 at 14:44
  • Thanks. I had the same issue after upgrading from 2.1.9 to 2.4.2 (Attribute with ID: "manufacturer" does not exist and need to be created but solution here works until 2.3.6 without ElasticSearch) – cap340 Mar 09 '21 at 05:12
1

Check the details and fix shared by Magento support - https://support.magento.com/hc/en-us/articles/360027356612-Elasticsearch-5-is-configured-but-search-page-does-not-load-with-Fielddata-is-disabled-error

Rohan Patil
  • 111
  • 3
0
Please run following command and then try:

service elasticsearch start
Chirag Gondaliya
  • 1,324
  • 7
  • 20
  • Yes, it is running, if you read the question I tested querying the easticsearch service using curl, and I am able to delete and recreate the indices. The search also works, just not the categories, and I have run a reindex many times. – doublesharp Sep 18 '20 at 17:42
  • It's also running in Docker so it is running in a different container than Magento, and it is running under Alpine Linux so there is now service command. Connecting to the container with docker exec and running ps aux shows the java process though. – doublesharp Sep 18 '20 at 17:43
0

In my case,
The issue is happened because of 2 attribute “jajuma_product_latitude” and “jajuma_product_longitude“ don’t show in store view.
I’ve change 2 parameters of this 2 attributes and run this command:

php bin/magento c:f && php bin/magento indexer:reindex

And the products are show as expected in categories.

enter image description here enter image description here
Hope it help.

fudu
  • 1,278
  • 2
  • 22
  • 41
0

For anyone still struggling with this, I did a clean install and compared the databases. I found that changing website_id in the table cataloginventory_stock to 0 fixed both search and categories on the frontend.

Andrew P
  • 1
  • 1
-4

I got similar issue (no products listing) and solved by following command:

php bin/magento index:reindex
Kashif
  • 285
  • 2
  • 6