#StackBounty: #magento2 #magento-2.1 #checkout #onepage-checkout #performance pages loading slow

Bounty: 100

I’m currently facing a slow page load with the checkout being the slowest of all with:

  • 28 requests
  • 18.5 KB transferred (rest cached from disk or memory)
  • Finish: 15.24s (loader disappears and the user can do something)
  • DOMContentLoaded: 6.45s
  • Load: 10.28s

Loading checkout/cart ends with:

  • 29 requests
  • 28.5 KB transferred (rest cached from disk or memory)
  • Finish: 6.35s
  • DOMContentLoaded: 1.9s
  • Load: 3.79s

whilst an empty cart has this:

  • 22 requests
  • 8.2 KB transferred (rest cached from disk or memory)
  • Finish: 2.78s
  • DOMContentLoaded: 1.22s
  • Load: 2.65s

I’m using redis for caching and all caches are active. Javascript is minified, merged and bundled as well as css and html. The server is pretty well situated with 8 CPUs, 16GB RAM and an SSD. Load etc. is never high enough to be even mentioned. Basically the server is sleeping…

There are just ~80 products and one store. We don’t use the content part of Magento. Just product detail pages, checkout and customer area (as well as backend) is served by Magento. There is a CMS system “in front” of Magento which serves pages in less than a second total including media.

I see that TTFB for the checkout document is already 5.66s. With the Magento profiler enabled I can find that magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body causes most of that time. Although it is pretty unclear what exactly causes it. Here the profiler can not help much (me at least).

magento->routers_match                                                                                                                                                                                                                                                                                            5.347600 5.347600 1  42,063,304 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index                                                                                                                                                                                                                                                    5.143997 5.143997 1  15,976,176 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body                                                                                                                                                                                                                                       5.143980 5.143980 1  15,975,304 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->EVENT:checkout_allow_guest                                                                                                                                                                                                           0.000609 0.000609 1  82.464     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->EVENT:checkout_allow_guest->OBSERVER:checkout_allow_guest                                                                                                                                                                            0.000592 0.000592 1  80.944     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT                                                                                                                                                                                                                               0.106220 0.035407 3  13,085,544 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_load                                                                                                                                                                                                                  0.000666 0.000666 1  136.56     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_xml                                                                                                                                                                                                          0.005658 0.005658 1  174.632    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks                                                                                                                                                                                                       0.099472 0.099472 1  12,755,648 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements                                                                                                                       0.099462 0.099462 1  12,902,296 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements                                                                                                    0.091426 0.091426 1  11,885,184 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements->EVENT:core_layout_block_create_after                                                              0.000169 0.000003 51 -16.128    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements                    0.000609 0.000609 1  52.76      0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements 0.000455 0.000455 1  26.952     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->EVENT:layout_generate_blocks_after                                                                                                                                                                                           0.000297 0.000297 1  16.32      0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->EVENT:layout_generate_blocks_after->OBSERVER:persistent                                                                                                                                                                      0.000279 0.000279 1  14.776     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch                                                                                                                                                                                                                         0.002972 0.002972 1  116.752    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch                                                                                                                                                                                   0.002913 0.002913 1  110.192    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor                                                                                                                                                        0.002900 0.002900 1  108.656    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor->EVENT:clean_cache_by_tags                                                                                                                             0.001175 0.001175 1  77.784     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor->EVENT:clean_cache_by_tags->OBSERVER:invalidate_varnish                                                                                                0.000516 0.000516 1  35.056     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor->EVENT:clean_cache_by_tags->OBSERVER:invalidate_builtin                                                                                                0.000629 0.000629 1  41.08      0

I also enabled the database profiler but no query takes more than 1ms even though there are almost 120 calls. But that adds up to just 120ms then.

I generated an xdebug profile locally and can see that most of time is wasted in autoloading. But I’m not sure if that’s “normal”. I just wonder why file_exists is called so often. As I was under the impression everything is precompiled and cached.
I use composer dump-autoload -o --apcu after compiling di. And of course apc is up and running.

enter image description here

enter image description here

enter image description here

enter image description here

Yet I’m stuck here for now. I don’t seem to be able to figure why TTFB is 5 secs already. And even if I solved that there is still 10 more seconds to explain and fix.
My main guess currently is that these are some xhr requests in the background blocking. Ie. I see estimate-shipping-methods only starts after an intensly long gap of about 7 seconds at 13 seconds and takes about 647ms.
Why does it wait so long to trigger that? Until that finishes the loader animation is displayed making the page seem to “hang”.

estimate-shipping-methods "delay"

checkout page load

Any input is highly welcome! I hope for a page load of 2 – 4 secs max. as that’s what I started with.

Edit:
I fixed the high TTFB. As said I’m using Redis for caching. Also for session caching but it leads to a lockup between two concurring sessions. Actually being the checkout request and the estimate-shipping-request leading to a 5 second timeout. I switched sessions back to normal file mode. Unfortunately this did not fix the long load time at all!
My current best guess is that something makes estimate-shipping-methods wait before being called resulting in a blocking of the page.
Additionally I tried to defer all JS with no change whatsoever.

Edit2:
As @peter-ocallaghan notes the problem with the checkout loading so slow (+10 secs) is actually just a chrome specific problem having the dev-console opened. Checkout/cart does not seem to be affected.


Get this bounty!!!

#StackBounty: #magento2 #magento-2.1 #checkout #onepage-checkout #performance pages loading slow -> checkout finish in 15 secs

Bounty: 100

I’m currently facing a slow page load with the checkout being the slowest of all with:

  • 28 requests
  • 18.5 KB transferred (rest cached from disk or memory)
  • Finish: 15.24s (loader disappears and the user can do something)
  • DOMContentLoaded: 6.45s
  • Load: 10.28s

Loading checkout/cart ends with:

  • 29 requests
  • 28.5 KB transferred (rest cached from disk or memory)
  • Finish: 6.35s
  • DOMContentLoaded: 1.9s
  • Load: 3.79s

whilst an empty cart has this:

  • 22 requests
  • 8.2 KB transferred (rest cached from disk or memory)
  • Finish: 2.78s
  • DOMContentLoaded: 1.22s
  • Load: 2.65s

I’m using redis for caching and all caches are active. Javascript is minified, merged and bundled as well as css and html. The server is pretty well situated with 8 CPUs, 16GB RAM and an SSD. Load etc. is never high enough to be even mentioned. Basically the server is sleeping…

There are just ~80 products and one store. We don’t use the content part of Magento. Just product detail pages, checkout and customer area (as well as backend) is served by Magento. There is a CMS system “in front” of Magento which serves pages in less than a second total including media.

I see that TTFB for the checkout document is already 5.66s. With the Magento profiler enabled I can find that magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body causes most of that time. Although it is pretty unclear what exactly causes it. Here the profiler can not help much (me at least).

magento->routers_match                                                                                                                                                                                                                                                                                            5.347600 5.347600 1  42,063,304 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index                                                                                                                                                                                                                                                    5.143997 5.143997 1  15,976,176 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body                                                                                                                                                                                                                                       5.143980 5.143980 1  15,975,304 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->EVENT:checkout_allow_guest                                                                                                                                                                                                           0.000609 0.000609 1  82.464     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->EVENT:checkout_allow_guest->OBSERVER:checkout_allow_guest                                                                                                                                                                            0.000592 0.000592 1  80.944     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT                                                                                                                                                                                                                               0.106220 0.035407 3  13,085,544 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_load                                                                                                                                                                                                                  0.000666 0.000666 1  136.56     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_xml                                                                                                                                                                                                          0.005658 0.005658 1  174.632    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks                                                                                                                                                                                                       0.099472 0.099472 1  12,755,648 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements                                                                                                                       0.099462 0.099462 1  12,902,296 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements                                                                                                    0.091426 0.091426 1  11,885,184 10,485,760
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements->EVENT:core_layout_block_create_after                                                              0.000169 0.000003 51 -16.128    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements                    0.000609 0.000609 1  52.76      0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->layout_generate_blocks->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements->MagentoFrameworkViewLayout::MagentoFrameworkViewLayout::generateElements->generate_elements 0.000455 0.000455 1  26.952     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->EVENT:layout_generate_blocks_after                                                                                                                                                                                           0.000297 0.000297 1  16.32      0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->LAYOUT->EVENT:layout_generate_blocks_after->OBSERVER:persistent                                                                                                                                                                      0.000279 0.000279 1  14.776     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch                                                                                                                                                                                                                         0.002972 0.002972 1  116.752    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch                                                                                                                                                                                   0.002913 0.002913 1  110.192    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor                                                                                                                                                        0.002900 0.002900 1  108.656    0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor->EVENT:clean_cache_by_tags                                                                                                                             0.001175 0.001175 1  77.784     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor->EVENT:clean_cache_by_tags->OBSERVER:invalidate_varnish                                                                                                0.000516 0.000516 1  35.056     0         
magento->routers_match->CONTROLLER_ACTION:checkout_index_index->action_body->postdispatch->EVENT:controller_action_postdispatch->OBSERVER:customer_visitor->EVENT:clean_cache_by_tags->OBSERVER:invalidate_builtin                                                                                                0.000629 0.000629 1  41.08      0

I also enabled the database profiler but no query takes more than 1ms even though there are almost 120 calls. But that adds up to just 120ms then.

I generated an xdebug profile locally and can see that most of time is wasted in autoloading. But I’m not sure if that’s “normal”. I just wonder why file_exists is called so often. As I was under the impression everything is precompiled and cached.
I use composer dump-autoload -o --apcu after compiling di. And of course apc is up and running.

enter image description here

enter image description here

enter image description here

enter image description here

Yet I’m stuck here for now. I don’t seem to be able to figure why TTFB is 5 secs already. And even if I solved that there is still 10 more seconds to explain and fix.
My main guess currently is that these are some xhr requests in the background blocking. Ie. I see estimate-shipping-methods only starts after an intensly long gap of about 7 seconds at 13 seconds and takes about 647ms.
Why does it wait so long to trigger that? Until that finishes the loader animation is displayed making the page seem to “hang”.

estimate-shipping-methods "delay"

checkout page load

Any input is highly welcome! I hope for a page load of 2 – 4 secs max. as that’s what I started with.

Edit:
I fixed the high TTFB. As said I’m using Redis for caching. Also for session caching but it leads to a lockup between two concurring sessions. Actually being the checkout request and the estimate-shipping-request leading to a 5 second timeout. I switched sessions back to normal file mode. Unfortunately this did not fix the long load time at all!
My current best guess is that something makes estimate-shipping-methods wait before being called resulting in a blocking of the page.
Additionally I tried to defer all js with no change whatsoever.


Get this bounty!!!

#StackBounty: #magento2 #magento-2.1 #catalogsearch #performance #indexer Improve performance – Catalog Search Fulltext

Bounty: 50

Do you have some suggestion how can I improve the performance of the catalogsearch_fulltext indexer for Magento 2.1.X

I already set this indexer to ‘Update By Schedule’ so only the updated products are refreshed thanks to Mview pattern.

I have ElasticSearch as search engine, but it is not really related to it.

I have around

  • 20k products
  • 7k products visible in search by store
  • 31 stores
  • less than 10 new custom searchable attributes

For a full re-index, Magento took around 10 minutes by stores, so around 5 hours for all stores.

I plan to re-index each store in parallel and increase the server CPU and memory because each catalog search index seems independent by stores.

I try to import products with a differential approach, but sometime my customer needs a full product import.

Edit: I did a POC thanks to https://github.com/amphp/parallel reducing from 5 hours to 14 min using the c5.9xlarge EC2 instance with 36vCPU. But I want to know if there is alternative solution. (>1000$ per month)


Get this bounty!!!

#StackBounty: #magento2 #magento-2.1 #paypal-express Paypal express checkout magento 2 – Shipping address is invalid when place order u…

Bounty: 50

I am using magento 2.1.13 and facing issue with paypal express checkout.

When I place and order using China, Japan, and Us. it’s working but when I place an order using Russian country with their address it’s showing me error “we can’t place an order” I check debug log and it’s showing “Shipping address is invalid.” I try to many more address from Russia but same issue is there. here is paypal log.

 array (
  'url' => 'https://api-3t.sandbox.paypal.com/nvp',
  'DoExpressCheckoutPayment' => 
  array (
    'TOKEN' => 'EC-33D21907WX527562D',
    'PAYERID' => 'UTZY4EHG8787G',
    'PAYMENTACTION' => 'Authorization',
    'AMT' => '13.00',
    'CURRENCYCODE' => 'USD',
    'BUTTONSOURCE' => 'Magento_Cart_Community',
    'NOTIFYURL' => 'https://mysite/paypal/ipn/',
    'RETURNFMFDETAILS' => 1,
    'SHIPPINGAMT' => '5.00',
    'ITEMAMT' => '8.00',
    'TAXAMT' => '0.00',
    'L_NUMBER0' => NULL,
    'L_NAME0' => 'Calming oz',
    'L_QTY0' => 1,
    'L_AMT0' => '8.00',
    'BUSINESS' => NULL,
    'EMAIL' => 'pratik.kk@yahoo.com',
    'FIRSTNAME' => 'Pratik',
    'LASTNAME' => 'asasas',
    'MIDDLENAME' => NULL,
    'SALUTATION' => NULL,
    'SUFFIX' => NULL,
    'COUNTRYCODE' => 'RU',
    'STATE' => '',
    'CITY' => 'Nizhnelunohodsk ',
    'STREET' => 'Svetlaya ul. Apt 191',
    'ZIP' => '877561 ',
    'PHONENUM' => '35535353',
    'SHIPTOCOUNTRYCODE' => 'RU',
    'SHIPTOSTATE' => 'AD',
    'SHIPTOCITY' => 'Nizhnelunohodsk ',
    'SHIPTOSTREET' => 'Svetlaya ul. Apt 191',
    'SHIPTOZIP' => '877561 ',
    'SHIPTOPHONENUM' => '35535353',
    'SHIPTOSTREET2' => '',
    'STREET2' => '',
    'SHIPTONAME' => 'Pratik asasas',
    'ADDROVERRIDE' => 1,
    'METHOD' => 'DoExpressCheckoutPayment',
    'VERSION' => '72.0',
    'USER' => '****',
    'PWD' => '****',
    'SIGNATURE' => '****',
  ),
  'response' => 
  array (
    'TOKEN' => 'EC-33D21907WX527562D',
    'SUCCESSPAGEREDIRECTREQUESTED' => 'false',
    'TIMESTAMP' => '2018-07-03T11:30:03Z',
    'CORRELATIONID' => 'd23e3f82f3c47',
    'ACK' => 'Failure',
    'VERSION' => '72.0',
    'BUILD' => '000000',
    'L_ERRORCODE0' => '10424',
    'L_SHORTMESSAGE0' => 'Transaction refused because of an invalid argument. See additional error messages for details.',
    'L_LONGMESSAGE0' => 'Shipping address is invalid.',
    'L_SEVERITYCODE0' => 'Error',
  ),


Get this bounty!!!

#StackBounty: #magento2 #magento-2.1 #shopping-cart-price-rules #coupon Magento cart price rules conditions not working properly

Bounty: 50

I have a discount of free shipping which I want to apply when the shipping address is in the U.S. and the subtotal is $100 or over.

I am trying to exclude Hawaii and Alaska from receiving the free shipping but even after excluding it, its still giving the free shipping to Alaska and Hawaii.

enter image description here

Can someone help? Am I doing it wrong?


Get this bounty!!!

#StackBounty: #magento-2.1 #sorting #php-7 How to convert Magento 2 Sort By Dropdown to Horizontal List

Bounty: 50

I was trying to Convert Magento 2 Drop down to Horizontal list.

Code Path:

/app/design/frontend/Vendor/Theme/Magento_Catalog/templates/product/list/toolbar/sorter.phtml

Actual Code:

getAvailableOrders() as $_key => $_order): ?> " isOrderCurrent($_key)): ?> selected="selected" > escapeHtml(__($_order)) ?>

What I tried :


Did some research on this and found this LINK

But Couldn’t able to find the Solution. Please help me on this.

Actual Display

Actual Display

Expected Result : On Click Products should sort according to the selection

Expected

Sorter.phtml File

<?php
/**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */

// @codingStandardsIgnoreFile

?>
<?php
/**
 * Product list toolbar
 *
 * @var $block MagentoCatalogBlockProductProductListToolbar
 */
use MagentoCatalogModelProductProductListToolbar;
?>
<?php if ($block->getCurrentDirection() == 'desc'): ?> <a title="<?= /* @escapeNotVerified */ __('Set Ascending Direction') ?>" href="#" class="action sorter-action sort-desc" data-role="direction-switcher" data-value="asc"> <span><?= /* @escapeNotVerified */ __('Set Ascending Direction') ?></span> </a> <?php else: ?> <a title="<?= /* @escapeNotVerified */ __('Set Descending Direction') ?>" href="#" class="action sorter-action sort-asc" data-role="direction-switcher" data-value="desc"> <span><?= /* @escapeNotVerified */ __('Set Descending Direction') ?></span> </a> <?php endif; ?> </div>


Get this bounty!!!