Articles Anywhere

Update of Articles Anywhere consume all server resources

5 months 2 weeks ago #89004

Sukinoz's Avatar Sukinoz

Hello,

- We have Joomla 3.9.12 and all the components updates, except one: Articles Anywhere
- We check the server and everything seems to be OK.
- We update Articles Anywhere
- If we navigate throughout the website all the server resources are consumed and we finally cant access to the web.
- We uninstall it, and install the previous version of Articles Anywhere and goes OK again

We have repeated it several times, and every time the same result.

We discovered this problem a lot of months ago, and we have been testing with new versions of Articles Anywhere, but every time the same result.

I have recorded a video where you can see exactly this:

Confidential information:
(hidden)


I am not and expert in servers, so we asked to the hosting support and they said:
"many calls staying in loop. The calls that remained in a loop blocking the processes were two above all:
public_html / plugins / fields / list / tmpl
public_html / cache / regularlabs /

Processes like /home/cacom/public_html/index.php, are taking a long time to finish (tens of minutes), so they accumulate and come to consume all the resources of the accommodation.
Reviewing the calls that make these processes the truth is that we see many. We have monitored one of them for three minutes, and the files and directories that have had the most access are:
/ home / cacom / public_html / administrator / -> 20436 accesses
/ home / cacom / public_html / administrator / components / -> 20436 accesses
/ home / cacom / public_html / administrator / components / com_fields / -> 20436 accesses
/ home / cacom / public_html / administrator / components / com_fields / helpers / -> 20436 accesses
/home/cacom/public_html/administrator/components/com_fields/helpers/fields.php -> 20436 accesses
/home/cacom/public_html/templates/purity_iii/index.php -> 20682 accesses
/ home / cacom / public_html / templates / -> 20804 accesses
/ home / cacom / public_html / templates / purity_iii / -> 20804 accesses
/ home / cacom / public_html / -> 42232 accesses
/ home / -> 63191 accesses
/ home / cacom / -> 63191 accesses"

In the error_log we have:
[19-Oct-2019 19:47:22 Europe/Berlin] PHP Fatal error: Out of memory (allocated 138911744) (tried to allocate 31429232 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 140
[19-Oct-2019 19:47:55 Europe/Berlin] PHP Fatal error: Out of memory (allocated 71802880) (tried to allocate 31429232 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 140
[19-Oct-2019 19:48:25 Europe/Berlin] PHP Fatal error: Out of memory (allocated 23068672) (tried to allocate 20480 bytes) in /home/cacom/public_html/libraries/joomla/database/driver/mysqli.php on line 891
[19-Oct-2019 19:48:29 Europe/Berlin] PHP Fatal error: Out of memory (allocated 4194304) (tried to allocate 32768 bytes) in /home/cacom/public_html/libraries/joomla/database/query.php on line 1396
[19-Oct-2019 19:48:32 Europe/Berlin] PHP Fatal error: Out of memory (allocated 23068672) (tried to allocate 36864 bytes) in /home/cacom/public_html/components/com_sh404sef/cache/shCacheContent.2f02afd0cd5ae619ab52a94a7d3600b8.php on line 3834
[19-Oct-2019 19:48:36 Europe/Berlin] PHP Fatal error: Out of memory (allocated 96993280) (tried to allocate 31437456 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 130
[19-Oct-2019 19:48:50 Europe/Berlin] PHP Fatal error: Out of memory (allocated 36175872) (tried to allocate 31437456 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 130
[19-Oct-2019 19:48:51 Europe/Berlin] PHP Fatal error: Out of memory (allocated 126328832) (tried to allocate 31429232 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 140
[19-Oct-2019 19:49:02 Europe/Berlin] PHP Fatal error: Out of memory (allocated 36175872) (tried to allocate 31437456 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 130
[19-Oct-2019 19:49:02 Europe/Berlin] PHP Fatal error: Out of memory (allocated 14680064) (tried to allocate 1310720 bytes) in /home/cacom/public_html/administrator/components/com_rsform/helpers/rsform.php on line 3021
[19-Oct-2019 19:49:10 Europe/Berlin] PHP Fatal error: Out of memory (allocated 48758784) (tried to allocate 77824 bytes) in /home/cacom/public_html/plugins/system/sef/sef.php on line 127
[19-Oct-2019 19:49:11 Europe/Berlin] PHP Fatal error: Out of memory (allocated 67608576) (tried to allocate 31429232 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 140
[19-Oct-2019 19:49:15 Europe/Berlin] PHP Fatal error: Out of memory (allocated 128425984) (tried to allocate 31429232 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 140
[19-Oct-2019 19:50:07 Europe/Berlin] PHP Notice: unserialize(): Error at offset 17111656 of 17113055 bytes in /home/cacom/public_html/libraries/src/Cache/Controller/OutputController.php on line 184
[19-Oct-2019 19:50:17 Europe/Berlin] PHP Fatal error: Out of memory (allocated 36175872) (tried to allocate 31459768 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 130
[19-Oct-2019 19:50:18 Europe/Berlin] PHP Fatal error: Out of memory (allocated 172486656) (tried to allocate 31451576 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 192
[19-Oct-2019 19:50:39 Europe/Berlin] PHP Fatal error: Out of memory (allocated 2097152) (tried to allocate 20480 bytes) in /home/cacom/public_html/libraries/loader.php on line 399
[19-Oct-2019 19:50:43 Europe/Berlin] PHP Fatal error: Out of memory (allocated 111673344) (tried to allocate 31459768 bytes) in /home/cacom/public_html/libraries/src/Cache/Storage/FileStorage.php on line 130
[19-Oct-2019 19:50:56 Europe/Berlin] PHP Fatal error: Out of memory (allocated 164917248) (tried to allocate 23887872 bytes) in /home/cacom/public_html/libraries/src/Cache/Controller/OutputController.php on line 212
[19-Oct-2019 19:50:56 Europe/Berlin] PHP Fatal error: Out of memory (allocated 12582912) (tried to allocate 8192 bytes) in /home/cacom/public_html/libraries/src/Installer/Installer.php on line 2170

Any idea?
Thank you

5 months 2 weeks ago #89005

Peter van Westen's Avatar Peter van Westen Admin

You are probably using Articles Anywhere in a way where article A outputs article B which outputs article A, Etc.

Can you give me (super) admin and (s)ftp access so I can take a look?
And tell me on what url I can see/reproduce the issue.
You can use the 'Hidden text' button in the forum editor to hide the confidential information.

5 months 2 weeks ago #89006

Sukinoz's Avatar Sukinoz

Can you give me (super) admin and (s)ftp access so I can take a look?

Confidential information:
(hidden)


And tell me on what url I can see/reproduce the issue.
To reproduce the issue just update Articles Anywhere. The server will go slow until it crash, or until you disable the plugin

We use Articles Anywhere, in combination with Rereplacer for example here:
Confidential information:
(hidden)


{entidad-organiza}
is changed to:
{article}[entity-for-name]{/article}

and

{boton-contacto}
is changed to:
{article}{if show-contact-button}{modal url="index.php?option=com_rsform&view=rsform&formId=54&articleid=[id]&tmpl=component" iframe="true" width="500" height="95%" class="btn btn-primary"}<span class="fa fa-info-circle"></span> Solicita InformaciĆ³n{/modal}{/if}{/article}

5 months 2 weeks ago #89009

Peter van Westen's Avatar Peter van Westen Admin

I would also need the (s)ftp access to be able to see what is going on.

And would it be ok to upgrade to the latest Articles Anywhere version on that site?
I take it this is your live site. So maybe better to reproduce the issue on a test site first.

5 months 2 weeks ago #89021

Sukinoz's Avatar Sukinoz

I have made a copy of the site for testing.

Confidential information:
(hidden)

5 months 2 weeks ago #89026

Peter van Westen's Avatar Peter van Westen Admin

Unfortunately the ftp (and cpanel) login doesn't work. Maybe an IP block?

5 months 2 weeks ago #89027

Sukinoz's Avatar Sukinoz

Sorry, there is a block by countries in the hosting.

From which country are you connecting?

5 months 2 weeks ago #89028

Peter van Westen's Avatar Peter van Westen Admin

Netherlands.

PS: In the meantime, can you check if changing the 'Cache DB Queries' option in the Articles Anywhere system plugin settings makes a difference?

5 months 2 weeks ago #89029

Sukinoz's Avatar Sukinoz

Netherlands is in the whitelist.

I need to contact to the hosting support for more information.

5 months 2 weeks ago #89032

Peter van Westen's Avatar Peter van Westen Admin

FTP and cPanel are still blocked.

5 months 2 weeks ago #89033

Sukinoz's Avatar Sukinoz

I have to tell you that in the test site, i can not reproduce the problem

In the live site, if I upgrade Articles Anywhere, I can see inmediatly the server changes

Confidential information:
(hidden)


Both of them are exactly. I make the copy with Akeeba Backup.

Also I updated Articles Anywhere in the copy site. I navigated through the website and everything seems to be ok. So, I moved the website from the folder /copy to the root , and automaticaly the server resources are consumed.

I can't imagine why the test site is going well, and then just move it to the root and the problems begin.

PD: You should have access now to the FTP. For access Cpanel maybe i need to unblock your ip.

5 months 2 weeks ago #89034

Peter van Westen's Avatar Peter van Westen Admin

Hmmm, that makes it pretty hard, if not impossible to bugfix.
If it works on a different folder with the exact same code, then it seems to me the issue is not in the code.
But probably a server thing causing some issue.

Going by the earlier mentioned errors/feedback from the host, I suspect it has something to do with the caching that is going haywire.
Maybe a folder permissions issue.
See if you can completely delete the /cache and /adminstrator/cache folders. And let Joomla automatically recreate them.
Also see if switching of caching (or changing the caching type) makes any difference.

5 months 2 weeks ago #89035

Sukinoz's Avatar Sukinoz

Thanks for answering.

I will keep you informed.

5 months 11 hours ago #89245

Sukinoz's Avatar Sukinoz

Hello,

Just inform you that we are still making tests with this problem.

Now we have replicated the problem in a test website:

Confidential information:
(hidden)


We updated Articles Anywhere, caches OFF, and some days working ok, but today the problem has appears again. 🙁

5 months 10 hours ago #89246

Peter van Westen's Avatar Peter van Westen Admin

Sorry, but I don't think I can help you with this. If the issue is not reproduceable, it is pretty impossible to figure out what is causing this.

4 months 3 weeks ago #89481

Sukinoz's Avatar Sukinoz

Peter van Westen wrote:

Sorry, but I don't think I can help you with this. If the issue is not reproduceable, it is pretty impossible to figure out what is causing this.

Hello Peter

Two news:

- The problem is reproduced in the test website.
Confidential information:
(hidden)


Details:
- The problem begins when we update from version 8.5.2 of Articles Anywhere. (with 8.5.2 goes ok)
- system cache is off
- Not sure if it is important, but We use Articles Anywhere, Modals, Rereplace and Articles Field (articles and related articles, with custom html views) in different combinations.

This what the hosting support has detected:
(note: custom_field_18 is an Article Custom Field)
You make huge SQL queries continuously of the type:

SELECT items.id,`custom_field_18`.`value` AS `custom_field_entity-for-name`
FROM `jmm_content` AS `items`
LEFT JOIN `jmm_fields_values` AS `custom_field_18`
ON `custom_field_18`.`item_id` = `items`.`id`
AND `custom_field_18`.`field_id` = '18'
WHERE `items`.`id` IN ('1','2','3','4','5','6','7','8','9','10','11','12','13','14','16','17','18','19','21','22','23','24','25','26','27','28','29','30','31','32','33','34','35','37','38','39','40','41','42','43','44','45','46','47','48','50','51','52','53','54','55','56','57','58','59','60','61','62','63','64','65','66','67','68','69','70','71','72','73','74','75','76','77','78'
,'79','80','81','82','83','84','85','86','87','88','89','90','91','92','93','94','95','96','97','98','99','100','101','102','103','104','105','106','108','109','110','111','112','113','114','115','116','117','118','119','120','121','122','123','124','125','126','127','129','130','131','132','133','134','135','136','137','138','139','140','141','142','143','144','145','146','147','
148','149','150','151','152','153','154','156','157','158','159','160','161','162','163','164','165','166','167','168','169','170','171','172','173','174','175','176','177','178','179','180','181','182','183','184','185','186','187','188','189','190','191','192','193','194','195','196','197','198','199','200','201','202','203','204','205','206','207','208','209','210','211','214',
'215','216','217','218','219','220','221','222','223','224','225','226','227','228','229','230','231','232','233','234','235','236','237','238','239','240','241','242','243','244','245','246','247','248','249','264','265','267','275','276','277','282','297','298','303','309','310','334','340','344','346','353','358','360','361','366','367','378','379','403','407','415','416','419'
,'425','428','436','437','439','449','452','453','455','462','473','484','488','492','493','496','503','504','517','532','534','542','544','561','563','571','574','582','585','587','596','598','604','609','615','622','627','628','629','630','633','634','635','657','658','663','673','678','679','680','685','686','688','689','694','701','705','706','713','714','730','750','752','756
','758','760','768','777','778','781','784','785','789','791','797','799','800','801','807','809','810','813','861','876','883','889','894','903','907','908','918','919','920','921','924','928','949','951','957','961','985','986','988','1000','1007','1010','1014','1015','1016','1019','1023','1024','1029','1030','1031','1032','1033','1035','1039','1040','1049','1052','1060','1063',
'1067','1069','1071','1076','1077','1084','1085','1086','1087','1091','1092','1093','1094','1102','1103','1105','1106','1107','1108','1109','1110','1113','1114','1116','1118','1119','1122','1125','1128','1131','1136','1137','1139','1142','1146','1150','1155','1168','1169','1171','1176','1185','1188','1195','1204','1206','1213','1220','1231','1250','1253','1254','1264','1268','1289
','1295','1296','1297','1298','1326','1327','1332','1333','1362','1371','1372','1378','1392','1395','1399','1408','1417','1418','1420','1428','1432','1438','1444','1449','1455','1457','1465','1469','1471','1473','1479','1486','1490','1494','1495','1498','1508','1514','1523','1524','1527','1531','1533','1536','1540','1544','1546','1549','1553','1557','1558','1564','1569','1577','15
86','1587','1590','1592','1597','1598','1603','1604','1606','1607','1608','1615','1620','1623','1625','1637','1641','1642','1645','1652','1653','1654','1656','1661','1664','1667','1669','1670','1676','1677','1680','1684','1685','1689','1690','1692','1694','1695','1699','1710','1712','1714','1715','1716','1725','1726','1732','1733','1738','1739','1744','1766','1770','1771','1775','
1782','1785','1787','1803','1814','1823','1825','1832','1833','1836','1837','1839','1841','1842','1847','1850','1859','1868','1869','1875','1876','1885','1901','1903','1908','1909','1910','1911','1913','1918','1919','1920','1925','1926','1929','1935','1937','1944','1951','1959','1960','1967','1969','1970','1975','1983','1985','1986','1991','1993','1994','1999','2000','2001','2002'
,'2004','2005','2011','2012','2013','2020','2021','2027','2028','2029','2030','2035','2036','2047','2049','2052','2056','2057','2063','2064','2068','2069','2075','2078','2085','2088','2091','2092','2093','2097','2098','2106','2107','2108','2109','2114','2120','2124','2126','2127','2134','2138','2139','2140','2146','2147','2148','2149','2154','2155','2156','2161','2165','2166','217

continues till 18180

and SHOW TABLES LIKE 'jmm_fields_values' y SET `hits` = (`hits` + 1) WHERE `id` = '17005'

I have activate debug mode, but I cant understand the information.

This is the access, if you can help me or give any idea or something to try will be very usefull:

Confidential information:
(hidden)

4 months 3 weeks ago #89485

Peter van Westen's Avatar Peter van Westen Admin

That huge query would indeed be an issue.
But I cannot see where that is happening.
Articles Anywhere is now also outputting the queries it makes in that test website.
There every time it is just looking for data from a single article. Not a massive list as you stated...

So no idea where that is coming from or how to reproduce that.

4 months 2 weeks ago #89532

Sukinoz's Avatar Sukinoz

Hi Peter,

After many tests, and despite the fact that the error occurs only when we update "Articles Anywhere" and that errors associated with "Regular Labs" are registered in the error_log file, as for example "PHP Fatal error: Maximum execution time of 120 seconds exceeded in /libraries/regularlabs/src/StringHelper.php on line 207, ...

In the end it turned out that the error came from some kind of conflict with the BT Content Slider extension!

Anyway ... at least it's already clear.

Thanks for your help and forgive the dizziness

4 months 2 weeks ago #89533

Peter van Westen's Avatar Peter van Westen Admin

Ok, thanks for the heads up.
Yeah, unfortunately some extensions just don't play nicely. And figuring out what causes what can be a pain.

3 months 3 weeks ago #89764

Sukinoz's Avatar Sukinoz

Sukinoz wrote:

Hi Peter,
In the end it turned out that the error came from some kind of conflict with the BT Content Slider extension!

Hello again Peter

I am sorry to bother you again with this issue, but it turns out that it has not been fixed. On the production site, the Update of Articles Anywhere from version 8.5 onwards causes the server to run out of resources.

The technical staff of the hosting has been very friendly and has been researching on the subject.

Finally they tell me that:
We see that in those moments huge queries of the type with a WHERE items.id IN of almost 18 thousand ids.

SELECT items.id, `custom_field_18`.`value` AS` custom_field_entity-for-name`
FROM `jmm_content` AS` items`
.....
WHERE `items`.`id` IN ('1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', .....
.....
'18211', '18212', '18213', '18214', '18215', '18216', '18217', '18218', '18219', '18220', '18221', '18222', '18223 ',' 18224 ',' 18226 ',' 18227 ',' 18228 ')

(note1: we have 18 thousand joomla articles on the website, but only some of them have custom fields)
(note2: custom_field_entity-for-name is a Regular Labs Articles Field)
It seems that the query comes from ./plugins/system/articlesanywhere/src/Collection/Fields/CustomFields.php

We have left you the entire snapshot query now at
Confidential information:
(hidden)


You have to review that query, either creating some kind of index or optimizing the query itself.

I am not a programmer, so I do not know if what I am told is correct or not.

That's why I tell you this, to know your opinion about it.