many possible dead lock on a new server with a simple script.

#1
Hi,

The mistery "No request delivery notification has been received from LSAPI application, possible dead lock." flooded my error log files
of my openlitespeed server and i can't find a way to fix it after much checking.

I decided to setup another new server on DigitalOcean Ubunut 18.04LTS and run a really simple short script to see how the error is triggerred.

I will appreciate if all of you can weigh in to give your findings in the interest of the community and get the attention of developers,
as this seems to be long running issue on and off.

Setup:
======================
Envirnoment:
1. Ubuntu 18.04.2 (1 cpu 2 Gb ram)
2. LITESPEED/1.4.49 OPEN
3. test script on the web host with no other things installed.

Steps to re-produce the deadlock errors:

1. Create and place the .htaccess at the root of web server.
(with reference about "No abort" settings: https://www.litespeedtech.com/support/wiki/doku.php/litespeed_wiki:php:run-without-timeouts)

# ############# .htaccess ############
# RewriteEngine On

# BEGIN litespeed noabort
# SetEnv noabort 1
# END litespeed noabort

<IfModule Litespeed>
RewriteEngine On
RewriteRule .* - [E=noconntimeout:1]
RewriteRule .* - [E=noabort:1]
</IfModule>

# BEGIN litespeed noabort
#<IfModule rewrite_module>
#RewriteEngine On
#RewriteRule .* - [E=noabort:1]
#</IfModule>
# END litespeed noabort
# ############# .htaccess ############

1. place the really simple test script "test.php" at the root of web server:

Test script:

<?php
// Test OpenliteSpeed deadlock error: No request delivery notification has been received from LSAPI application, possible dead lock.

// ini_set('max_execution_time', '300'); //300 seconds = 5 minutes
ini_set('max_execution_time', '0'); // for infinite time of execution

// ob_end_flush();
// set_time_limit(0);
ini_set('output_buffering', 0);
ini_set('implicit_flush', 1);
ob_end_flush();
ob_start();

// turn on and off echo to screen.
$echo = 1;

// the length duration trigger the deadlocks message, longer duration triggers. shorter may not.
$sleep_sec = 5;

$max = 20 ; $i = 0 ;
echo "Max Loop: $max => ";

while ($i < $max)
{
sleep($sleep_sec);
if ( $i == $max-1 AND $echo == 1)
{
echo "test$i (Completed)";
}
else if($echo == 1)
{
echo "test$i (slept $sleep_sec sec) => ";
}

# header("Status: 200");
ob_flush();
flush();

$i++ ;
}
?>

================ Results ==============
Test Scenarios by Turning on and off echo to screen ($echo = 1 or $echo =0).

off => deadlock message occurs and followed by error 503 "Service unavailable (Server is busy, try again later!)", script failed half way.
on => and sleep_sec > 5, deadlock occurs, script may complete or failed.
on => on and sleep_sec <= 5, deadlock may not occurs, script completed

Question 1: When deadlock error message appear at the error logs (view from the control panel), will the script stop?
Answer: Depends. Script may continues till the end or 503 error (Service Unavailable).

Question 2. What is the effect of the deadlock error?
Answer: No description about this on the error list at Openlitespeed wiki error list :
1. https://www.litespeedtech.com/support/wiki/doku.php/litespeed_wiki:php:execution-errors?s[]=deadlock
2. https://openlitespeed.org/kb/

Question 3. The wiki docu for Openlitespeed No abort setting (https://www.litespeedtech.com/support/wiki/doku.php/litespeed_wiki:php:run-without-timeouts)
mentioned that: "Setting No abort Globally via the WebAdmin => WebAdmin console > Configuration Server > General > External Application Abort"

But the setting is not found after i login, checked throught all place in WebAdmin, there is no "External Application Abort". Where is "External Application Abort"?
=====================================

I am running back ground php script for certain tasks:
1. backup and zip files task.
2. run a batch job => Query DB for records, loop through records, send api request to external host, process, update database, repeat.
it is inevitable my script will run longer.

Also for another server having the same issue, that runs wordpress, i found this URL https://www.wordfence.com/help/advanced/system-requirements/litespeed/
mentioned: "wordfence for wordpress mentioned about litespeed is known for killing process that runs a few sec longer."

I believe we have a common needs above (i.e. sometime our script needs to run longer), but seems the deadlock problems keep appearing and flooded my error logs,
and we don't know if the process are completed or failed and the current fix (no abort settings) is more of trial and error. I tried but i can't get the on abort working too as reported above.

Please test and share your experiences, i see many asked the same problems before but no definite fix or ways to solve this.
The solution is "try this, if it does not work, try that or that".

We need a solution to this and should be a simple solution.

Thank you.
 
#2
Hi

I'm having the same problem with lots of clients too
At least two wordpress-api sites are broken when using OLS because of dead locks. If some script takes too long (a few seconds, actually) to return any data, we get a lot of deadlocks on log and eventually everything dies.

I've searched a lot, but found no fix yet. This was the better topic about the subject I found. I was able to replicate the problem using the OPs script too. :/
 
Top