How to turn debug on in Directadmin + OLS

[webbasica]

I've been watching these events lately (once a day), where sites are suddenly down and I get a flood of the infamous "No request delivery notification has been received from LSAPI application, possible dead lock." error.

It doesn't seem to be caused by traffic (only) and server resources seem to be OK when it happens, so I'm lost about what's causing it.
I've changed the limits on maxConns (to 10) and env PHP_LSAPI_CHILDREN (first 10, then 5) and that didn't make a big difference.

I disabled lsmcd and that brought back the sites. Also, the error flood stops. Logic tells me, it's related, but can't say for sure.

My question is, how do I debug this error?
lsmcd log shows this:

pool free bucket [53] corruption, at offset: 44042736, invalid value: 369419314, usage: alloc 481, free 642

But doesn't seem to be related.

I have a Centos VPS, running Directadmin. I guess there is a way to turn on debug in OLS? How?

 

[rperper]

If this is an LSMCD product error I might suggest the following:
- We have a new release of LSMCD. You should check it out, compile and install it.
- Before bringing up LSMCD delete your existing shared memory files: `sudo rm -rf /dev/shm/lsmcd`
If the problem continues, please include your `/usr/local/lsmcd/conf/node.conf` file as well as your `/tmp/lsmcd.log` file.
Thanks,

Bob Perper
rperper@litespeedtech.com

 

[webbasica]

Thanks for the reply, but a little context would be nice. Was it LSMCD causing this after all?

 

[rperper]

Sorry, yes context is important here. I responded because you mentioned your lsmcd log and I'm the lsmcd specialist. Let me get your back to an OLS specialist who might understand the initial issue better. You'll get a message shortly.
Thanks.

Bob Perper

 

[rperper]

Hey, let's rule out lsmcd for sure. If you stop the lsmcd service and restart it that ought to free it up. What I might suggest is clearing everything along the way:
```
sudo /usr/local/lsmcd/bin/lsmcdctrl stop
sudo rf -rf /dev/shm/lsmcd
sudo /usr/local/lsmcd/bin/lsmcdctrl start
```
Then try your PHP application again and see if it works out.
Thanks,

Bob Perper
rperper@litespeedtech.com

 

[rperper]

The next step would be to strace the lsphp process to see if it's waiting for lsmcd. We'll need you to try to identify the lsphp process to run this strace if it goes to that point.
Let us know if you're still having that problem and we'll help you with that.
Thanks,

Bob Perper
rperper@litespeedtech.com

 

[webbasica]

I was trying to find out which version of lsmcd I'm running before I install the update (since last month on github you recommended me to update because of another error), so started lsmcd again and the server got stuck. Couldn't even telnet to lsmcd.
Then I cleaned the directory you mentioned `/dev/shm/lsmcd` and lsmcd worked again. So I'm leaning towards lsmcd being the issue.
BTW, the version command says I'm running v 1.0.0

Since you are a lsmcd specialist, how do I find out if lsmcd is doing more harm than good? I don't even know if having made any positive difference.

 

[rperper]

Clearly you need either lsmcd or memcached for your product.
Through telnet you always get v1.0.0 because that's the API version. To get the actual version run:
sudo /usr/local/lsmcd/bin/lsmcd -v
lsmcd is better than memcached if you're using the separate user feature of it, which I suspect you are not or if you need persistence between restart, which I suspect you don't. Memcached does simple caching better than lsmcd.
I'll be glad to work with you on lsmcd, but if you'd like the simpler product, you might do better with vanilla memcached.
If you want to continue with lsmcd you can send to my email address
- Your configuration file: /usr/local/lsmcd/conf/node.conf
- The /tmp/lsmcd.log file.
...and I'll get back to you right away.
Thanks,

Bob Perper
rperper@litespeedtech.com

 

[rperper]

Are you using lsmcd (memcached) because of Litespeed WordPress Caching?

 

[webbasica]

I found the previous VERSION.txt file, it says 1.4.17
I just updated to 1.4.20, but after what you said, I think I'll be better off with vanilla memcached (I'm not using that feature you mentioned).

Are you using lsmcd (memcached) because of Litespeed WordPress Caching?

Yup. All I want is to alleviate some of the load my DBs have to handle.

 

[rperper]

To remove lsmcd:
- sudo /usr/local/lsmcd/bin/lsmcdctrl stop
- sudo systemctrl disable lsmcd
- sudo rm -rf /dev/shm/lsmcd
- sudo rm -rf /tmp/lsmcd
- sudo rm -rf /usr/local/lsmcd
...and you should be clean to install memcached.

 

[webbasica]

Thanks!
Any special steps I should consider when installing memcached, since this is OLS with LSPHP and not Apache?
The main reason I went for LSMCD was to avoid any conflict with OLS and LSPHP.

 

[rperper]

No, it should be fine.