The Impossible POST Request to my NodeJS API (OLS Bug??)

[verflext]

Guys, I'm struggling with this for days and it got me really, like REALLY really frustrated! And I really tried to figure this out by myself but all my experience in web development seems worthless when it comes to this issue. And I really tried everything! Everything, instead of deleting everything and recreating the whole DigitalOcean droplet.

I have a simple web application. It's React on the frontend and NodeJS on the backend as my "API server" instance. And even a very simple one. It has only one job, one possible request. Not even a database.

It's running fine locally. For my production server I have created a DigitalOcean droplet.
- Ubuntu 18.04.02 LTS
- OpenLiteSpeed 1.4.44 (I can confirm I also had this with 1.4.43)
- NodeJS v8.10.0

I admit, this is the first time I am using OLS. So what have I configured?


1. SSL

I set a Listener for port 443, I entered valid certificate files and at the SSL Protocol Version I checked all checkboxes (SSL v3.0, TLS v1.0, TLS v1.1, TLS v1.3)
I also have a default port 80 Listener.

Both are mapping to my Virtual Host "Whatever" with "my.domain.tld" being in the Domains field.

The Document Root is $VH_ROOT/html (which is /usr/local/lsws/Whatever/html) and the Index Files are "index.html". This is of course where my built react files lie.

I have set the following Rewrite Rules:

rewriteCond %{HTTPS} !on
rewriteRule ^(.*)$ https://%{SERVER_NAME}%{REQUEST_URI} [R,L]

It is supposed to redirect any http:// request with https:// and therefore force https. This works.


2. Context NodeJS Server

Then I learned how to integrate a NodeJS app in OLS. I created the App Server in the Context tab of my Virtual Host.
We have the following settings:

URI: /api/
Location: $VH_ROOT/node (which is /usr/local/lsws/Whatever/node and where my NodeJS application files lie)
Binary Path: /usr/bin/node
Application Type: Node
Startup File: src/index.js


WHAT'S THE PROBLEM?

A while ago. Everything was fine. I was using axios to send api requests to GET /api and received the JSON response. I had no problems, everything was fine. Until I wanted to change the GET to a POST request because it was more appropriate.

The post request kept pending and I was not able to get a response. I will tell you what I tried have tried since that.
After a few minutes I got this error in my console:

VM256:17 POST https://my.domain.com/api/ net::ERR_SPDY_PROTOCOL_ERROR

From the network tab of the developer tools I can also show you the headers (Google Chrome):

General
Request URL: https://my.domain.com/api/
Referrer Policy: no-referrer-when-downgrade
Request Headers
Content-Type: application/json
Origin: https://my.domain.com
Referer: https://my.domain.com/no-referrer
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.109 Safari/537.36
Request Payload
{username: "name", password: "secret"}

1. I suspected a CORS issue, so I made sure to include headers that allowed every request from everywhere.
2. I changed the URL to /api/ (so POST /api/) because I noticed a 301 or 302 redirection when requesting without a trailing slash.
3. I have also tried giving the absolute URL.
4. I got rid of axios and used JavaScript's native fetch() function instead.
5. I got rid of all the code logic inside my NodeJS index.js and set simple responses
6. I tried various browsers (Chrome, Firefox, Safari) on different devices and also in incognito tab.
7. I was even connecting to a VPN from time to time to test things.
8. In Postman the same request was working! You might know that Postman lets you copy the code in your desired language and yep I did that 1:1. Making sure it is using the same headers, the identical request.
9. From that point I was only sending request through console inputs on my website (in the developer tools).
10. I was not using express before, just http.createServer() back then but for the purpose of testing, I deleted my index.js, installed express and copied a dummy post request from a tutorial website.
11. I tried logging. But log4js was not writing anything to the file/ not creating the file. I never understood why. Locally it worked.
12. And of course I checked existing logs, but there was never something usable.

So here we have my index.js:

var express = require('express');
var server = express();

server.use(function(req, res, next) {
    res.header("Access-Control-Allow-Origin", "*");
    res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
    next();
});

server.get('*', function (req, res, next) {
    res.json({msg: 'This is CORS-enabled for all origins!'})
})

server.post('*', function(req, res, next) {
    res.end("yes");
})

server.listen(3000, () => {
    console.log('Listenning at http://localhost:3000' )
})

There has been body-parser and there has been app.use(cors()) before. As I said, I tried everything. But this should work without.

In Postman:

I can GET https://my.domain.com/api/
I can POST https://my.domain.com/api/
I can POST https://my.domain.com/api/ with Body data with the Content-Type application/json

I have copied the code from Postman to submit in the console on my website.

var data = JSON.stringify({
  "username": "value"
});

var xhr = new XMLHttpRequest();
xhr.withCredentials = true;

xhr.addEventListener("readystatechange", function () {
  if (this.readyState === 4) {
    console.log(this.responseText);
  }
});

xhr.open("POST", "https://my.domain.tld/api/");
xhr.setRequestHeader("Content-Type", "application/json");
xhr.setRequestHeader("cache-control", "no-cache");
xhr.setRequestHeader("Postman-Token", "fe673a8c-a619-4378-95ec-d6c1c18d35c0");

xhr.send(data);

In the console of my website:

It is not working. The POST request keeps pending and after minutes it cancels with net::ERR_SPDY_PROTOCOL_ERROR.
GET works!
And what works aswell is sending the POST request without data!!!
xhr.send() works
xhr.send("") works
xhr.send(data) does not!
xhr.send("{}") does not!
xhr.send("a") does not!

=> I find it IMPOSSIBLE to send a POST request WITH Payload FROM my website to the NodeJS server and receiving response.

This is really everything I want: Sending a successfull POST request with data to my NodeJS server. Maybe it is a bug with OLS, but how come I'm the first experiencing this? Because of the net::ERR_SPDY_PROTOCOL_ERROR error, I think it is maybe a problem with the SSL setup, encrypting the payload. At first I was thinking it's a CORS issue, but I did everything to make sure it cannot be. And what could be wrong about my SSL? I'm still a noob when it comes to this. But the certificate works. In chrome the https is not crossed out. It's grey and it says the connection is secure. On the other hand it is working in Postman, so...? I really don't know.

I hope somebody can help.

 

[verflext]

UPDATE: Something about my SSL is wrong. At least the request is working when I use http. I renewed the cert with certbot certonly --webroot -w /usr/local/lsws/Whatever/html/ -d my.domain.tld but no change... Is there any other way to obtain a SSL certificate? Did I even do something wrong?

 

[verflext]

I ordered a new domain and pointed it to the server. I'm not even using a subdomain anymore, it's a clean domain. And from where I got the domain I could download private key, certificate key and intermediate key.

This time I was following this tutorial: https://www.thesslstore.com/knowledgebase/ssl-install/litespeed-ssl-installation/ and did everything like him.

The domain works with SSL but still, the POST request cannot send data (but only from its website, Postman works).

What's going on

 

[Cold-Egg]

Hi @verflext ,

Thanks for your report, we will try to reproduce it locally first and update you here soon.

Best

 

[David]

So you still have the example such as 8088 opening?
Can you test the example page which has an file upload test? such as
http://yourdomain:8088/upload.html
Does this work?

 

[Cold-Egg]

Hi @verflext,

I am not familiar with JS code so just grabed a simple post script and try. It seems works fine with Post request.

const http = require('http');

const server = http.createServer((req, res) => {

    res.end(`

        <!doctype html>

        <html>

        <body>

            <form action="/" method="post">

                <input type="text" name="fname" /><br />

                <input type="number" name="age" /><br />

                <input type="file" name="photo" /><br />

                <button>Save</button>

            </form>

        </body>

        </html>

    `);

});

server.listen(3000);

I can see `Request Method: POST` from browser tool after clicking `Save` button

Please check if this is a correct method to reproduce post request issue.

Best

 

[verflext]

@David David i will check that tomorrow. Maybe I also create a new fresh droplet and test. But it must be with https and my certificate. Http works on my app too.

@Cold-Egg ok, I will try something similiar to this aswell tomorrow

 

[David]

Yes. It is all right for https. Server should handle https without problem.
Thanks.

 

[verflext]

I'm back with the test results!

First, I added a test.html on my server with the form from @Cold-Egg. It is the same like with XHR. It is loading very long and then I'm getting an ERR_SPDY_PROTOCOL_ERROR error.

Then, as suggested by @David I tested the Example application shipped with OLS. Furthermore, I have recreated an entire new droplet with DigitalOcean with OLS & NodeJS preinstalled. I did not change anything and there is also no domain certificate.

I have created a html file with the form in /usr/local/lsws/Example/html/test.html and the action pointing to /node/. Works with http, does not work with https!
I have also sent the form with NodeJS.

Spoiler: app.js

const http = require('http');

const server = http.createServer((req, res) => {
  if (req.method == 'post') {
    res.end('post sent')
  } else {
    res.end(`
        <!doctype html>
        <html>
        <body>
            <form action="/node/" method="post">
                <input type="text" name="fname" /><br />
                <input type="number" name="age" /><br />
                <input type="file" name="photo" /><br />
                <button>Save</button>
            </form>
        </body>
        </html>
    `);
  }
});

server.listen(3000);

And it makes no difference.

I invite you to test it by yourself.
https://138.197.182.151/test.html (html file)
https://138.197.182.151/node/ (html form delivered by nodeJS)

Try to submit the form. It should load very long and then end with the ERR_SPDY_PROTOCOL_ERROR error. At least for me, with Google Chrome. HTTP should work for both links.

 

[Cold-Egg]

Hi @verflext ,

Thanks, I can reproduce this locally by following your steps, will take a deeper look and update you soon.

Best

 

[verflext]

Hi @Cold-Egg. Oh good, it's not just me. Thank you!
But can you explain to me. There were no such remarkable steps by me. How can I be the first user complaining that POSTing with HTTPS does not work? I can only imagine the pre-setup of DigitalOcean was wrong.

Thanks again!

 

[Cold-Egg]

Hi @verflext,

So the issue has been fixed and will be released on 1.4.45 release version, you will see more details on change log.

Best