TLS broken - who can prove otherwise :)

ols_skeptic · Mar 19, 2024

Claim
TLS not fit for purpose (messing with this for weeks now with no resolution)

Wish
A setting or two that can be used to resolve the issue

Issue
Maven Uploads to OLS Secure Listener on port 1661 Proxying to upstream server HTTP:41661 =

Maven Uploads to NGINX Secure Listener on port 1661 Proxying to upstream server HTTP:41661 =

Logs
OLS

Bash:

[DEBUG] Using transporter HttpTransporter with priority 5.0 for https://nexus.xyz.global:1661/repository/maven-releases/
[DEBUG] Using connector BasicRepositoryConnector with priority 0.0 for https://nexus.xyz.global:1661/repository/maven-releases/ with username=maven, password=***
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.pom
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.pom (1.9 kB at 208 B/s)
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-noarch.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-static.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-executable.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-sources.jar
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.nar (19 kB at 2.1 kB/s)                                                                                             
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-noarch.nar (19 kB at 2.0 kB/s)
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-sources.jar (19 kB at 2.0 kB/s)
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-executable.nar (2.4 MB at 262 kB/s)
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-static.nar (2.3 MB at 254 kB/s)
Downloading from nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml
Downloaded from nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml (304 B at 34 kB/s)
[DEBUG] Writing tracking file '/home/X5-133/.m2/repository/global/xyz/prod/prod-core-service/resolver-status.properties'
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml (304 B at 33 B/s)

NGINX

Bash:

Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.pom
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.pom (1.9 kB at 14 kB/s)
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-noarch.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-static.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-executable.nar
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-sources.jar
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-sources.jar (19 kB at 169 kB/s)                                                                                     
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-noarch.nar (19 kB at 135 kB/s)
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1.nar (19 kB at 124 kB/s)
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-static.nar (2.3 MB at 9.1 MB/s)
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/1/prod-core-service-1-amd64-linux-gnu-executable.nar (2.4 MB at 8.8 MB/s)
Downloading from nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml
Downloaded from nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml (304 B at 10 kB/s)
[DEBUG] Writing tracking file '/home/X5-133/.m2/repository/global/xyz/prod/prod-core-service/resolver-status.properties'
Uploading to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml
Uploaded to nexus-releases: https://nexus.xyz.global:1661/repository/maven-releases/global/xyz/prod/prod-core-service/maven-metadata.xml (304 B at 4.5 kB/s)

Config
OLS

Bash:

#
# PLAIN TEXT CONFIGURATION FILE
#
# If not set, will use host name as serverName
serverName               
user                      ols
group                     ols
priority                  0
inMemBufSize              60M
swappingDir               /tmp/lshttpd/swap
autoFix503                1
gracefulRestartTimeout    300
mime                      conf/mime.properties
showVersionNumber         0
adminEmails               root@localhost

errorlog logs/error.log {
  logLevel                DEBUG
  debugLevel              0
  rollingSize             10M
  enableStderrLog         1
}

accesslog logs/access.log {
  rollingSize             10M
  keepDays                30
  compressArchive         0
}
indexFiles                index.html, index.php

expires  {
  enableExpires           1
  expiresByType           image/*=A604800,text/css=A604800,application/x-javascript=A604800,application/javascript=A604800,font/*=A604800,application/x-font-ttf=A604800
}

tuning  {
  maxConnections          10000
  maxSSLConnections       10000
  connTimeout             300
  maxKeepAliveReq         10000
  keepAliveTimeout        5
  sndBufSize              0
  rcvBufSize              0
  maxReqURLLen            32768
  maxReqHeaderSize        65536
  maxReqBodySize          2047M
  maxDynRespHeaderSize    32768
  maxDynRespSize          2047M
  maxCachedFileSize       4096
  totalInMemCacheSize     20M
  maxMMapFileSize         256K
  totalMMapCacheSize      40M
  useSendfile             1
  fileETag                28
  enableGzipCompress      1
  compressibleTypes       default
  enableDynGzipCompress   1
  gzipCompressLevel       6
  gzipAutoUpdateStatic    1
  gzipStaticCompressLevel 6
  brStaticCompressLevel   6
  gzipMaxFileSize         10M
  gzipMinFileSize         300

  quicEnable              1
  quicShmDir              /dev/shm
}

fileAccessControl  {
  followSymbolLink        1
  checkSymbolLink         0
  requiredPermissionMask  000
  restrictedPermissionMask 000
}

perClientConnLimit  {
  staticReqPerSec         0
  dynReqPerSec            0
  outBandwidth            0
  inBandwidth             0
  softLimit               10000
  hardLimit               10000
  gracePeriod             15
  banPeriod               300
}

CGIRLimit  {
  maxCGIInstances         20
  minUID                  11
  minGID                  10
  priority                0
  CPUSoftLimit            10
  CPUHardLimit            50
  memSoftLimit            1460M
  memHardLimit            1470M
  procSoftLimit           400
  procHardLimit           450
}

accessDenyDir  {
  dir                     /
  dir                     /etc/*
  dir                     /dev/*
  dir                     conf/*
  dir                     admin/conf/*
}

accessControl  {
  allow                   ALL
}

extprocessor lsphp74 {
  type                    lsapi
  address                 uds://tmp/lshttpd/lsphp.sock
  maxConns                100
  env                     PHP_LSAPI_MAX_REQUESTS=500
  env                     PHP_LSAPI_CHILDREN=35
  env                     LSAPI_AVOID_FORK=200M
  initTimeout             60
  retryTimeout            0
  persistConn             1
  respBuffer              0
  autoStart               2
  path                    /usr/bin/lsphp74
  backlog                 100
  instances               1
  priority                0
  memSoftLimit            2047M
  memHardLimit            16384M
  procSoftLimit           1400
  procHardLimit           5000
}

extprocessor Nexus Repository Server {
  type                    proxy
  address                 http://localhost:41661
  maxConns                2000
  pcKeepAliveTimeout      30
  initTimeout             20
  retryTimeout            4
  respBuffer              0
}

module cache {
  internal                1

checkPrivateCache   1
checkPublicCache    1
maxCacheObjSize     10000000
maxStaleAge         200
qsCache             1
reqCookieCache      1
respCookieCache     1
ignoreReqCacheCtrl  1
ignoreRespCacheCtrl 0

enableCache         0
expireInSeconds     3600
enablePrivateCache  0
privateExpireInSeconds 3600
  ls_enabled              1
}

listener NEXUS REPOS (HTTPS) {
  address                 *:1661
  secure                  1
  keyFile                 /afs/global-xyz-tls-certs/global.xyz.certs/nexus/nexus.xyz.global.key
  certFile                /afs/global-xyz-tls-certs/global.xyz.certs/nexus/nexus.xyz.global.crt
  certChain               0
  CACertFile              /afs/global-xyz-tls-certs/global.xyz.ca/telesis/telesis.ca.xyz.global.pem
  sslProtocol             24
  map                     nexus.xyz.global nexus.xyz.global
}

NGINX

Bash:

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include *.conf;

events {
   worker_connections 2048;
}

http {
   log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                     '$status $body_bytes_sent "$http_referer" '
                     '"$http_user_agent" "$http_x_forwarded_for"';

# access_log  /var/log/nginx/access.log  main;
    access_log          off;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;
    types_hash_max_size 4096;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
#   include /etc/nginx/conf.d/*.conf;


    server {
        listen       1661 ssl http2 default_server;
        server_name  _;
        root         /usr/share/nginx/html;

        ssl_certificate "/afs/global-xyz-tls-certs/global.xyz.certs/nexus/nexus.xyz.global.crt";
        ssl_certificate_key "/afs/global-xyz-tls-certs/global.xyz.certs/nexus/nexus.xyz.global.key";
        ssl_session_cache shared:SSL:1m;
        ssl_session_timeout  10m;
        ssl_ciphers PROFILE=SYSTEM;
        ssl_prefer_server_ciphers on;

        location / {
           proxy_pass http://127.0.0.1:41661/;
           client_max_body_size 10M;
        }
    }
}

Cold-Egg · Mar 20, 2024

@ols_skeptic Do you mean that you have an SSL config issue?

ols_skeptic · Mar 20, 2024

@Cold-Egg it's not clear so far to me if this is merely a "config" issue.
I've been using Nexus Repository Manager in HTTP mode for months.
Wanting to protect this instance with a TLS terminating proxy, I of course looked to OLS for all of my needs.

I currently have both OLS and NGINX typically configured to terminate TLS on port 1661 and proxy through to 41661 and can observe the following:

Maven to Nexus Upload: NO PROXY (HTTP, Port 8080) - Full speed Uploads (2.4 MB at 9.2 MB/s)
Maven to Nexus Upload: OLS Proxy (HTTPS, Port 1661) - Impaired Uploads on all files (2.3 MB at 254 kB/s)
Maven to Nexus Upload: NGINX Proxy (HTTPS, Port 1661) - Full speed Uploads (2.4 MB at 8.8 MB/s)

ALL traffic is SAME-MACHINE-LOCAL from Maven client to Nexus Repository Manager backend.

OpenLiteSpeed 1.8.0 / Linux fedora 6.7.9-200.fc39.x86_64 #1 SMP PREEMPT_DYNAMIC x86_64 GNU/Linux

These observations persist across different versions of Maven, Maven Wagon HTTP Client, OLS 1.7/1.8, different Server TLS versions, different networks (separate ISPs/physical routers).

Is it a config issue?
Is it an OLS runtime issue?

I'm here in the hope that experts will offer possible answers and a resolution

Cold-Egg · Mar 21, 2024

So it's not a TLS issue but an upload speed issue. Please create a ticket to support@litespeedtech.com with this forum post link appended. Once resolved, we can share the result here.

ols_skeptic · Mar 21, 2024

@Cold-Egg will do re. the Support Ticket

And you're right

it's not a TLS issue at all as I've now reproduced it over HTTP

OLS/1661/HTTPS/TLS => JETTY/41661/HTTP
nexus-releases::https://nexus.xyz.global:1661/repository/maven-releases/
Uploaded to nexus-releases prod-core-service-1-amd64-linux-gnu-static.nar (2.3 MB at 254 kB/s)
Uploaded to nexus-releases prod-core-service-1-amd64-linux-gnu-executable.nar (2.4 MB at 261 kB/s)

OLS/1662/HTTP ===> JETTY/41661/HTTP
nexus-releases::http://nexus.xyz.global:1662/repository/maven-releases/
Uploaded to nexus-releases prod-core-service-1-amd64-linux-gnu-static.nar (2.3 MB at 255 kB/s)
Uploaded to nexus-releases prod-core-service-1-amd64-linux-gnu-executable.nar (2.4 MB at 262 kB/s)

JETTY/41661/HTTP [Direct Access]
nexus-releases::http://nexus.xyz.global:41661/repository/maven-releases/
Uploaded to nexus-releases prod-core-service-1-amd64-linux-gnu-static.nar (2.3 MB at 11 MB/s)
Uploaded to nexus-releases prod-core-service-1-amd64-linux-gnu-executable.nar (2.4 MB at 10 MB/s)

TLS broken - who can prove otherwise :)

ols_skeptic

New Member

Cold-Egg

Administrator

ols_skeptic

New Member

Cold-Egg

Administrator

ols_skeptic

New Member