What is causing this Apache 401 error?

Question

I have Apache 2.4 installed on Debian 9.9, basically doing nothing but serving an instance of Roundcube (currently 1.4 RC-1).

The document root of Apache is /home/web (because reasons), and the Roundcube instance is installed in /home/web/mail.

The Apache user (www-data) has full read/write access to the entire /home/web directory tree.

The document root has an index file that redirects to www.mydomain.co.uk/mail and I'm using mod_rewrite to change any HTTP requests to HTTPS.

Today I noticed (and don't know how long it's been happening) that when I hit mydomain.co.uk it logs a 401 error in Apache:

192.168.10.79 - - [09/Jun/2019:18:49:59 +0100] "GET /mail/ HTTP/1.1" 401 2955 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:67.0)
Gecko/20100101 Firefox/67.0"

This is particularly a problem when accessing it externally as it's triggering a fail2ban rule and banning my clients.

The 401 error means "authentication failed" but I have not set up any authentication on this directory with Apache which AIUI requires the presence of a .htpasswd file - I've checked for the existence of this file in my Apache document root, and there is none - or the use of an auth clause in the virtualhost file - of which also there is none.

How can I find the cause, and prevent, this 401 error?

Some debug information from Fiddler:

GET https://www.mydomain.co.uk/mail/ HTTP/1.1
Accept: text/html, application/xhtml+xml, image/jxr, */*
Accept-Language: en-GB
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Accept-Encoding: gzip, deflate
Host: www.mydomain.co.uk
Connection: Keep-Alive
Cookie: roundcube_sessid=sadh0s1j2rn769rvebhpsv6m81
HTTP/1.0 401 Unauthorized
Date: Mon, 10 Jun 2019 08:27:28 GMT
Server: Apache/2.4.25 (Debian)
Strict-Transport-Security: max-age=63072000; includeSubdomains;
Expires: Mon, 10 Jun 2019 08:27:28 GMT
Cache-Control: private, must-revalidate
Pragma: private
Last-Modified: Mon, 10 Jun 2019 08:27:28 GMT
X-UA-Compatible: IE=edge
X-Frame-Options: sameorigin
Content-Language: en
Vary: Accept-Encoding
X-Robots-Tag: noindex, nofollow
Connection: close
Content-Type: text/html; charset=UTF-8
Content-Length: 5435

If I look in Fiddler under the "Auth" tab:

No Proxy-Authenticate Header is present.
No WWW-Authenticate Header is present.

Some further debug information from wget:

DEBUG output created by Wget 1.18 on linux-gnueabi.
Reading HSTS entries from /home/darren/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'mail' (UTF-8) -> 'mail' (UTF-8)
--2019-06-10 13:34:43--  https://www.mydomain.co.uk/mail
Certificates loaded: 151
Resolving www.mydomain.co.uk (www.mydomain.co.uk)... XX.XX.XX.XX
Caching www.mydomain.co.uk => XX.XX.XX.XX
Connecting to www.mydomain.co.uk (www.mydomain.co.uk)|XX.XX.XX.XX|:443... connected.
Created socket 3.
Releasing 0x004bf2e0 (new refcount 1).
---request begin---
GET /mail HTTP/1.1
User-Agent: Wget/1.18 (linux-gnueabi)
Accept: /
Accept-Encoding: identity
Host: www.mydomain.co.uk
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 301 Moved Permanently
Date: Mon, 10 Jun 2019 12:34:43 GMT
Server: Apache/2.4.25 (Debian)
Strict-Transport-Security: max-age=63072000; includeSubdomains;
Location: https://www.mydomain.co.uk/mail/
Content-Length: 331
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
---response end---
301 Moved Permanently
Registered socket 3 for persistent reuse.
Parsed Strict-Transport-Security max-age = 63072000, includeSubDomains = true
Added new HSTS host: www.mydomain.co.uk:443 (max-age: 63072000, includeSubdomains: true)
URI content encoding = ‘iso-8859-1’
Location: https://www.mydomain.co.uk/mail/ [following]
Skipping 331 bytes of body: [<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a hreSkipping 157 bytes of body: [f="https://www.mydomain.co.uk/mail/">here</a>.</p>
<hr>
<address>Apache/2.4.25 (Debian) Server at www.mydomain.co.uk Port 443</address>
</body></html>
] done.
URI content encoding = None
Converted file name 'mail' (UTF-8) -> 'mail' (UTF-8)
--2019-06-10 13:34:43--  https://www.mydomain.co.uk/mail/
Reusing existing connection to www.mydomain.co.uk:443.
Reusing fd 3.
---request begin---
GET /mail/ HTTP/1.1
User-Agent: Wget/1.18 (linux-gnueabi)
Accept: /
Accept-Encoding: identity
Host: www.mydomain.co.uk
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.0 401 Unauthorized
Date: Mon, 10 Jun 2019 12:34:43 GMT
Server: Apache/2.4.25 (Debian)
Strict-Transport-Security: max-age=63072000; includeSubdomains;
Set-Cookie: roundcube_sessid=9dd09n33d6k3f1kil69f90bkk6; path=/; secure; HttpOnly
Expires: Mon, 10 Jun 2019 12:34:43 GMT
Cache-Control: private, no-cache, no-store, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Last-Modified: Mon, 10 Jun 2019 12:34:43 GMT
X-Frame-Options: sameorigin
Content-Language: en
Vary: Accept-Encoding
X-Robots-Tag: noindex, nofollow
Connection: close
Content-Type: text/html; charset=UTF-8
---response end---
401 Unauthorized
Stored cookie www.mydomain.co.uk -1 (ANY) / <session> <secure> [expiry none] roundcube_sessid 9dd09n33d6k3f1kil69f90bkk6
Disabling further reuse of socket 3.
Username/Password Authentication Failed.
Saving HSTS entries to /home/darren/.wget-hsts

score 1 · Answer 1 · answered Jun 09 '19 at 20:34

The 401 Unauthorized Error is an HTTP response status code indicating that the request sent by the client could not be authenticated, but can have many causes.

I would suggest:

Post the text entire request and response for us to examine. Of special interest is the WWW-Authenticate response header, which contains one or more challenges strings, each indicating how proper authentication can be obtained to access the requested resource.
Check the Apache server logs (may require enabling verbose logging).
If the client is banned, the error code is usually 401.
Clear all cookies and the browser cache on the client browser.
Use the browser with all extensions disabled. In Firefox this is the Safe mode, and in Chrome the Incognito mode.
Search on the server if your application is set to return this error code.

Darren · Accepted Answer · 2019-06-13T08:42:42.960

This is a new "feature" in Roundcube.

We return 401 when user is not logged in. The code is in index.php and it can be modified by a plugin.

Source

These are the offending lines in the Roundcube index.php:

    $plugin = $RCMAIL->plugins->exec_hook('unauthenticated', array(
            'task'      => 'login',
            'error'     => $session_error,
            'http_code' => !$session_error ? 401 : 200
    ));

    $RCMAIL->set_task($plugin['task']);

//    if ($plugin['http_code'] == 401) {
//        header('HTTP/1.0 401 Unauthorized');
//    }

I've commented out the last three lines to stop the 401 error being produced.

What is causing this Apache 401 error?

2 Answers2