10

I'm using X applications over an SSH connection to some machine. When I run an X app, say, xclock for simplicity, I get a console message saying:

Warning: Missing charsets in String to FontSet conversion

... but the app runs. My locales are:

$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Now, I've found this old suggestion to just set export LC_ALL=C. And - that works, the error message goes away. However, I really don't want to change my locale like that!

Is there something I can do to avoid the error and keep my locale?

Additional info:

  • X server is MobaXterm's integrated X server on Windows.
  • remote machine is SLES GNU/Linux 11 (by SUSE).
einpoklum
  • 10,666

2 Answers2

7

You are correct, you don't want to change your locale like that! I found similar suggestions, and chose the rabbit hole instead.

The single locale variable you could change is LC_CTYPE=C (this is the character classification settings, see POSIX section 7, Locale for details).

The main issue here, I believe, is that old X programs have minor font-related issues with contemporary multi-byte/multi-locale systems.

My own locale, on a desktop X Org (Mobaterm uses X Org under the hood) is en_IE.utf (mapped to the file en_US.UTF-8/XLC_LOCALE via /usr/share/X11/locale/locale.dir). Within that XLC_LOCALE file there are fontset and charset definitions, and it is the presence of some of these that (tend to) trigger the warnings (specifically for Cyrillic and Asian encodings which I do not use, and don't have great font coverage of).

I get several errors from some programs, e.g. xfig:

Warning: Missing charsets in String to FontSet conversion
Warning: Missing charsets in String to FontSet conversion
Warning: Unable to load any usable fontset

Commenting out various unwanted fsN and csN entries will fix that (each index N defines a matching fontset/charset pair, maintain this pairing noting each pair is ~200 lines apart). This may affect presentation of certain texts. Or, setting LC_CTYPE=C before starting xfig removes all warnings.

The warnings are issued from XtCvtStringToFontSet()(libXt), as a result of XCreateFontSet() reporting missing character sets (i.e. character sets used/required by the locale, but not found in the font(s) matching the requested specification).

XCreateFontSet() returns the list of missing character sets, but libXt outputs no details of the request or missing character sets to help you track it down. (I cheated using LD_PRELOAD and a wrapper .so for XCreateFontSet() to find the culprits, see below.)

The real issue then is that a fontspec (e.g. "--helvetica-medium-r-normal--16-------") is checked for all of the encodings that are set in the file XLC_LOCALE, and if some cannot be found then you see these warnings. If none are found at all you will see "Warning: Unable to load any usable fontset" too.

Some fixes then:

  1. set LC_CTYPE=C (or other "simple" encoding, e.g en_US) for errant programs
  2. remove unwanted pairs of fontset/charset from the XLC_LOCALE file in effect (locale), this takes effect without restarting X or your desktop
  3. install more fonts and encodings (something of a game of whack-a-mole)
  4. set a more generous application fontspec (via xrdb or command line)
  5. set a more generous default fontspec (via xrdb)

You may need both of the final two fixes, since the warning can be issued if any of the default fonts don't match an encoding at time of check, even if a subsequent font suffices.

One final problem is that some fallback fontspecs might continue to trigger the problem. Specifically, the hardcoded Xt default of:

-*-*-*-R-*-*-*-120-*-*-*-*,*

can fail completely triggering the same warnings. Some quick testing indicates that fontspecs that don't contain a family (including just a single "*") may not match anything, even though matches are expected, i.e. with:

xlsfonts -fn "*-*-*-R-*-*-*-120-*-*-*-*"

And adding a trailing ",*" as a complete wildcard can break a fontspec that would otherwise work. Observed on X Org 1.18.3.

I use this in my .Xdefaults:

XtDefaultFontSet: -*-fixed-medium-r-normal--16-*-*-*-*-*-*-*,-*-fixed-*-*-*--16-*,-*-*-medium-*-*--16-*

which seems to keep most things quiet for me, no XLC_LOCALE changes required. This fallback manages to match every requested encoding across the melange of fonts I have somehow accreted...


Here's the very simple wrapper (with minimal error checking) I used to help track down what is being requested (as it's often only defined in the application source) and what is not being found. Tested on x86_64 Slackware, other distros/systems may need minor adjustment.

gcc -fPIC -shared -Wl,-soname,xfontset.so -ldl -o xfontset.so xfontset.c

then

LD_PRELOAD=$PWD/xfontset.so xfig

to run this from the directory you compiled it in.

#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <link.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>

#include <X11/Xlib.h>

#define DEBUG 1 #define dfprintf(fmt, ...)
do { if (DEBUG) fprintf(stderr, "[%14s#%04d:%8s()] " fmt,
FILE, LINE, func,##VA_ARGS); } while (0)

// gcc -fPIC -shared -Wl,-soname,xfontset.so -ldl -o xfontset.so xfontset.c

// see X11/Xlib.h typedef XFontSet XCreateFontSet_fp( Display display, _Xconst char base_font_name_list, char *missing_charset_list_return, int *missing_charset_count_return, char def_string_return );

static XCreateFontSet_fp *real_XCreateFontSet;

void myinit(void) attribute((constructor));

void myinit(void) { void *dl; if ((dl=dlopen(NULL,RTLD_NOW))) { dfprintf("found dl at %p ()\n", dl);

    real_XCreateFontSet=dlsym(RTLD_NEXT,&quot;XCreateFontSet&quot;);
    if (!real_XCreateFontSet) dfprintf(&quot;error: %s\n&quot;,dlerror());
    dfprintf(&quot;found fn() at %p\n&quot;, (void *)real_XCreateFontSet);
} else {
    dfprintf(&quot;dlopen() failed...\n&quot;);
}

}

// https://www.debian.org/doc/manuals/intro-i18n/ch-examples.en.html // https://invisible-island.net/xterm/xtoolkit/intrinsics.html

XFontSet XCreateFontSet(Display display, _Xconst char base_font_name_list, char *missing_charset_list_return, int *missing_charset_count_return, char def_string_return) { XFontSet rc; int fnum,ii; XFontStruct xfonts; char font_names;

 dfprintf(&quot;wrapped XCreateFontSet()\n&quot;);
 dfprintf(&quot; base_font_name_list: &lt;%s&gt;\n&quot;,base_font_name_list);

 rc=real_XCreateFontSet(display, base_font_name_list, missing_charset_list_return,missing_charset_count_return, def_string_return);

if (*missing_charset_count_return) {
    int ii=0;
    dfprintf(&quot; #missing charsets: %i\n&quot;,*missing_charset_count_return);

    for (ii=0; ii&lt;*missing_charset_count_return; ii++) {
        dfprintf(&quot; %2i: &lt;%s&gt;\n&quot;,ii, (*missing_charset_list_return)[ii]);
    }
}
if (rc) {
    fnum=XFontsOfFontSet(rc,&amp;xfonts,&amp;font_names);
    for (ii=0; ii&lt;fnum; ii++) {
        dfprintf(&quot; XFontSet[%2i]=&lt;%s&gt;\n&quot;,ii,font_names[ii]);  
    }
} else {
    dfprintf(&quot;no FontSet found...\n&quot;);
}
return rc;

}

sampi
  • 212
mr.spuratic
  • 2,758
2

I would like to elevate the priority of the #2 option in the answer by #mr.spuratic, i.e. "edit the relevant XLC_LOCALE file".

The problem really is there. The default libX11 source pretends that everyone using a UTF-8 locale will have their chosen font(s) installed with all the default fallback charset encodings (given in the XLC_LOCALE file, e.g. en_US-UTF-8/XLC_LOCALE) available!

However no default X11 fonts even have all the given charset encodings, and most people outside China, Japan, and Korea will never install any third party fonts with those encodings either, so the code "rightfully" complains that there are missing charsets.

This all happens because way back in 2004 someone (see the source), decided to push the iso10646 choice in en_US-UTF-8/XLC_LOCALE down to the end of the list.

The new selection order presumes someone wants the (primarily) 8-bit encodings to be used even if they have installed a new font with an iso10646 encoding.

Previously the comment in the changed database entry said:

We leave the legacy encodings in for the moment, because we don't have that many ISO10646 fonts yet.

So the older order preferred any fonts with iso10646 encodings first and only fell back if no iso10646 font was available.

For modern eyes the old settings seemed more logical.

Well actually it seems there was some logic at the time to moving the iso10646 encoding last -- but it depended on at least two factors: (1) that there were fonts listed in the relevant fontset for the encodings coming first; and (2) that these other fonts had characters missing from the iso10646 encoding.

But that was a very long time ago -- the state of iso10646-1 support in fonts has vastly improved in the intervening decades!

Any fallback to 8-bit encodings is irrelevant these days when the primary encoding for modern fonts is iso10646, and all modern fonts I know of have all the necessary characters in the 8-bit ranges.

The end result of the fallback logic has been this warning message, which has resulted in vast amounts of lore on the internet everywhere, and a great amount of it is confused, misleading, and more often than not just plain wrong.

So, anyway, if you have good fonts with iso10646 encodings installed then the whole mess of old alternatives in the XLC_LOCALE files should just be removed entirely. Not doing so just leads to ongoing confusion, and possibly poor performance too.

BTW, if you're an ordinary user without permission to edit the necessary XLC_LOCALE file(s), you should be able to just copy the hierarchy into your home directory and set XLOCALEDIR in the environment to point there.

The only big problem is that this has to be done everywhere and anywhere one wants to run any X11 client application while using a UTF-8 locale (that's because the problem is in a configuration file for the core libX11 client library). Though in the end of course this isn't a fatal error -- just a warning that some characters may not display, or may not display correctly (even though it's probably wrong).