36

USB keyboards must have been one of the first devices that could be connected to a USB port.

When you are from the US it's possible that you have never faced this problem. But in all other parts of the world you have to select your keyboard layout in the OS. Otherwise it is possible for example that you press 'z' and get 'y' (on German keyboards).

It's obvious that the keyboard could automatically tell the PC its layout. But for some reason such a function has never been part of the standard. Did they forget to add such a function?

Thorbjørn Ravn Andersen
  • 2,262
  • 1
  • 14
  • 25
zomega
  • 5,362
  • 4
  • 25
  • 52
  • 4
    USB 1.0 may be pretty old (IIRC it was designed in 1997), but USB is otherwise current technology. Is this still retrocomputing? – user3840170 Dec 06 '22 at 17:36
  • 2
    @user3840170 The protocol used by keyboards is at least 20 years old. But if there are newer versions of this protocol it is possibly not retro. – zomega Dec 06 '22 at 17:44
  • 4
    The relevant standard (Device Class Definition for HID, v1.11) was last updated in 2001, so I think this is OK. The HID Usage Tables are constantly updated but they don’t add core features. – Stephen Kitt Dec 06 '22 at 19:15
  • 8
    @user3840170 You're right, USB as technology is by no way Retrocomputing, No matter what level. Though, I would classify this rather as a historical question, as it asks for some design choice way back in time, not about applying a still actual technology. – Raffzahn Dec 06 '22 at 23:02
  • 3
    @zomega No, time allone doesn't make it on topic - especially not as all newer levels are still fully compatible. Even most modern controllers contain fall back hardware supporting the first spec. – Raffzahn Dec 06 '22 at 23:03
  • 2
    USB keyboards needed to coexist with traditional PS/2 keyboards for quite a while, and depending on OS, it was the BIOS that did the USB handling and faked a PS/2 interface to the OS. Now imagine the confusion if you swap your PS/2 model with a USB model where the OS needs to translate the layout of the former but not the latter. – Guntram Blohm Dec 07 '22 at 07:28
  • 1
    Also, adapters to connect PS/2 keyboards to USB ports are a thing (and were important when USB was starting out if you had a keyboard that you really liked). Having to make a separate adapter for each layout would have been rather onerous. – jamesdlin Dec 07 '22 at 09:15
  • @Raffzahn There are no newer levels. As StephenKitt wrote the latest standard is from 2001. This is not about USB1/2/3. – zomega Dec 07 '22 at 10:12
  • 3
    Not worth a whole answer, but the absence of this function is obvious to anyone using more than one keyboard layout. A lot of languages have more than one layout (my native has 3 competing layouts, thank you very much, Microsoft), one generally needs to type basic Latin letters anyway and a lot of people generally blind-type on a basic qwerty keyboard. – fraxinus Dec 07 '22 at 11:38
  • 1
    I am glad that this doesn't exist as on more than one occasion I got a "wrong" keyboard fixed by just swapping the keycaps, it would be annoying as hell to always have to again and again tell the OS to use the one I want – PlasmaHH Dec 07 '22 at 12:28
  • 2
    I have a UK ISO QWERTY keyboard, but I generally type COLMAK and swap back to QWERTY when I'm playing games so I don't have to rebind everything... Would be a pain in the arse if I had to reflash the HID firmware every time I wanted to change my layout – ScottishTapWater Dec 07 '22 at 12:58
  • @PlasmaHH Only because the keyboard transmits its layout that doesn't mean it cannot later be overridden in the settings. – zomega Dec 07 '22 at 13:53
  • In the US when we install a new OS, we are asked localization questions by the installer (keyboard, language, time zone) but we can just accept the defaults for the first two, which are en-us. – Todd Wilcox Dec 07 '22 at 15:34
  • @jamesdlin I don’t think those work with the USB HID protocol. – user3840170 Dec 07 '22 at 16:06
  • 1
    I am from Germany and I regularly use Linux live ISOs. I then have to use "/" (divide) from the num block because I do not know where it is on the US keyboard. That's annoying. – zomega Dec 07 '22 at 16:25
  • 2
    @zomega: sure it can, but it would be annoying as hell to all the time have to do it. USB devices are notorious for not being detected as the exact same device being plugged in the last time – PlasmaHH Dec 07 '22 at 16:31

8 Answers8

51

They didn’t forget to add such a function, they chose not to add one. The HID Usage Tables explain this as follows (section 10, page 88):

Note: A general note on Usages and languages: Due to the variation of keyboards from language to language, it is not feasible to specify exact key mappings for every language. Where this list is not specific for a key function in a language, the closest equivalent key position should be used, so that a keyboard may be modified for a different language by simply printing different keycaps. One example is the Y key on a North American keyboard. In Germany this is typically Z. Rather than changing the keyboard firmware to put the Z Usage into that place in the descriptor list, the vendor should use the Y Usage on both the North American and German keyboards. This continues to be the existing practice in the industry, in order to minimize the number of changes to the electronics to accommodate other languages.

Basically, keyboards don’t encode their layout so that manufacturers can change layouts only by swapping keycaps (for layouts with the same number of keys and same physical template).

Stephen Kitt
  • 121,835
  • 17
  • 505
  • 462
  • Comments are not for extended discussion; this conversation has been moved to chat. – Chenmunka Dec 07 '22 at 15:44
  • 21
    It's not just manufacturers, also users. I regularly switch my keyboard layout when I use a different language, typing letters which are not printed on the keyboard. – Paŭlo Ebermann Dec 07 '22 at 22:58
  • 1
    This wouldn't make sense anyway: the layouts not only mix Latin letters, they may also change sets of letters e.g. to Cyrillic or kana, which would require addition of corresponding Usages. – Ruslan Dec 08 '22 at 08:46
  • 1
    I understand the motivation behind OPs question. The issue by OP mostly affects people using non-US keyboards. I have to go through "layout hell," e.g., when using remote desktop because the client computers all have different keyboard layouts. Unfortunately, this is a major problem only for those using non-US keyboards; so any push to solve this problem must come from EU or others, who sadly do not seem to have sufficient influence on global hardware and software designs :( Also, majority of us don't tinker with our keyboards and use it as is. – Aravindh Krishnamoorthy Dec 08 '22 at 09:03
  • 5
    @AravindhKrishnamoorthy: That's a problem with the design of your remote desktop software -- it should allow forwarding either characters or scan codes, at the user's option. – Ben Voigt Dec 08 '22 at 15:58
21

No, they did not forget it.

The HID descriptor does include a bCountryCode field, so it is in the specification, but it is not used for keyboard layout indication.

Keyboards may use it to indicate the localization based on country.

It is just not required and indicating non-localized is allowed for all keyboards.

All keyboards send the same codes for buttons in the same location. This is how it worked even before USB keyboards.

This simply allows the same keyboard hardware to just have different keycaps for different localizations and layouts.

This means it is also not required to have any customized HID descriptor programming needed to keyboard chips, so it is just a matter of building the keyboard with the right keycaps.

So on a German keyboard, the Z button sends exactly the same key code as a US keyboard with the Y button.

This also enables just switching the localization to another type, so even if the keycaps do not match, you can still use a foreign keyboard hardware with your PC with your native layout.

I think the motivation to leave the layout off is that there are countries with many languages and layouts. Many people also speak multiple languages. It would be near impossible to bring in new layouts to USB standards when something changes or updates.

And it would be just as annoying to plug in a keyboard with layout X and select layout X by default, even if you want to use layout Y because it is your native layout.

Peter Mortensen
  • 260
  • 2
  • 7
Justme
  • 31,506
  • 1
  • 73
  • 145
  • 5
    As I understand it, bCountryCode isn’t intended to reflect the keyboard layout, but the language of the keycaps themselves (e.g. “Shift” or “Maj” etc.). The supported values aren’t granular enough to uniquely describe keyboard layouts. – Stephen Kitt Dec 06 '22 at 18:36
  • 1
    @StephenKitt You are right - the bCountryCode was not for defining the actual layout, just for identifying the country code of localized hardware. – Justme Dec 06 '22 at 18:46
  • I have two questions: 1. Is the string "bCountryCode" only used by Linux lsusb? 2. When I run a Linux live Image, why doesn't it automatically use German key mappings? – zomega Dec 06 '22 at 20:25
  • Is there any means via which an input device can indicate that the user has entered a code which should be treated as "Z" without a particular key code, regardless of localization? – supercat Dec 06 '22 at 21:34
  • 1
    @zomega Like I edited, bCountryCode can't be used to detect if you have German or any other layout. Linux live image can't know your keyboard layout so it does not know which layout it should use, so it just uses whatever is the default and the default is US layout. Even if keyboard sets country is German, no OS will likely use that information to set the layout. Which is good, since I would hate if I would plug in a random keyboard and it uses some random layout I could not care less, instead of the one I have already selected. – Justme Dec 06 '22 at 22:56
  • @supercat No and that's the point of keyboards, each normal button has a specific code based on the location and your OS can interpret those button codes in any way you want, and the keyboard can't tell whether some code should be interpreted as 'Y' or 'Z'. – Justme Dec 06 '22 at 23:00
  • @Justme: There are many specialized input devices that should generally be usable to enter text into programs that expect keyboard input, but otherwise have no relationship to a qwerty-board. Requiring that such a device guess at what keyboard configuration the attached computer might be expecting seems like a rather nasty way of doing things, compared with allowing such a device to indicate what Unicode code point has been entered. – supercat Dec 06 '22 at 23:22
  • @supercat That's why there is no guessing, you will need to select your keyboard layout you have in your OS and that's how standard HID devices currently work. I have no problem with that and I can easily use default US layout until I can select what I really have. – Justme Dec 07 '22 at 00:49
  • @supercat that would be useful in a variety of contexts, not just for real keyboards — for example, OTP keyfobs pretending to be keyboards. Unfortunately it isn’t possible, so various workarounds are used; for example, Yubikeys use ModHex, which is a subset of symbols whose scan codes are the same across many keyboard layouts. – Stephen Kitt Dec 07 '22 at 09:08
  • @StephenKitt: Given that the need to emulate scan codes had posed a problem for many kinds of input devices for more than a decade before USB came on scene, I would regard the failure as either a severe oversight or a prioritization of some misguided notion of elegance over practicality. – supercat Dec 07 '22 at 15:50
  • @supercat I wonder whether the USB-IF at the time imagined that the USB HID keyboard class would only be used for keyboards, and that other devices would get their own device class instead of pretending to be keyboards... – Stephen Kitt Dec 07 '22 at 15:54
  • @StephenKitt: If one were to presuppose that the only USB hosts to which one might want to attach an unusual keyboard-style device would be either PCs or Unix boxes, that might almost make sense, but since many other kinds of embedded or specialized devices interfaced with XT-interface or occasionally AT-interface keyboards or devices emulating those interfaces, I would think it would have been natural to expect that in future such devices might interface with USB keyboards or devices emulating those. I wonder if any designs were proposed that would have allowed a keyboard to indicate... – supercat Dec 07 '22 at 16:11
  • ... a "recommended" Unicode translation for a key, along with a keycode, in a fashion somewhat analogous to the way the PC BIOS worked, allowing software to use special handling for keycodes it recognized as requiring it, but using the supplied character codes for others? My favorite text editor prior to Windows 7 used such a split-code design for numeric keypad keys, so that when num-lock was off, the keypad + and - keys could act as (IIRC) search forward/back rather than typing + and - characters, but the top-row + and - would work normally. – supercat Dec 07 '22 at 16:25
18

As a historical side note and an extension to Stephen's Answer, a few historical notes may be helpful:

  • As seen in the citation, it was 'to minimize the number of changes to the electronics' more exactly

    • cost reduction by scale and as well,
    • not changing existing production process.

  • One side effect of the PC keyboard with its location addressing and handling of location to character/function handling on the host side was that, unlike earlier machines (*1), keyboards for different countries only differed in the keycaps put on top of common hardware

    • Keyboards could be produced in great numbers
    • Maximum number for each and every component (including mask-programmed controllers)
    • No low-volume parts needed for small markets
    • Only in the end, when keycaps were put on top (*2), did they become country-specific
    • Keycaps could be even exchanged in the field for very low volume countries.

  • Manufacturers wanted to stay with that model. Adding even a single country-specific identifier would have meant having

    • either different controllers, one per country, or
    • additional external components (switches, cuttable traces, etc.) and
    • different handling way before the keycaps were attached.

  • Around 1997 I worked in a project with Microsoft (one of their team members was (some time before) involved in USB definition and I learned about the effort Microsoft made to push exactly such an identifier). A HID descriptor for this was added, but it was not supported by any keyboard manufacturer.

  • Microsoft still went ahead to support auto detection, except, to keep compatibility with the quirk introduced by unwilling manufacturers, their keyboards had to report US-KBD in the default descriptor. To make it happen, a set of manufacturer-specific descriptors was added. As a result, some later 1990s Microsoft keyboards have their layout auto-detectable.

*1 - Classic terminals and many non-PC computers had either switchable keyboards that produced different codes, depending on country setting, or had to have different decoder chips, to serve different markets.

In fact, character generators (think National MM42xx) and keyboard decoders (NSC MM57xx or GI AY-5-3600, as used in Apple II) were the first major applications of large-scale ROM production.

*2 - Or marked by laser. The Siemens terminal plant, which was also making PCs, started (around 1990) to produce keyboards with only blanks to be later marked by laser according to whatever keyboard version the customer ordered. So it wasn't just a saving for Asian manufacturers using quick hands to press in keycaps to order.

Peter Mortensen
  • 260
  • 2
  • 7
Raffzahn
  • 222,541
  • 22
  • 631
  • 918
  • How adversely would manufacturing efficiency have been affected by having to use a breakaway, punch-out, or other such means to select a keyboard layout when keycaps are installed on high-volume units, or ask a controller to program flash or EEPROM for more specialized ones? I would think being able to order a keyboard with a custom layout, that could be plugged in and simply work with its new layout, would be much more useful than having to configure a custom keyboard driver for every custom layout. – supercat Dec 08 '22 at 18:41
  • custom? one customisable keyboard driver supports every layout. – Jasen Dec 09 '22 at 08:33
  • @Jasen: Even if one has a driver that is configurable for arbitrary keyboard layouts, requiring manual configuration would still be a severe nuisance, especially if access to the configuration would require typing a password at a time when the computer isn't yet configured for the proper keyboard layout. – supercat Dec 09 '22 at 19:01
  • keyboard layout is "settings" whether that is from OS or user provided config files. it's not software than need to be rewriten a hundered times. – Jasen Dec 09 '22 at 20:29
13

It's obvious that the keyboard could automatically tell the PC its layout.

In 2005 I was unlucky enough to work on a Solaris system at a university computer lab in Sweden. Normally, I don't care what's printed on the keyboard. US QWERTY, some other type of QWERTY, AZERTY, QWERTZ — the first thing I do when I have a new login somewhere is to go to the settings and change the layout to Dvorak, because I touch type anyway.

Oh the horror. I was unable to do so on this Solaris. The keyboard informed the OS of its layout — changing it was not possible.

I don't think it was USB. I don't think it was PS/2 either. It was a Solaris keyboard attached to a Solaris computer, probably with a Solaris connector.

I don't remember what I used the computer for that day, but I remember my frustration of being stuck with a keyboard layout I did not choose or prefer. Thank you, USB consortium, for not enabling such behaviour on modern systems.

gerrit
  • 825
  • 1
  • 6
  • 11
  • 5
    Just because the keyboard can inform the computer what layout it is meant for doesn't mean the OS couldn't override that by mapping it to a different one. Seems like a design flaw with Solaris, not with keyboards knowing their own layouts. – Radvylf Programs Dec 07 '22 at 19:38
  • 4
    Most likely a Type 6, or prior (4 or 5), Sun keyboard, which used a proprietory cable/connector. See also Sun Microsystems Keyboard - Type 6 vs Type 7, where someone also wants to use a Type 6 for Dvorak (but on Windows). Also, should the situation ever arise again, see How to change the Keyboard layout on Solaris – Greenonline Dec 08 '22 at 00:02
  • 1
    @RadvylfPrograms I don't know if it was impossible by design, impossible by a bug, or impossible by sysadmin configuration for a mere user to change their own layout; I just remember trying and failing, and asking the IT support who told me it was not possible. It was 17 years, so forgive me for not remembering any more details :) – gerrit Dec 08 '22 at 07:18
  • 1
    Amen to that, brother. Except, I use Neo2 instead of Dvorak. But typing on a standard layout is always pain if you are used to a good layout like Neo2 or Dvorak. In the end, a keyboard is just an array of keys, and it's the sane choice to let the user decide how characters are mapped onto these keys. – cmaster - reinstate monica Dec 08 '22 at 23:27
  • @Greenonline SFAIK xmodmap did not require any special permissions, (but obviously that won't help if you're using the console in text mode). – Jasen Dec 09 '22 at 08:40
  • On the flip side, if your keyboard carried its layout information with it, you wouldn't need to change the layout on every system you you use. You could pitch up and start pair-programming without any inconvenience to either you or the colleague you just plugged into (and no hunting around for how to type loadkeys on the unfamiliar system...) – Toby Speight Dec 11 '22 at 16:14
  • @TobySpeight I would, however, need to carry a physical keyboard around… – gerrit Dec 11 '22 at 17:59
  • @gerrit: Yeah, but if someone wants to use a keyboard with e.g. a different amount of "clickiness" from what others prefer, that might be unavoidable and thus not a disadvantage. – supercat Jan 05 '23 at 18:15
4

In addition to other good answers here, consider the following. At the time of the first USB keyboards, it wasn't a given that Unicode would come to dominate character encodings. So we'd need to adopt or invent a way of describing the characters on each key. Not only that, but to cope with the different input methods in use, we'd need to represent group shifts, dead keys, compositions and the rest. Trying to get all the interested parties in agreement on the representation would delay the adoption by years, and likely stall the uptake of USB altogether.

Skipping this feature allowed vendors to quickly and cheaply adapt existing keyboards to the new interface, and that was essential to get any traction with users.

Toby Speight
  • 1,611
  • 14
  • 31
3

In addition to the other good answers note that there are keyboards with multiple language layouts printed on them. That would require an awkward switch on the keyboard if the keyboard decided the language - instead of the computer.

E.g., I have a Swedish keyboard - but the keys also have indications for Danish and Norwegian layout; something like https://www.corsair.com/eu/en/Categories/Products/Gaming-Keyboards/RGB-Mechanical-Gaming-Keyboards/k95-rgb-platinum-config/p/CH-9127014-ND (I guess it is cheaper than having 3 different products and such products have at least existed for a few years - even though I don't recall if they are retro yet). So, not even changing keycaps.

Hans Olsson
  • 316
  • 1
  • 4
  • 2
    Come on that's just another excuse! A key combination could simply switch the layout. No need for a switch! – zomega Dec 08 '22 at 16:32
  • @zomega: Indeed, if the layout switching were accomplished in the keyboard, there would be no need to worry about whether the host was aware of any layout which matched the printed legends. – supercat Dec 08 '22 at 17:07
  • Yes, that is the case for all newer keyboards (some vendors call it "Nordic" layout). But it is a recent development (for some definition of "recent"); it wasn't always like that. My Logitech K260 (long retired for mechanical keyboards) has it, but not Logitech "Easycall Desktop" (also long retired, serving as dust covers for other keyboards...). – Peter Mortensen Dec 08 '22 at 18:02
  • BTW, I can't tell whether your reference to "an awkward switch" is real or sarcasm, given that many keyboards in the days before installable keyboard drivers included switches to change layout options, and such switches imposed far less hassle than having to open up driver settings. – supercat Dec 08 '22 at 18:06
  • @supercat: Some newer ones have it for slight variation of location of keys (e.g., the Windows key). The Ducky I am typing on right now has its DIP switches (page 25) set different from the default (location of the "context" key (close to the right hand Ctrl key), etc.) – Peter Mortensen Dec 08 '22 at 18:08
  • BTW, if one has a computer which is locked at a password screen and one doesn't know what keyboard layout is selected, how is one supposed to get in? – supercat Dec 08 '22 at 18:10
  • this, but even better: I put Hebrew layout stickers on one laptop keyboard at some point. There was no way in the keyboard itself to switch the layout, since the keyboard wasn't even aware of the alternative layout it was being used for. – Esther Dec 09 '22 at 19:26
1

There are a small number of physical keyboard layouts, that is keys in different arrangements. When my operating system detects a keyboard that is doesn't know, it asks the user to press the key to the left of the space bar. If you do that, it will know which physical keyboard you have. There are keyboards with or without arrow keys and numerical keypad; that's no problem, if you don't have these keys it is as if you never pressed them.

When you press a physical key, a code identifying that key is sent to the computer. That code will be the same, no matter what language you are using. For example pressing Z on a German keyboard or Y on an English keyboard, both users actually press the same key and the same code is sent.

The OS will have a translation table that translates key codes into letters. It will have different translation tables for different countries, and the user will choose the country. Usually a developer can read the key code (for example when writing a game played with the keyboard, there it's the physical keys that count), and what letters are produced - that can be quite complicated, for example some letters are produced by pressing two or more keys in a row.

Sometimes there are different translation tables for the same country - for example, Mac UK keyboards and PC UK keyboards have different letters printed on the keys, and you need to select the right translation table so that the letters on the keys match what you see on the screen (or if you type blind, you can just use the translation table that you are used to and don't care what's printed on the keys).

BTW. The USB standard allows you to specify a manufacturer and a model number. That's what you should do anyway. Now you just need someone to collect the information which manufacturer and model has which keys, and pray that nobody uses the same model number for different keyboards.

AND my Mac has a thing called "keyboard viewer" which is supposed to show the keyboard. It does not show the keyboard that I am using :-(

gnasher729
  • 567
  • 2
  • 6
  • 1
    Manufacturers use the same model number for keyboards with different layouts. – Stephen Kitt Dec 09 '22 at 07:24
  • @StephenKitt Oh well. Why am I not surprised? Maybe in the case I noticed the OS knew "Manufacturer X model Y could be one of three keyboards, we ask the user to press the key left to the space bar and then we know which one it is". – gnasher729 Dec 09 '22 at 10:12
  • @gnasher729 The key left from space bar is different on Mac and PC keyboards. If you plug in a Windows keyboard to a Mac, it knows the button left of space is Alt button, and if you plug in a Mac keyboard, it knows the button left of space is not Alt. It just allows detection between a Mac and PC keyboard. – Justme Dec 09 '22 at 23:52
-1

As far as I remember, plug and play came out later, and up to then nearly all devices had to be identified manually.

Jonathan Rosenne
  • 99
  • 2
  • Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. – Community Jan 05 '23 at 16:19
  • 1
    Wrong. USB standard was released in 1996 and "Plug n Play" as a specific autoconfiguration mechanism for PCs was released before that in 1994. Before that many other autoconfiguration mechanisms have existed on other platforms years before PC platform got PnP support. Besides your answer does not explain why an USB keyboard has no way to tell the layout. – Justme Jan 05 '23 at 17:21
  • 2
    General 'plug and play' support is orthogonal to the idea that one specific device, the keyboard, may or may not provide an indication of its configuration. – dave Jan 05 '23 at 23:57