27

I can find information about how characters were used to structure transmissions in general, and I can find information about transmission characters which are still commonly used today, though perhaps not for their originally intended purpose. But I am unable to find anything more on the "end of medium" character (EM, 0x19, decimal 25) than that name among listings of all the ASCII characters.

What was this character intended to be used for (AKA what sense of the word "medium" was meant)? Was it ever actually widely used for that purpose, or any other?

Simon Kissane
  • 2,031
  • 7
  • 21
Ryan1729
  • 525
  • 4
  • 7
  • 4
    Tape. (and some filler characters to make a comment) – tofro Jun 30 '19 at 10:40
  • 3
    In 1968, punched paper tape, 7 bits wide, and NOT of infinite length, was used with Model ASR-35 Teletypes, DEC PDP-8s, Honeywell DDP-516,. Many of ASCII's "Control Characters" come from the Teletype universe. – waltinator Jun 30 '19 at 16:56

3 Answers3

22

Wikipedia

“Intended as means of indicating on paper or magnetic tapes that the end of the usable portion of the tape had been reached.”, if en.wikipedia.org/wiki/C0_and_C1_control_codes#EM is to be believed.

ECMA-6

ASCII was codified internationally as ISO/IEC 646 which in turn was ratified by ECMA as ECMA-6. Section 8.12 EM END OF MEDIUM of a PDF image document of the 5th Edition (March 1985) reads:

A control character that may be used to identify the physical end of a medium, or the end of the used portion of a medium, or the end of the wanted portion of data recorded on a medium. The position of this character does not necessarily correspond to the physical end of the medium.

So EM wasn't the same as the reflective physical end of tape marker (bane of flash photographers), but rather more likely to be the equivalent of the soft end-of-tape markers used in cpio/tar/pax archives. This allowed data sets larger than available media to be stored/transmitted in multi-volume sets.

Data formats and file systems used to be far more record/block oriented than they are today and these control characters would have been developed to support record formats in transmission. I'd love to learn of any actual use of these ASCII features in the wild, as I've yet to see one.

scruss
  • 21,585
  • 1
  • 45
  • 113
  • FYI, this source mentions a note about ASCII 25 EM: “(VMS: stronger user interrupt)” which I assume refers to the VMS operating system of a DEC VAX. – Basil Bourque Jun 30 '19 at 00:14
  • I’d still be interested in information about any usages of this character, or some kind of evidence that it was never widely used for its intended purpose. But if it was never actually widely used, I suppose there wouldn’t be an easy way to determine that! – Ryan1729 Jun 30 '19 at 00:19
  • 2
    On reel-to-reel magnetic tape drives, there were typically two separate data block formats, signifying "end of file" and "end of tape". One tape reel might contain several files, or one file might occupy more than one tape. The basic function of the "end of tape" block was to stop the tape drive over-winding the tape. I wonder if the ASCII EM code was intended as a device-independent coding this end-of-tape condition. Confusingly, the ASCII EOT code (end of transmission) effectively meant "end of *file*", not "end of tape", at least in Unix. – alephzero Jun 30 '19 at 00:38
  • The EOT sentinel on magtapes was generally not a recorded mark, but reflective tape. However, there was certainly a convention to distinguish end of file from end of volume; the latter was (if memory serves) two consecutive tape marks. – dave Jun 30 '19 at 01:34
  • 2
    @another-dave Your article seems contradictory- it talks about physical markers but then says the were able to write an EOT marker. Back in the day, our mainframe tape librarians had a device looking a bit like a magnifying glass filled with fluid containing magnetic particles, which could be used to "see" the magnetic data on the tape. I have memories of trying to figure out how to read tapes in unknown formats and this being used to recognise EOD and EOT blocks, (as well as identify the basic parameters like number of data tracks and recording density) but I might be wrong. – alephzero Jun 30 '19 at 08:35
  • 1
    There's some confusion in terminology in the commentry here. To old tape-slingers EOT means end of tape, a physical reflective marker embedded near the end of a tape. To ASCII/ECMA-6, EOT is end of transmission (‘A transmission control character used to indicate the conclusion of the transmission of one or more texts.’) – scruss Jun 30 '19 at 12:07
  • @alephzero your memory serves you well. The silver stickers were uses as physical end-of-tape (that includes begin) markers, while data structure was written as blocks, which includes an end-of-tape block, serving to mark the logical end of a tape - the last valid written data, no matter how much was still present thereafter. Which all together is different from the EOT code, which is part of protocol framing - although with a quite similar meaning. – Raffzahn Jun 30 '19 at 12:09
  • Scruss, I've seen lot of transmissions - and files - structured using the various marks. Mostly in or around Mainfame environments, includign lots of minis of the 70s and 80s, From Dietz to Bull and ICL (sizewise). – Raffzahn Jun 30 '19 at 12:27
  • Ah - the contradictory part; I see it now. The reference that I scooped up at random was from a data-recovery service. They seem to be using modified equipment so I'm not sure about the general applicability. Either "writing an EOT" is an error compared to their previous definition of EOT, or else their modified hardware recognizes some other mark. On reflection (sic), a poor choice of reference-page by me, sorry. – dave Jun 30 '19 at 12:53
  • In my own experience, the terms BOT and EOT referred to the points indicated by the reflective parts of the tape, i.e., physical markers. End of file was indicated by a tape mark; end of volume by two consecutive tape marks. My original point was in response to alephzero's comment about something written on the tape signifying end of tape; now it just looks like a terminology mismatch, logical end of tape versus physical end of tape. The former, I call "end of volume". – dave Jun 30 '19 at 13:10
  • 1
    @another-dave There are in fact are three different kinds of markings and their names collide, so it's all about the context. The lowest level are two little Silver Stickers, to be detected by a dedicated sensor, marking the physical BOT and EOT. Then there is the file mark a specific pattern to mark the end of a file. If written twice it forms an EOT marking the logical end of tape. Above all of this, there are Standard Labels which are basically tiny files. There are "headers" and "trailer" types carrying meta data (like file names) - one of them is (again) called EOT. Pick any:) – Raffzahn Jun 30 '19 at 16:28
  • @Raffzahn: Would it be common to cut and splice tapes (using silver stickers to find cut/splice points)? Would a computer facility have tape-winding equipment for that purpose that technicians could use without interfering with what computers were doing? If it was common to cut and splice tapes into the order needed for various jobs, that might be an interesting adjunct to the question about tapes as a random-access medium. – supercat Jan 14 '22 at 17:44
  • @supercat No. These stickers mark the end of a tape, there is nothing behind, especially no other tape. Also, noone splices tapes. they only get cut or thrown away. It's not worth the hassle. – Raffzahn Jan 14 '22 at 19:16
  • @Raffzahn: Sorry--I'd misread the above as saying one sticker was end-of-file and two was end-of-tape. As for splicing, I would think that for some kinds of batch jobs that need different combinations of files, splicing a composite tape that contains files in the order required could use computer resources more efficiently than having to use a computer to copy data from each source tape onto a common output tape, especially since the splicing could be done off-line. – supercat Jan 14 '22 at 19:27
  • @supercat Tapes can (and will) be rewind to search for files. It makes simply no sense to damage expensive tape drives to save a few cent. Also, copying files between tapes to create new tapes is a very basic job done all the time. It's a job a machine can do on itself. no manual work, beside mounting the tapes, needed. It's many times faster than any manual cut and splice. – Raffzahn Jan 14 '22 at 19:34
  • @Raffzahn: By my understanding, half-inch tape drives would almost universally include write, read, and erase heads in that order, so that if a block wasn't written correctly, it could be erased and rewritten. Were tapes not tested before use, or what would cause a tape that wasn't spliced to have bad spots that couldn't be written? – supercat Jan 14 '22 at 20:07
  • @Raffzahn: What would be a good resource to get a sense of the economies involved among cost of computing power, tape drives, printers, card punches, card scanners, etc. along with the media for them? I can imagine some data centers might have dozens of tape drives that could all be used simultaneously, but some places might want to get by with the minimum equipment necessary to do what needed to be accomplished. Did any places try to get by with only three or four tape drives, or was there an "in for a penny, in for a pound" philosophy? – supercat Jan 14 '22 at 20:24
  • @supercat We're talking professional use, not hobbyist playground. An hour of operator time is way more expensive than a new tape - which can be used anyway. Sure, there may have always been business that wasted a pound of their employees time to save a penny - but as soon as they had to call in service for damaged heads even thy would start to learn. – Raffzahn Jan 14 '22 at 21:22
  • @Raffzahn: An hour of operator time may have been worth more than a blank tape, but an ability to concatenate tapes, off-line, onto a single reel could reduce the number of tape drives that would be needed for a computer to efficiently process various combinations of jobs. Maybe I'm imagining tape drives as more expensive than they actually were, which is why I was curious about the relative costs of things. – supercat Jan 14 '22 at 21:55
  • @supercat Tape drives were expensive, but there are not many jobs that need more than a few. If at all. Also, mounting a tape is a matter of less than a minute. Using a single reel saves not much, but introduces extreme risk of not only splitting the tape, but as well damaging the drives. these are not your modern slow moving single direction tapes. – Raffzahn Jan 14 '22 at 22:04
  • 1
    Anent VMS' use of EM, more generally known as ^Y. This has nothing to do with 031 being EM, it was just a control code that had no prior known use in DEC terminal drivers. ^C was an interrupt that was often intercepted by user mode; ^Y was an interrupt generally intended to get you back into supervisor mode -- where the CLI ran -- if user mode had completely locked up. That need is a consequence of the VMS process design, in which a process is reused for successive user-mode programs. – dave Nov 14 '22 at 22:40
21

TL;DR;

EM (or EOM as in early documents - and Unicode as well) was and is widely used in data transmission to mark

  • the physical end of a medium,
  • the end of the used portion of a medium, or
  • the end of meaningful (wanted) data on a medium.

(as Scruss already described in his answer)

It was used with most RJE stations as well as with terminals. The confusion/missing information may be due to its quite low visibility to average (terminal device) programmers, as it's used at the core of many data protocols - especially in finance. Each and every credit card transaction will go over various communication links encoded with EM in certain fields.

The Long Form:

But I am unable to find anything more on the "end of medium" character (EM, 0x19, decimal 25) than that name among listings of all the ASCII characters.

That may be because its most prominent usage was within the (IBM forged) Mainframe world. It denotes the end of a variable-length (or shortened) input or output. On some environments it could be used to shorten transmissions to transmit only the characters entered within this (logical) line, not the whole line or the whole screen (keep in mind, mainframe terminals operate in block mode), saving quite some transmission time.

Unbeknown to most, all remote (terminal) I/O in mainframe systems is (usually) ASCII-based (*1). To avoid possible code issues, EM belongs to the group of characters with the same encoding in ASCII and EBCDIC.

The usage developed over time. Generations of terminals, including the 2260, 2741 and 1050 series, had implied handling (by message framing) of EOM as it was called back then (and still is in Unicode). For example, for explicit display the 2260 used ASCII X'21' or !. Later it became available as a user-handled function.

EOM was officially integrated into ASCII-1967 (USAS X3.4-1967) as EM, mostly to satisfy the needs of interoperable RJE handling. The IBM 2280 Data Transmission Terminal manual (GA27-3005-3 of 1971) describes the EM function in context with input/output media nicely for handling of punch cards:

EM (End of Media)
The EM control character is used to indicate the end
of a record when transmitting records of variable
length. The EM character is punched in the card
column following the last column containing data.
When the EM character is read from a card it prevents
further reading of that card. The transmitting
2780 then generates automatically and transmits an
end-of-record (US or IUS in EBCDIC), or end-of-block
(ETB) character following the EM character.

Handling - or more likely, generating - such input might have been the most common usage for ASCII-based environments when hooking up to the IBM world.

What was this character intended to be used for

Intention implies that character sets were created to form an intended use. In reality it's usually the other way around: there are many usages in many different environments; to form a standard, committees try to squeeze in as many different functions, while finding overlapping usage to reduce code points needed.

(AKA what sense of the word "medium" was meant)?

In its most pure sense of any medium: no matter whether a line on a screen, a (part of a) punch card, a paper tape or magnetic tape or some data stream.

Was it ever actually widely used for that purpose, or any other?

Yes. With any credit card transaction you send several data blocks, and EM will be used in some fields. Mainframe will never die :))

*1 - RJE stations like the 2280 can often operate in EBCDIC or ASCII.

Toby Speight
  • 1,611
  • 14
  • 31
Raffzahn
  • 222,541
  • 22
  • 631
  • 918
8

The EM marker goes back way before computers. As early as the 1850s, the telegraph system was making more efficient use of the wired network assets by offline punching of paper-tape, and store-and-forward methods. All kinds of sending and receiving methods (including automatic typewriters) were in use.

The EM (end of medium) code was hard-wired into the machines to prevent them running off the end of the reel when reading: without that, you would get a couple of random characters from the ragged end of the tape, and then a succession of DEL characters (all holes punched).

When punching tape, the inside end of the blank reel was teased out and an EM punched into it by hand. The automated punch then stopped automatically so you could load up a fresh reel. I was doing that on mainframes as late as 1970. You could also get a serious static shock from paper-tape unless the dump bin was properly earthed: a fast paper tape reader is a very effective Van de Graaff generator.

Nobody ever spliced mag-tape that I heard of. But paper tape frequently wrecked, and was repaired by hand-splicing, using an alignment gate, razor blade, sticky patches, and a hand dibber.

If there was a wrong character, you could erase it by dibbing all the holes: that is why the DEL character is all bits set (octal 177 plus parity). Correspondingly, blank tape (used to separate records) is NUL (all zeros, aka run-out). CR/LF is two separate mechanisms for printing -- return platen to column 1, then pull a gear tooth to take the paper roller up one row.

Good book to find is "The Victorian Internet" by Tom Standage.

Paul_Pedant
  • 461
  • 3
  • 4