17

Looking at the C code from the Fast Inverse Square Root, the casting of a float to a long is done via pointer arithmetic:

i  = * ( long * ) &y;  // evil floating point bit level hacking

The hacking in question isn't of immediate interest to me. The method, using pointer arithmetic to access the memory values of one type and assign them to another is where my question lies. Later in the article they note that this behavior is now proscribed in the C standard, but which standard is not specified.

We can see from this StackOverflow answer that in C++, "dereferencing a pointer that aliases an object that is not of a compatible type or one of the other types allowed by C 2011 6.5 paragraph 7 is undefined behavior." (internal footnotes and links omitted) That's C++, but this answer (on the same issue, the strict aliasing rule) in C notes that a way around the problem of producing undefined behavior appeared with unions in C99.

From K&R C (2nd edition), page 103 we have:

“The valid pointer operations are assignments of pointers to the same time, adding or subtracting a pointer and an integer, subtracting or comparing two pointers to members of the same array, and assigning or comparing to zero. All other pointer arithmetic is illegal. It is not legal to add two pointers, or to multiple or divide or shift or mask them, or to add float or double to them, or even, except for void*, to assign a pointer of one type to a pointer of another type without a cast.”

This seems pretty explicitly to disallow the trick above. However I don't know what force this statement had in terms of code in the wild. Worth noting that Ritchie, writing in 1993, remarks "Compilers in 1977, and even well after, did not complain about usages such as assigning between integers and pointers or using objects of the wrong type to refer to structure members. Although the language definition presented in the first edition of K&R was reasonably (though not completely) coherent in its treatment of type rules, that book admitted that existing compilers didn't enforce them."

I recognize that this behavior has always been undefined. What I am trying to figure out is when and where is became specifically disallowed, meaning:

  1. When did we start modifying compilers to emit errors or warnings for this behavior? GCC will still compile the Quake FISR with arguments -O0
  2. Was this behavior identified in the c99 standard? in the c90 standard?
  3. Apart from reading the standard, how might a C programmer learn that this was behavior to be avoided and not the good kind of cleverness? K&R C strikes me as a good example, are there others?

That's three sub-questions, so I apologize for that. I'm looking for information on this pattern and so any help will be useful.

Adam Hyland
  • 609
  • 2
  • 12
  • Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on [meta], or in [chat]. Comments continuing discussion may be removed. – Chenmunka Feb 01 '23 at 09:35

2 Answers2

13

As mentioned here for instance, the aliasing rule was present in the first ANSI C standard (C89).

The second edition of K&R is based on ANSI C. The first edition doesn't seem to discuss the issue, but it also seems to contain no code that violates the ANSI C aliasing rule. The only examples of access through multiple types involve alloc/calloc and read/write, and the other type in all cases is char. (I didn't read the whole book, but I looked in the index and searched for *) among other things.)

I think it was never explicitly allowed, and I disagree with Justme's comment suggesting that it's by-the-book K&R C. But was never exactly forbidden either; it was a nonissue since sophisticated optimizing compilers didn't exist.

benrg
  • 1,957
  • 11
  • 15
  • 5
    it was a nonissue since sophisticated optimizing compilers didn't exist. Precisely. A plausible answer to the OP is "When compilers started to be really optimizing, and those hacks became tedious to handle for the compiler writers". Regarding the original hack, the guy was either clueless or too lazy to write it the right way, using a union to start with. – Leo B. Jan 31 '23 at 02:47
  • 9
    If people had known how today's compilers would interpret the "Strict Aliasing Rule", the Standard would have been soundly rejected, or at least recognized by programmers as not being a valid description of the language the Committee was chartered to describe. – supercat Jan 31 '23 at 03:20
  • Because nothing in C89 suggests that an assignment could cause the lifetime of an object to begin or end, the aliasing rules were widely interpreted prior to C99 as applying only to objects with lifetimes that were established by the rules of the language, and compilers were expected to process function calls between translation units according to a platform's calling conventions and semantics, rather than those of the C language. – supercat Jan 31 '23 at 17:58
  • @LeoB.: In this particular scenario, using a union would have made the intention of the code clearer, but unions are unsuitable for many scenarios which involve acting upon data in place. If code may receive pointers to structures of different types with different alignment requirements, accessing a structure through a union pointer would invoke UB unless the structure's address would satisfy the alignment requirement of all structure types within the union, including types that are never accessed at all. – supercat Jan 31 '23 at 18:02
  • Note that the early C ethos was to separate compilation from checking for ambiguity. If there was a plausible interpretation of the code, the compiler was supposed to use it without comment. There was a separate tool, lint, that checked code for ambiguity. – John Doty Jan 31 '23 at 18:39
  • @supercat "the Standard would have been soundly rejected" - more's the pity. Maybe they would have been forced to come up with something that wasn't quite so full of UB traps. – Karl Knechtel Jan 31 '23 at 18:59
  • @supercat In this particular scenario, the function receives a float by value in the stack. The stack is always aligned. The proper way is to assign it to the float field of a local variable of the type union, then to use the long field. An optimizing compiler is able to handle that as intended, eliminating copies. – Leo B. Jan 31 '23 at 19:03
  • 2
    @KarlKnechtel: The Standard states that UB occurs as a result of "non-portable or erroneous" constructs. The authors never imagined that anyone would take seriously the notion that such a phrase means "non-portable, and therefore erroneous". – supercat Jan 31 '23 at 19:52
  • @LeoB.: Many--probably most--situations which would involve type punning of automatic-duration objects which are used within their scope could be best handled by using unions. The situations where type punning is most advantageous, however, are those where it type-punned pointers can perform "in place" either on partial objects or on sequentially-stored groups of objects. – supercat Jan 31 '23 at 20:06
  • @LeoB. I don't think using a union for that purpose was guaranteed to work by K&R or C89 either. C99 added language saying it was okay (even calling it type punning). "the function receives a float by value in the stack. The stack is always aligned." - all of that is implementation-specific, and in fact most ABIs in common use today would pass that argument in a register and would have to spill it to memory to take the address. – benrg Feb 01 '23 at 00:17
  • 5
    @supercat Your comments suggest there is some easy way to fix this problem (just stop interpreting the standard in a stupid way), but I'm not seeing how. Dealing with platform differences in the encodings of float and long is the easy part, you just need a nightmarish thicket of #ifdefs. The hard part is (assuming sizeof(x) == sizeof(y) and proper alignment and no traps) how can the standard, or even an implementation, guarantee x = *(X *)&y is equivalent to memcpy(&x, &y, sizeof(x)) without precluding important optimizations? – benrg Feb 01 '23 at 00:34
  • @benrg: The memcpy equivalence shouldn't hold, since that would imply that a compiler assume the storage couldn't identify things of types that had no relationship to either of the types involved with the assignment. Instead, I would write rules in terms of sequencing relationships. In general, operations involving things of different types would be unsequenced, but actions which derive a pointer or lvalue of one type from a pointer or lvalue of another would behave as an "acquire" of the old type and a "release" of the new one, with the release sequenced after the acquire. – supercat Feb 01 '23 at 01:01
  • Use of a pointer or lvalue to access its own type would represent an acquire and release. Additionally, if a pointer or lvalue of one type is used to derive one of another, then--with one notable exception--each use of the derived reference would represent a release on its parent. The exception to this rule is that a compiler which performs a release on the parent type before entering a function or bona fide loop need not treat operations on the derived type as happening on the parent within that function or loop. – supercat Feb 01 '23 at 01:10
  • @benrg: Essentially, if code casts a T* to a U*, a compiler would have to treat any operations attributable to U* which occurred before the cast as sequenced before any operations using the resulting pointer, and--outside of a few tricky scenarios involving nested functions--operations with the resulting pointer which precede the next use of T in relationship to the same storage would be sequenced before that latter use. In most cases where TBAA could usefully allow operations to be consolidated, they would be unsequenced with regard to anything between them, and... – supercat Feb 01 '23 at 01:19
  • ...even in most cases that would require -fno-strict-aliasing under gcc/clang rules, the sequencing-based rules would ensure the required behavior. Additionally, most code that presently relies upon clang and gcc honoring the the "character type exception" wouldn't need it if compilers recognized the sequencing implications of character types just like any others. – supercat Feb 01 '23 at 01:24
3

When the C89 Standard was ratified, programmers and compiler writers alike recognized that it didn't partition constructs into categories that were "allowed" and "forbidden", but instead those over which it for which it would mandate support and those over which it would waive jurisdiction.

Type punning constructs were allowed in most dialects of the language the Standard was chartered to describe. The authors of the Standard, however, did not want to forbid a compiler from optimizing code like

int x;
int test(double *p)
{
    x=1;
    *p = 1.0;
    return x;
}

by consolidating the load of x with the preceding store. They recognized that there would be situations where this would be incorrect (and they used that term in the published Rationale), but there was no need to forbid compilers from performing such optimizations in cases where they wouldn't affect practical programs.

Note that the C89 Standard was never intended to imply that compiler writers should be willfully blind to situations where such optimizations would be likely to affect program behavior. Judging from the published Rationale, it would have been sufficiently mind numbingly obvious to the Committee that only an obtusely designed compiler would look at something like:

unsigned get_float_bits(float *fp)
{ return *(unsigned*)fp; }

and decide to assume that such a function could never actually access an object of type float whose address was passed to it in a pointer of type float*, that there was no need to try to forbid implementations from doing such silly things.

A key thing to understand about the Standard is that there are two categories of conformance for programs; while the Standard does not allow type punning in Strictly Conforming C Programs, it does not forbid type punning in Conforming C Programs. If the Standard had forbidden type punning in Conforming C Programs, it would have been soundly rejected.

Further, there has never been a consensus understanding as to what the type aliasing rules mean. In the absence of the aliasing rules, any region of storage which could be accessed by a program could be viewed as simultaneously holding every kind of object that could fit therein, with the value of all of those objects being encapsulated within the contents of those bits. An action which modified the value of value would change the bits to match, and changes to the bits would change the value represented by an object.

Under that view, however, almost every action that accessed a region of storage would simultaneously access the stored value of objects of every type that would fit, meaning that all non-character-type accesses to anything would violate the constraint. This could be resolved by interpreting the rule as saying:

An object which is accessed within a particular context must not have its stored value accessed in conflicting fashion within that same context by any lvalue which isn't visibly associated with an lvalue of one of the following types...

The C99 Effective Type rules were supposed to clarify things, but they instead gave the false impression that the Standard was intended to specify everything programmers should need to accomplish the things they would need to do, and that the Standard was intended to forbid actions over which it waived jurisdiction.

supercat
  • 35,993
  • 3
  • 63
  • 159
  • Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on [meta], or in [chat]. Comments continuing discussion may be removed. – Chenmunka Feb 01 '23 at 09:35