3

I've recently written a Wiki article on the Creative Computing benchmark. I was wondering if anyone has examples of it being used outside Creative Computing?

I've found one example in Compute!, but I'd like to add more for balance. Ideally these would be examples of people using it as a contemporary test, but modern examples would add flavour too.

Examples of the Byte Sieve would also be appreciated.

UPDATE: I see I missed some key context: I'm looking for examples of people using the benchmark today to compare retrocomputers. I see many examples of this in blogs and forums, but I don't get around enough the print world outside the Atari space to find examples there.

Maury Markowitz
  • 19,803
  • 1
  • 47
  • 138
  • Well, maybe not the Creative Computing Benchmarks, but the "Ultimate Benchmark" is still alive and done across many machines. The idea is similar, but it uses FORTH as language, as FORTH is quite portable and there are many good optimizations available for common machines, eliminating the troubles of bad BASIC implementations, while staying portable. – Raffzahn Mar 18 '19 at 15:41
  • The Wiki link quotes a run time on the Cray-1, but AFAIK the Cray never had a Basic compiler or interpreter (not one provided by Cray Research, anyway) - so was it actually a Fortran translation of the benchmark, or something similar? – alephzero Mar 18 '19 at 16:25
  • 1
    @alephzero-interesting question, none of the listings mentions it one way or the other. – Maury Markowitz Mar 18 '19 at 16:29
  • @Raffzahn interesting that the ultimate benchmark reproduces the Galbreaths' errors and inefficiencies in the Sieve section, Also, do Forth compilers exist that take advantage of modern processor and memory features? – scruss Mar 18 '19 at 19:29
  • 2
    Another benchmark that came BEFORE Ahl's Benchmark (Creative Computing, Nov 1983, pg 260), but quite unknown, is the Broucke Benchmark which was designed to find flaws in floating point arithmetic. It was published on page 16 of the Oct/Nov 1982 issue of ANTIC magazine. Of the 75 languages I have tested, none get the S result 100% correct. The closest is MSX BASIC, Tandy 100 BASIC and TI-99/4A BASIC. The code follows:

    50 S=0 100 X=0 200 FOR N=1 TO 1000 300 S=S+X*X 400 X=X+0.00123 500 NEXT N 600 PRINT “RESULTS: S=”;S;”, X=”;X 700 PRINT “CORRECT: S=503.543802149, X=1.23”

     – Tim Locke Mar 18 '19 at 19:51
  • @scruss I belive it wasn't so much about having the best solution for the problem, as havin one benchmark that can be used on many machines to deliver comparable results. There is still development in FORTH, especially with regards of embedded and DSP processors, so it may depend on what you consider modern and feature. Maybe drop Carsten (see the web page) a mail, he's deep enbedded into the FORTH scene - and a nice guy to talk with anyway. – Raffzahn Mar 18 '19 at 20:44
  • 1
    @TimLocke the correct value for s is not 503.543802149, but 503.54380215 (= 10070876043 / 20000000). The somewhat obscure Microsoft BASIC v2.00 [Decimal Math] for Macintosh interpreter (1984) gets it right, but it uses decimal arithmetic so avoids the issue entirely. – scruss Mar 18 '19 at 20:55
  • @scruss My answer was from the original article but your answer means the BASICs I mentioned above were the only ones to get it right. – Tim Locke Mar 18 '19 at 22:14
  • 1
    @TimLocke I just found out that MSX Basic used BCD maths, which is why it gets the right answer. Most systems that use binary floating point will get that last digit of s wrong. – scruss Mar 18 '19 at 23:44
  • @scruss From what I've found, it uses 64-bit math with a 56-bit mantissa and an 8-bit exponent, so that should give up to 14 decimal digits of precision.

    I also just noticed that Kyan Pascal for the C64 gets the right answer and I had a note stating it uses 64-bit BCD FP (56M, 8E) so this lines up with your point that MSX BASIC uses BCD FP.

    Can you confirm whether the Tandy Model 100 and TI-99/4A use BCD? I know they use 64-bit FP. It seems very likely.

     – Tim Locke Mar 19 '19 at 15:07
  • 1
    @TimLocke - yes, they both do. In their respective manuals they give limits for floating point that are clearly decimal with binary exponents: the model 100 manual gives a single precision (4 byte) range of ±1×10⁶² and ±1×10¯⁶⁴ – scruss Mar 20 '19 at 01:02
  • @MauryMarkowitz - I'm still confused about what you're asking. If you're comparing actual retro hardware, the numbers for Ahl's benchmark will be the same as published, so why bother going to the effort of typing in this trivial benchmark? If you're emulating retrocomputers, then most emulators have much more useful stats such as frame rate and effective CPU MHz. I'm not sure there's an immediately useful answer out there beyond the front page of Google. – scruss Mar 23 '19 at 14:32
  • @scruss - "actual retro hardware, the numbers for Ahl's benchmark will be the same as published" - no! many new BASICs have been written in the decades since and they all use it. The Atari forums are filled with such examples. The Compute! example is similar. – Maury Markowitz Mar 24 '19 at 19:47
  • @MauryMarkowitz - I was meaning on the same stock BASIC interpreter on the same hardware the results would be independent of when the test was done. New BASIC interpreters are mostly (IME) an Atari thing because the supplied BASIC was so slow. It hasn't been as much of a priority for other systems. And even then, the Ahl benchmark isn't the most useful benchmark - where does the "and they all use it" come from? – scruss Mar 24 '19 at 22:58
  • 1
    @TimLocke - I seem to have missed your comment, I will be following up on this lead as well. – Maury Markowitz May 28 '21 at 13:00

1 Answers1

6

While it's impossible to give a definitive negative answer, I'd say that it's unlikely that other publications depended on Ahl's benchmark, because:

  1. It tests just the looping performance and accuracy of interpreted floating-point BASIC. Rugg & Feldman's BASIC Timing Comparisons (updated edition, Kilobaud Microcomputer magazine, October 1977) that later morphed into the PCW Benchmarks, tested many more aspects of BASIC performance and were a more useful benchmark suite.

  2. Later single tests such as Byte's Sieve (1981) showed — as far as a benchmark that may be susceptible to rigged results can — that different compilers and interpreters amongst many different platforms had wildly varying performance results. As built-in BASIC performance became less of a sales draw, simple interpreter tests faded away.

  3. Consequently, ports of Ahl's benchmark to other languages are more down to the skill of the porter and the compiler's optimizer than the intrinsic speed of the target. The presence/absence of additional whitespace characters could affect the performance of some BASIC interpreters, so results from Ahl's benchmark even in its original language could also depend on the typist.

  4. As founder and editor-in-chief of Creative Computing, David Ahl would naturally promote his own benchmark in his own publication. Other publishers might not have felt so charitable towards something “not invented here”.

It might be interesting to hear Roy Longbottom's opinion (he's a SE Member!) on these kinds of benchmarks, as he's been involved in system benchmarking since Algol 60.

scruss
  • 21,585
  • 1
  • 45
  • 113
  • While not an answer, I think the reasoning used is quite compelling. – Raffzahn Mar 18 '19 at 23:28
  • 1
    It's an answer: I'm saying no, or at least, very unlikely. Maybe I was too subtle? – scruss Mar 18 '19 at 23:41
  • Being subtle and make upt a good reasoning is eventually the better way - there have been (and still are) way to many publication on this planet to be sure. – Raffzahn Mar 19 '19 at 07:55
  • Heads up: on iOS at least, that geocities.ws link heads directly to a malware site triggering fake dialogues that look superficially like the system dialogues to try to trick me into installing their VPN product. Is it definitely not pulling any of that nonsense on other platforms? – Tommy Mar 19 '19 at 11:44
  • @Tommy It leads to what looks like a normal page for me on Safari for macOS and on my iPhone. – JeremyP Mar 19 '19 at 12:23
  • 1
    @JeremyP as it now does for me; no doubt a laissez-faire banner advertising provider was involved in my experience earlier. – Tommy Mar 19 '19 at 12:46
  • 1
    @Tommy - looked fine to me, but I adblock everywhere. Apologies if the link was dubious. I've replaced it with something less dodgy (but missing some of the commentary). – scruss Mar 19 '19 at 13:02
  • @Tommy It is geocities. Even though it looks fine, it may not be. – JeremyP Mar 20 '19 at 09:18
  • @JeremyP - I have replaced the link. It is no longer to anything on geocities – scruss Mar 20 '19 at 14:19