2

the NIST Randomness Test, Testu01, Dieharder all seem to offer a suite of tests to check psuedo random number generators(PRNG) for randomness.
I was hoping someone with experience of those tools could tell me if they require the PRNG you want to test, or can you just pipe in a series of integers and see what happens?

Also, when they talk about bitstreams - in the case of a series of integers, would those integers have to be converted to their binary equivalent and just piped in also? If that's the case, should the binary equivalents be padded to some word length?

Is there any consensus regarding which testsuite to use for testing a series of integers for randomness? For version and release dates: TestU01-1.2.3 (18th Aug 2009) NIST SP 800-22 (11th Aug 2010) RDieHarder 0.14 (the package was published 2018-03-15)

Each option seems to have a varying set of tests, so I'm not sure which I'd try to use/why exclude one over another.

Ferdi
  • 5,179
Rob
  • 49

2 Answers2

7

The accepted answer is straight up wrong, at least about Dieharder. I don't know about TestU01 (I came across this while researching it), but I am currently using Dieharder to test a random series of integers generated by a Python program. Following is what my command line looks like:

time (python hashseq.py | dieharder -a -g 200)

hashseq.py generates a series of random integers and uses sys.stdout.write() to write them to stdout, where the command line pipe pipes them into dieharder. The "-g 200" switch is what tells it to get the random numbers from stdin. It is possible to create a GSL wrapper around your PRNG, and this is the recommended practice for large scale projects, but pipes are working perfectly well for me.

A quick Google search on TestU01 brought up the same link in the accepted answer, which suggests it is easy to use TestU01 in C. I did not find anything about using pipes with TestU01, but I also only spent a minute or so looking.

About bitstrings: At least with Dieharder, if you are piping in the values, it interprets them how it needs. If it needs a bitstream, it will interpret the data coming in as a bitstream. You don't need to do anything special. I don't know if TestU01 works the same way, but I cannot see why it would work differently.

As far as I am aware, there is no consensus on which is best. Dieharder interprets the input as unsigned integers for some tests and bitstreams for others. This works perfectly when your input is signed or unsigned integers. TestU01 can catch some issues that Dieharder won't, but according to Wikipedia, it interprets the input as floats between [0, 1), which I believe means the tests are lower resolution (a significant space of the float range is not between 0 and 1, and ignoring those reduces the resolution of a 32 bit float well below 32 bits). This also makes TestU01 more sensitive to the most significant bits and less sensitive to the least significant bits. I don't know about NIST yet, so I cannot help you there. I should add though: The author and community of Dieharder do try to add any new tests that are worth adding. Given how old the other two are, Dieharder may already have all of those tests in it.

Rybec Arethdar
  • 71
  • 1
    I am now a year into the project that prompted this answer, and I want to add something: Dieharder is by far the most comprehensive PRNG test suite. It has at least twice as many unique tests as any of the others. Even in a standard run, several of the tests are run up to 20 times or more with different parameters. And the strongest statistical test for PRNGs is Dieharder's "test to destruction" mode. (Let me warn you though, it takes weeks to run, on good hardware, compared to TestU01's Big Crush which might take a few hours on lower end hardware.) Maybe I will update this answer... – Rybec Arethdar Nov 04 '19 at 05:32
  • I cannot fathom the purpose of a "test to destruction" feature. Philosophically, that mode is simply proving that a software construct can be broken. We know that, so how does it help? – Paul Uszak Apr 05 '21 at 12:38
  • The big problem when writing cryptographically secure RNGs is that there is no way to prove mathematically that it is resistant to predictive attacks. The best you can do is statistical analysis. The more output you analyze the more evidence you have that your RNG is secure. The point of "test to destruction" mode is so that you can compare with known good PRNGs, since even the tests themselves aren't perfect. If the test fails significantly before it does on a known good RNG, then your RNG isn't good. If you last the same or longer, you at least aren't significantly worse. – Rybec Arethdar Apr 07 '21 at 06:51
1

I have not worked with the NIST test suite, but both TestU01 and DieHarder require you to integrate the PRNG into the test suite. See for example here for an example how to do this with TestU01.

A simplier possibility is offered by PractRand, which is able to read the random bits from STDIN. See here for an example usage.

As which to use: PractRand is easiest to use. BigCrush from TestU01 might be the most comprehensive set of tests.

Ralf Stubner
  • 221
  • 2
    DieHarder also allows STDIN. See "File Input" at https://linux.die.net/man/1/dieharder – DUzun Feb 27 '20 at 11:16