Is it good to create random tests?

Question

For example, such a test:

[Test]
public void Test_With_Many_Random_Words()
{
  var randomWords = new string[100];
  for (var i = 0; i < 100; ++i)
    randomWords[i] = GenerateRandomWord();

  var trie = new PrefixTrie();
  for (var i = 0; i < 100; ++i)
    trie.AddWord(randomWords[i]);

  Assert.AreEqual(100, trie.CountOfWords);
  foreach (var word in randomWords)
    Assert.IsTrue(trie.ContainsWord(word));
}

private string GenerateRandomWord()
{
  var random = new Random();
  var builder = new StringBuilder();

  var length = 1 + (random.Next()%100);
  for (var i = 0; i < length; ++i)
  {
    var charCode = random.Next()%26;
    builder.Append((char) ('a' + charCode));
  }
  return builder.ToString();
}

Or it will be better to create tests with few known words like "testword1", "alsotestword2", "notword3" and check code for correctness with them?

Actually, it doesn't matter for my question, but for this time, we can assume that we are testing some data structure (for example, in my code I'm testing prefix trie). — Andrew, Dec 08 '15 at 12:06
You shouldn't rely on random test. Most of the time it will miss special cases. But you can add them, its better then no test — CoderPi, Dec 08 '15 at 12:10
One of many best practices for testing is to have known inputs that should yield expected outputs. With the key emphasis being on known inputs. If your inputs are random, are they known? That kind of seems counter intuitive, does it not? With your specific example I could see how this would make sense, but then again it would test the desired functionality with known inputs just the same. I believe that this question is actually a little too vague for this forum. — David Pine, Dec 08 '15 at 12:14
The answers to that question (the duplicate) are quite old (but still relevant). Since then, the approach got it's own name - property-based testing. You can look for more information about it. — Jakub Lortz, Dec 08 '15 at 12:16

score 2 · Accepted Answer · answered Dec 08 '15 at 12:12

2

You can work with generated test data, but I would discourage you from using random live-generated input for one reason: repeatability.

If there is a problem in your code, you might have a test that suddently go red because you accidentally found a "wrong" parameter combination... or not.

Such a test is highly unreliable.

answered Dec 08 '15 at 12:12

Timothée Bourguignon

2,190
3
23
39

1

this can be mitigated by using a fixed seed for the random number generation, which will result in a random, but repeatable sequence of number generated – Sam Holder Dec 08 '15 at 14:21
1

True but creating fake-random data is a bit overkill, isn't it? :P – Timothée Bourguignon Dec 08 '15 at 14:27
1

There are cases where the test data doesn't matter and randomly creating can indicate that the chosen values are not significant. [AutoFixture](https://github.com/AutoFixture) does this. – Sam Holder Dec 08 '15 at 14:41

Is it good to create random tests?

1 Answers1