Checking if a file exists before writing. Always avoid, or sensible with the right use case?

Question

There are few reliable absolutes in this world. One I have relied on is the idea that checking if a file exists before doing something with it just creates an unwanted race condition. Meaning between the check and the attempt at usage, the OS was free to do whatever it liked with the file. So I had no right to expect anything to stay the same and that there was no good reason to ever do this.

However, after offering this advice recently I was presented with a counter argument that such a check can preserve limited file lock resources. Thus the race condition is diffused, not by pretending we know what we don't, but by claiming to know what is more likely.

At first this just sounded to me like someone trying to rationalize some bad code but I can't say I'm as certain as I once was.

If a use case that makes sense of such code really exists I'd love to hear it. If it does I'll need a compelling argument for its need because such code is going to be very difficult to test completely.

To boil it down, I've made a habit of telling people to stop checking if a file exists before opening it. Are there exceptions to this rule profound enough to consider it bad advice?

Answer 1

When opening a file (regardless if it is opened for reading or writing), one has to do proper exception / error handling for the specific "open" statement (as well as for any further I/O). It should be pretty obvious that no file-existence test can make this obsolete for the reasons mentioned in the question. Hence, it should be clear that a prior test for file existence does not allow to simplify or omit the exception handling around the "open" in any way. In case the extra test does only duplicate some logic which is already inside the exception handler, then it only overcomplicates things and should be left out.

This is usually the case when trying to read a file. An extra "file exists" before opening makes not much sense - either the file can be opened for reading, or it cannot, which can be caught as part of the exception handler.

For writing into a file, however, I can imagine some different scenarios:

1. Appending to a log file (where it does not really matter if the file exists beforehand - if not, it can just be created). The correct approach in Java, for example, is described here, it includes a call to createNewFile, which does the test for existence implicitly. But beware, in other programming languages the "append if not exists" might require an explicit test for existence.

2. Writing or copying into a folder (and checking if that folder exists before writing into it). Here, a prior test if the folder exists can make sense to produce a precise error message (for example, for an UI, telling users they need to create the folder first). When write or copy fails afterwards, maybe because some process in the background renamed the new folder (which will probably a very rare case), this may result in a more technical failure message where the root cause may not be that clear, so the prior existence test can help to reduce the frequency of running into this case.

3. Obtaining a write-lock to a lock file / using this as a semaphore. Here, a file-existence test would be completely nonsensical, since getting the write-lock must be an atomic operation, otherwise it would not work.

4. Writing into a file with fixed block sizes at specific position like a file-based database (so the file must exist beforehand to make this a sensible operation). Since this usually involves to get exclusive write access to the file in stake, it is not really different from the former case - a file-existence test does not make much sense. In fact, in this scenario the file in stake will probably exists over a long term. The relevant information is not if it exists, but if it is writable, and obtaining the write-lock must be atomical.

5. Overwriting an existing file (and prevent this to happen accidentally). In a UI application, asking the user beforehand if they intentionally want to overwrite an existing file can make a lot of sense. Even if the file in stake is just magically appearing after the file-existence test returned "false", the prior test can reduce the number of accidental collisions.

So I think this is not just "black-and-white" - there are scenarios where a prior existence test can make it easier to handle certain situations more gracefully, or lower the probability of collisions. One should always be aware, however, that this test is no replacement for the proper handling of failing I/O operations.

Answer 2

You allude to the answer to your problem in your first statement, that is mostly correct. In the case of software engineering, there are no reliable absolutes.

The bad advice is dogmatically telling someone to apply a method blindly. The advice is solid as long as it is given as guidance, not gospel. "It is usually best not to check before write" is solid advice. "Never check before write" is the kind of poor advice that make sites like this so valuable.

If you want to, you can then go on to justify the reasons, and describe the exceptions. Keep in mind, like most dogmatic rules, you can rework every exception to fit the rule and make yet another convoluted mess of code to fit some arbitrary programming dogma that makes no sense in that particular use case.

Answer 3

The use case I can see for this is:

• check if file exists
• if it does, prompt user
• perform time-consuming operation
• write to file

If you know that the existence (or absence!) of a file will cause problems later, it makes sense to check for it upfront before doing anything expensive. However you still also need to handle the case where the file appears between checking for it and actually opening it.