create file of particular size in python

Question

I want to create a file of particular size (say, 1GiB). The content is not important since I will fill stuff into it.

What I am doing is:

f = open("E:\\sample", "wb")
size = 1073741824 # bytes in 1 GiB
f.write("\0" * size)

But this takes too long to finish. It spends me roughly 1 minute. What can be done to improve this?

1 min sounds about as good as it gets to me. BTW, what exactly are you doing with this file? — David Robinson, Jan 11 '12 at 08:14
1073741824 bytes != 1GB. Use an SSD instead of a mechanical HDD? Write to local disk rather than a network share? — johnsyweb, Jan 11 '12 at 08:17
@glglgl: Given how few people use gibibyte as a term, possibly due to how ridiculous it sounds (including many file system displays that summarize using base-2 GB instead of base-10 GB), getting pedantic about GiB is just that: Being pedantic. Hard drive manufacturers use base-10 to advertise larger sizes; almost no one else does. — ShadowRanger, Oct 30 '15 at 17:14
@ShadowRanger Some call it pedantic, other call it concise. Maybe it sounds ridiculous, maybe we are just not used to it yet. — glglgl, Oct 30 '15 at 20:00
It seems that your code is fine, it could not be done in a better way, IMHO. The bottleneck is your hard drive speed, with some profiling you can figure out that creating the 1 GiB string in memory is quite fast (~1 sec.) — Romain, Jun 27 '19 at 19:56

score 51 · Accepted Answer · edited Jul 26 '18 at 17:07

51

WARNING This solution gives the result that you might not expect. See UPD ...

1 Create new file.

2 seek to size-1 byte.

3 write 1 byte.

4 profit :)

f = open('newfile',"wb")
f.seek(1073741824-1)
f.write(b"\0")
f.close()
import os
os.stat("newfile").st_size

1073741824

UPD: Seek and truncate both create sparse files on my system (Linux + ReiserFS). They have size as needed but don't consume space on storage device in fact. So this can not be proper solution for fast space allocation. I have just created 100Gib file having only 25Gib free and still have 25Gib free in result.

Minor Update: Added b prefix to f.write("\0") for Py3 compatibility.

edited Jul 26 '18 at 17:07

Community

1
1

answered Jan 11 '12 at 08:17

Shamanu4

4,928
2
26
30

If your system supports it, the `truncate()` solution might - strictly seen - be better, as this solution here consumes 1 allocation block for this file although it would not be necessary. – glglgl Jan 11 '12 at 13:50
2

Doesn't that generate a sparse file, though? (This may be what was needed, but it's not quite the same) – Arafangion Jan 11 '12 at 13:51
7

In python 3 it's `f.write(b"\0")` – qed May 30 '14 at 14:32
Works on MacOS High Sierra from a file size enumeration point of view, even though it doesn't affect the overall system size. Great for writing tests for large files, which is exactly what I'm doing :) – brandonscript Feb 08 '18 at 16:49

score 40 · Answer 2 · edited May 23 '17 at 12:25

40

The question has been answered before. Not sure whether the solution is cross platform, but it works in Windows (NTFS file system) flawlessly.

with open("file.to.create", "wb") as out:
    out.truncate(1024 * 1024 * 1024)

This answer uses seek and write:

with open("file.to.create", "wb") as out:
    out.seek((1024 * 1024 * 1024) - 1)
    out.write('\0')

edited May 23 '17 at 12:25

Community

1
1

answered Jan 11 '12 at 08:18

Imran

81,919
23
95
127

7

[Docs:](http://docs.python.org/library/stdtypes.html#file-objects) _Note that if a specified size exceeds the file’s current size, the result is platform-dependent: possibilities include that the file may remain unchanged, increase to the specified size as if zero-filled, or increase to the specified size with undefined new content._ – Janne Karila Jan 11 '12 at 08:38
@JanneKarila In this case `file.to.create` is an empty file, so the specified size will always exceed the file's current size? – qed May 30 '14 at 14:34

create file of particular size in python

2 Answers2

Linked