Python strings and memory optimization

Question

This code will load 10000 instances of strings (with same content):

lst = []
for i in xrange(0, 10000):
    with open ('data.txt', 'r') as datafile:
        lst.append(str(datafile.read()))
print(lst)

Only by adding code after or before the above one, i wan't to have the same result as this.

lst = []
with open ('data.txt', 'r') as datafile:
    s = str(datafile.read())
    for i in xrange(0, 10000):
        lst.append(s)
print lst

This will load only 1 instance of string. => Less memory usage.

In java, there is String.intern() ? I look for a python equivalent.

Answer 1

You could use intern() to do the same.

Enter string in the table of “interned” strings and return the interned string – which is string itself or a copy. Interning strings is useful to gain a little performance on dictionary lookup – if the keys in a dictionary are interned, and the lookup key is interned, the key comparisons (after hashing) can be done by a pointer compare instead of a string compare. Normally, the names used in Python programs are automatically interned, and the dictionaries used to hold module, class or instance attributes have interned keys.

Store the return value:

lst = []
for i in xrange(0, 10000):
    with open ('data.txt', 'r') as datafile:
        lst.append(intern(datafile.read()))
print(lst)

Answer 2

In java, there is String.intern() ? I look for a python equivalent.

It is called intern() in Python, and it's a built-in function.

For further discussion, see Python string interning