Get rows from all .txt files in directory using python

Question

I have some txt files in a directory and I need to get the last 15 lines from all of them. How could I do it using python?

I chose this code:

from os import listdir
from os.path import isfile, join

dir_path= './'
files = [ f for f in listdir(dir_path) if isfile(join(dir_path,f)) ]
out = []
for file in files:
    filedata = open(join(dir_path, file), "r").readlines()[-15:]
    out.append(filedata)
f = open(r'./fin.txt','w')
f.writelines(out)
f.close()

but I get the error "TypeError: writelines() argument must be a sequence of strings". I think it's because of Russian letters in the lines.

score 7 · Answer 1 · edited Oct 10 '12 at 14:35

7

import os
from collections import deque

for filename in os.listdir('/some/path'):
    # might want to put a check it's actually a file here...
    # (join it to a root path, or anything else....)
    # and sanity check it's text of a usable kind
    with open(filename) as fin:
        last_15 = deque(fin, 15)

deque will automatically discard the oldest entry and peak the max size to be 15, so it's an efficient way of keeping just the "last" 'n' items.

edited Oct 10 '12 at 14:35

mgilson

283,004
58
591
667

answered Oct 10 '12 at 14:21

Jon Clements

132,101
31
237
267

just a matter of style, maybe `with open(filename) as f: last15 = deque(f,15)` – mgilson Oct 10 '12 at 14:22
@mgilson Thanks for edit. That was a bit of a brain burp by me! – Jon Clements Oct 10 '12 at 14:44

defuz · Accepted Answer · 2012-10-10T14:57:21.923

1

Try this:

from os import listdir
from os.path import isfile

for filepath in listdir("/path/to/folder")
    if isfile(filepath): # if need
        last_five_lines = open(filepath).readlines()[-15:]

# or, one line:

x = [open(f).readlines()[-15:] for f in listdir("/path/to/folder") if isfile(f)]

Updated:

lastlines = []
for file in files:
    lastlines += open(join(dir_path, file), "r").readlines()[-15:]
with open('./fin.txt', 'w') as f:
    f.writelines(lastlines)

edited Oct 10 '12 at 14:57

answered Oct 10 '12 at 14:11

defuz

25,583
9
37
60

Hmm ... Just make sure none of the files are too big and this should work fine. – mgilson Oct 10 '12 at 14:13
This code is good but as I edit in question I have error with writing to file. – Andrej Oct 10 '12 at 14:50
1

Using `out += filedata` instead of `out.append(filedata)`. – defuz Oct 10 '12 at 14:53

score 0 · Answer 3 · answered Oct 10 '12 at 14:17

from os import listdir
from os.path import isfile, join

dir_path= '/usr/lib/something'
files = [ f for f in listdir(dir_path) if isfile(join(dir_path,f)) ]

for file in files:
    filedata = open(join(dir_path, file), "r").readlines()[-15:]
    #do something with the filedata

score 0 · Answer 4 · answered Oct 10 '12 at 14:31

Hope this helps:

import os

current_dir = os.getcwd()
dir_objects = os.listdir(current_dir)
dict_of_last_15 = {}
for file in dir_objects:
    file_obj = open(file, 'rb')
    content = file_obj.readlines()
    last_15_lines = content[-15:]
    dict_of_last_15[file] = last_15_lines
    print "#############: %s" % file
    print dict_of_last_15[file]
    file_to_check.close()

Get rows from all .txt files in directory using python

4 Answers4