0

I have an app that (usually) will be only trying an http post one or two files at a time, but that could wind up with a backlog of files to upload (based on file size/net speed). I thought I might try a threaded queue system so that the user could continue populating the queue while, in the background, the thread queue chews through the files to send.

I attempted to butcher this answer to work, but to no avail.

Here's my code:

import threading, urllib2, Queue, os
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers

path = "path/to/files/"

urls_to_load = [os.path.join(path,x) for x in os.listdir(path)]

def read_url(file, queue):
    register_openers()
    datagen, headers = multipart_encode({"myfile": open(file, "rb")})
    request = urllib2.Request("http://localhost/appname/upload", datagen, headers)
    queue.put(urllib2.urlopen(request).read())

def fetch_parallel():
    result = Queue.Queue()
    threads = [threading.Thread(target=read_url, args = (url,result)) for url in urls_to_load]
    for t in threads:
        t.start()
    for t in threads:
        t.join()
    return result

result = Queue.Queue()
for url in urls_to_load:
    read_url(url,result)
print result

I am, however, getting a permission error:

Traceback (most recent call last):
  File "C:/Users/tmiller/Desktop/multiBro.py", line 78, in <module>
    datagen, headers = multipart_encode({"myfile": open(i, "rb")})
IOError: [Errno 13] Permission denied: 'the file'

I assume there is some access violation going on, but how might I prevent this?

The web server is essentially running a this. Thanks!

Community
  • 1
  • 1
Cryptite
  • 1,337
  • 1
  • 26
  • 49
  • You said you were using HTTP POST earlier-- are you now using GET? – Platinum Azure Apr 12 '12 at 17:18
  • Turns out I had taken the GET code out of the web.py example, so there was no return data. No GET be _gets_ the error, but now the situation has changed, I'm getting a permission denied error on the files. OP Updated to reflect – Cryptite Apr 12 '12 at 18:41

0 Answers0