0

Using sqlite3 in Python, it seems to me that there is a maximum on the number of connection that can be established to different databases at the same time.

For example, I can connect to 1000 different databases:

cons = [sqlite3.connect(infile) for infile in infiles[:1000]]

But I cannot open 

cons = [sqlite3.connect(infile) for infile in infiles[:1024]]

which gives sqlite3.OperatinError: unable to open database file on the databases that are above some threshold that seems to be around 1024.

Is there a way to increase this maximum to around, say 4096?

Radio Controlled
  • 792
  • 6
  • 21
  • https://stackoverflow.com/questions/9017762/what-is-the-maximum-connections-for-sqlite3-database – PV8 Aug 14 '19 at 08:01
  • 1
    @PV8 I don't think those are relevant; they're about connections into a single database, where the question here is about 1000 different databases. – AKX Aug 14 '19 at 08:02

1 Answers1

2

You're probably bumping into the maximum number of open files per process there.

If you're using Linux, see e.g. How do I change the number of open files limit in Linux?

AKX
  • 123,782
  • 12
  • 99
  • 138
  • Oh, great answer! I think I'll try a quick workaround if this becomes an OS issue. Otherwise I'll try to increase the max - although I guess there is a reason linux has this threshold... – Radio Controlled Aug 14 '19 at 08:03