rtorrent 0.8.6 on Ubuntu 10.04.02 32bit - Too many open files error [RESOLVED]

shwetha17

Member
May 24, 2018
785
0
16
This error baffles me. ulimit shows 4096 max open files, rtorrent shows 4096, yet even if only ~700 files are open, I get this annoying error in rtorrent:



My ulimit:

Code:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 20
file size (blocks, -f) unlimited
pending signals (-i) 16382
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 4096
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) unlimited
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

And lsof -u <username> | wc -l:
Code:
670
Wonslung, any ideas?
 

jith45

Member
May 25, 2018
960
0
16
show your .rtorrent.rc

Also, are you using something like openvz or some other container/virtualization software?


specifically, what is your "max_open_files" set to in .rtorrent.rc


also, any socket settings
 

shwetha17

Member
May 24, 2018
785
0
16
Hi mate, thanks for your reply!

I'm using no virtualization, it's running on Ubuntu 10.04.2 LTS bare metal.

This is the rtorrent.rc:

Code:
# Maximum and minimum number of peers to connect to per torrent.
min_peers = 40
max_peers = 400

# Same as above but for seeding completed torrents (-1 = same as downloading)
min_peers_seed = 20
max_peers_seed = 200

# Maximum number of simultaneous uploads per torrent.
max_uploads = 0

# Max upload and download slots allowed
max_uploads_global = 288
max_downloads_global = 0

send_buffer_size = 4M
receive_buffer_size = 2M
max_memory_usage = 1536M

# Global upload and download rate. "0" for unlimited.
download_rate = 0
upload_rate = 0

# Default directory to save the downloaded torrents.
directory = /home/<user>/torrents

# Default session directory.
# Make sure you don't run multiple instance of rtorrent using the same session directory.
session = /home/<user>/.session

# Watch a directory for new torrents, and stop those that have been deleted.
schedule = watch_directory,5,60,load_start=/home/<user>/watch/*.torrent
#schedule = untied_directory,5,60,stop_untied=~/watch/*.torrent

# Close torrents when diskspace is low.
schedule = low_diskspace,5,60,close_low_diskspace=1000M

# Stop torrents when
# reaching upload ratio in percent,
# or reaching total upload in Mbytes,
# or reaching final upload ratio in percent.# example: stop at ratio 10.0 with at least 200M uploaded, or else ratio 20.0

ratio.enable =
ratio.min.set = 1000
ratio.max.set = 2000
ratio.upload.set = 200M

# The ip address reported to the tracker.
ip = <xxx>

# The ip address the listening socket and outgoing connections is bound to.
#bind = 127.0.0.1

# Port range to use for listening.
port_range = 62910-62910

scgi_port = 127.0.0.1:5001

# Start opening ports at a random position within the port range.
port_random = yes

# Check hash for finished torrents. Might be useful until the bug is
# fixed that causes lack of diskspace not to be properly reported.
check_hash = no

# Set whetever the client should try to connect to UDP trackers.
use_udp_trackers = no

# Alternative calls to bind and ip that should handle dynamic ip's.
# schedule = ip_tick,0,1800,ip=rakshasa
# schedule = bind_tick,0,1800,bind=rakshasa

# Encryption options, set to none (default) or any combination of the following:
# allow_incoming, try_outgoing, require, require_RC4, enable_retry, prefer_plaintext
#
# The example value allows incoming encrypted connections, starts unencrypted
# outgoing connections but retries with encryption if they fail, preferring
# plaintext to RC4 encryption after the encrypted handshake
#
encryption = allow_incoming,enable_retry,prefer_plaintext
# encryption = allow_incoming,try_outgoing

#
# Do not modify the following parameters unless you know what you're doing.
#

# Hash read-ahead controls how many MB to request the kernel to read
# ahead. If the value is too low the disk may not be fully utilized,
# while if too high the kernel might not be able to keep the read
# pages in memory thus end up trashing.
hash_read_ahead = 10

# Interval between attempts to check the hash, in milliseconds.
hash_interval = 10

# Number of attempts to check the hash while using the mincore status,
# before forcing. Overworked systems might need lower values to get a
# decent hash checking rate.
hash_max_tries = 5


# Max number of files to keep open simultaneously.
max_open_files = 4096

# Number of sockets to simultaneously keep open.
#max_open_sockets = <no default>


# Example of scheduling commands: Switch between two ip's every 5
# seconds.
#schedule = "ip_tick1,5,10,ip=torretta"
#schedule = "ip_tick2,10,10,ip=lampedusa"

# Remove a scheduled event.
#schedule_remove = "ip_tick1"

dht = disable
peer_exchange = no

As you can see, max_open_files is set to 4096, *max_open_sockets* is not set, using the default one.

Just had the error happen again, rtorrent show only 173 files open, was adding a massive 100GB torrent with 700 files. *lsof* shows 590 files open.
 

das329717

Member
May 25, 2018
928
0
16
Quote
This error baffles me. ulimit shows 4096 max open files​


This is a per-process limitation. But per-system limitation is also exist.
Say
cat /proc/sys/fs/file-max
and
lsof | wc -l
 

shwetha17

Member
May 24, 2018
785
0
16
That is so high mate that the system would become unusable long before that limit was reached.

Still, its a fair point, so this is what I have:

cat /proc/sys/fs/file-max 1238504
lsof | wc -l 5334
 

shwetha17

Member
May 24, 2018
785
0
16
But I need about ~2,000 open files for rtorrent alone, wouldn't the 128 in rtorrent.rc be too restrictive?

In the past, I've set this limit to 65536 both in limits.conf & rtorrent.rc, it worked fine. All packages are the same now, only rutorrent was upgraded from v3.1 to v3.2, this error just won't go away!

I think I'm going to create a new user and set its open files limit to 4096, then set up rtorrent and see if it makes a difference. Any other suggestions are very welcome!
 

shwetha17

Member
May 24, 2018
785
0
16
OK, so it seems that I had to add /etc/pam.d/common-session the following line:

Code:
session required pam_limits.so
I'll come back to edit the title if this will fix the problem.
 

jith45

Member
May 25, 2018
960
0
16
dude, trust me

remove your max_open_files line and it will start working properly. 4096 is insanity.


this option should NEVER be changed from default unless you know what you are doing and it's obvious you don't

I'm not trying to be a dick, i just see this issue a lot. NEVER mess with max_open_files


this number is PER torrent, not global. 4096 is WAY too high. Leave this at default and let the kernel handle it.