Go Back   FlashFXP Forums > > > >

Suggestions Got a new idea or addition which would benefit IOFTPD? Post it here!

Reply
 
Thread Tools Rate Thread Display Modes
Old 08-24-2004, 05:31 AM   #1
Flamer
Junior Member
ioFTPD Foundation User
 
Join Date: Aug 2004
Posts: 10
Lightbulb Multi IP balancing

First I want to tell you how impressed I am of the speed and ressource usage of ioFTPd! WOW! GREAT!
I wonder if the registered version can top this (will reg soon)...

Currently I'm using v5.8.4u and I have 3 NICs -> 3 IPs in my server.
I have setup io to use these and it works very well.

BUT I still have a suggestion for the operation of the selection of the IP for the next PASV command
Instead of randomly or round-robin selecting the IP could you implement an algorithm that selects on usage? E.g. count the connections each IP has and choose the one with least connections for next PASV. (Shouldn't be too hard to implement I think)
And to improve this scheme: don't count connections for listings if clients don't use "STAT -l", not to mention the control connection
<- Umh, I just thought about it a second time, you had to wait what client does next after it gets PASV reply to decide if its listing or transfer, so it might not be that good since it consumes ressources...so don't bother about that too much, first suggestion would be great if you would implement it ->

Thank you!

Regards,
Flamer
Flamer is offline   Reply With Quote
Old 09-11-2004, 04:31 AM   #2
Flamer
Junior Member
ioFTPD Foundation User
 
Join Date: Aug 2004
Posts: 10
Question

Any comments on my suggestion, darkone? ;p
Flamer is offline   Reply With Quote
Old 09-11-2004, 08:29 AM   #3
darkone
Disabled
FlashFXP Registered User
ioFTPD Administrator
 
darkone's Avatar
 
Join Date: Dec 2001
Posts: 2,230
Default

What comes to registered version, it should be somewhat faster (as number of threads handling input/output, is limited to 1 in unregistered version + there's memory pooling in the registered version - which reduces memory fragmentation + it speeds up memory allocations)

There are certain problems related to load balancing per number of connections / bandwidth / bytes transfered per device.

Number of connections: directory lists make it nearly impossible to guess how many of the connections are being used for file transfers. If someone issues list at same time you begin your transfer, there's 50% chance that wrong device is selected (device 'x' has two file transfers while device 'z' has none). Not to mention problems that stalled connections, slow transfers etc. cause.

Bytes transfered: What if two clients concurrently begin transfering 'n'gb file. Server would need to know name of file that is going to be transfered, before client issues PASV/PORT.

Bandwidth: You'd need to know average transfer rate for transfer that has not yet been done, before the client begins transfering.

Number of connections is the only thing that _could_ actually provide decent accuracy, but it requires hundreds of simultanous transfers.
darkone is offline   Reply With Quote
Old 09-11-2004, 09:24 AM   #4
Flamer
Junior Member
ioFTPD Foundation User
 
Join Date: Aug 2004
Posts: 10
Default

Thank you for your answer!
(Although I'm not quite satisfied with it )

I should have mentioned my thoughts when I posted my suggestion: the intended usage is for a LAN where you have high-speed transfers.

I think in a situation where there are many users (lets say from twice the number of NICs) downloading not too large files from the ftp server, the round-robin scheme is a good selection.

I want to explain a scenario where IP selection based on connection count would be of great advantage:
(Lets assume my config with 3 NICs used for balancing)
Lets further assume there are mostly large (>500MB) files on the server, like videos or stuff...
Now let 2 or 3 clients browse and add files to their queues.
After that they begin downloading. So client1 will start on NIC1, client2 on NIC2 and client3 on NIC3, so far so fine.
But now client1 does not use full bandwidth cause it is downloading from other source or pc is slow or whatever so it gets only 4 mb/s. clients 2 and 3 get 11 mb/s and are far earlier ready and request the next file. So client2 will be on NIC1, together with client1 and gets only 7 mb/s. Client3 is fine alone on NIC2... now I assume worst case and client2 now has very large file to transfer, so both client 1 and 2 dl from NIC1 a long time. And Client 3 has some smaller files and will end up sharing NIC1 with the 2 others, effectively its download will be 1/3 what it could be, because NICs 2 and 3 idle... with greater number of users this -could- be worse.

I know this is not very likely to happen, BUT I want my clients always served at best possibilities , so if connections were counted they would get most bandwidth for their transfers.
For the listing connections, I don't think they affect this scheme too much, because 1.) who uses LIST anymore, we have STAT and 2.) its only a short time this connection is used, its unlikely that during this time a transfer starts.

So enough now...I think you already understand what I wanted to explain.

I don't want you to waste time implementing such an algorithm if you think its not worth it, I just want you to consider my suggestion next time you have some spare time and don't know what to do
But maybe you think now that its a not so bad idea after all and implement it in the not so far future, then I would be very glad!

Anyways, you've done great work on this ftp server!!

Regards,
Flamer
Flamer is offline   Reply With Quote
Old 09-11-2004, 10:23 AM   #5
Mouton
Posse Member
Ultimate Scripter
ioFTPD Administrator
 
Join Date: Dec 2002
Posts: 1,956
Default

I don't think any ftp client uses STAT by default, so I would say 99%+ of the users on a FTP server uses LIST.

ioFTPD is designed to optimally handle thousands of simultaneous connections. The target audience is not FTP servers with only 3 simultaneous clients. When you have thousands of clients, the chances of LISTs occuring at the same time as RETR is pretty high I would guess.
Mouton is offline   Reply With Quote
Old 09-11-2004, 10:56 AM   #6
Flamer
Junior Member
ioFTPD Foundation User
 
Join Date: Aug 2004
Posts: 10
Default

I totally agree with you, Mouton.

I know that my scenario is not the usual case, but nevertheless I have use for this balancing algorithm.
And that's also why I said -if darkone has some spare time- and does not know what to do he could think of my suggestion

Regards,
Flamer
Flamer is offline   Reply With Quote
Old 09-11-2004, 12:39 PM   #7
darkone
Disabled
FlashFXP Registered User
ioFTPD Administrator
 
darkone's Avatar
 
Join Date: Dec 2001
Posts: 2,230
Default

Well I think I can fit it into service/device rewrite. As I need to make them more generic (kinda like what my socket/file sturctures are atm with the new core)
darkone is offline   Reply With Quote
Old 09-11-2004, 07:13 PM   #8
Flamer
Junior Member
ioFTPD Foundation User
 
Join Date: Aug 2004
Posts: 10
Default

Sounds great
Looking forward to testing the new feature.
I just bought a license of ioFTPd, needed SSL and wanna see how the regged version compares in term of speed and resource usage to the unregged one...

Regards,
Flamer
Flamer is offline   Reply With Quote
Reply

Tags
connections, count, implement, pasv, usage

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Mysql wired57 Suggestions 4 01-25-2003 07:26 AM


All times are GMT -5. The time now is 01:46 AM.

Parts of this site powered by vBulletin Mods & Addons from DragonByte Technologies Ltd. (Details)