FlashFXP Forums ioFTPD v7.5.9 Released (Beta)
 Tickets Search Today's Posts Mark Forums Read

 05-18-2010, 02:27 PM #2 Yil Too much time... FlashFXP Beta TesterioFTPD Administrator   Join Date: May 2005 Posts: 1,163 Changelog Code: v7.5.0 Release Notes: 1) Files in \System: Changed : ioFTPD.[exe,pdb] - Version 7.5.0.0. Changed : ioFTPD-Watch.[exe,pdb] - Version 2.0.0.0 Changed : Theme.ini Changed : Help.ini, Help-SiteCmds.ini, Help-nxTools.ini, Help-ioNiNJA.ini Changed : ioFTPD.ini - summary of changes by section... [FTP_Service] : Description deleted Create_Certificate added Min_Cipher_Strength deleted Max_Cipher_Strength deleted OpenSSL_Options added OpenSSL_Ciphers added Deny_Port_Host_# added [Network] : Log_OpenSSL_Transfer_Errors added [VFS_PreLoad] : VFS comment/features changed. [FTP] : Site_Name added at top Data_Timeout added Chmod_Check added Site_Box_Header, Site_Box_Footer added Help_Box_Header, Help_Box_Footer added [Threads] : Restart_On_Deadlock comment/features changed. 2) Files in \text\ftp: Changed : [AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Header [AllDn, AllUp, WkDn, WkUp, MonthDn, MonthUp, DayDn, DayUp].Footer Changed : [ClientList, MyInfo, UserInfo, Who].Header [ClientList, MyInfo, UserInfo, Who].Footer Changed : ClientInfo.[Common, Download, Idle, List, Login, Upload] Changed : Welcome 3) Files in \scripts: Changed : FormatHelp.itcl 4) Files in \Source: Changed : Site-cmds.help 5) Files in \Doc: Changed : Cookies.txt, iTCL.txt Changed : FTP-cmds.txt, Site-cmds.txt, nxTools.txt, ioNiNJA.txt *** Incompatible changes/defaults: 6) The default settings for the new safety feature preventing the PORT command from accessing private/local LAN IP addresses means you will need to use PASV connections for local transfers using a FTP client on the LAN. This is usually the default method so you probably won't notice. However FXPing between 2 FTP servers across the local LAN (i.e. both 192.168.*) will no longer work with the default settings. The server will also not be able to FXP to itself (although if people want this I can probably find a way to allow it). To enable FXP between two local machines you can choose to either disable the new feature on one site, or you can just define a second Service that is only accessible to machines on the local LAN that has the feature disabled. The second Service method is HIGHLY recommended because you can also do things like relax the encrypted data transfer requirement which will improve local transfer speeds. 7) To take advantage of Diffie-Huffman ephemeral keying you will need a new new key file (.dhp) which means you need to re-generate the SSL certificate. To do this you can remove the old certificate by deleting the .key and .pem files in the system directory before starting the server or use "site removecert ". Then use the "site makecert" command, or enable the new auto-generate cert feature and re-start the server. 8) Removed ioFTPD.ini options Min_Cipher_Strength and Max_Cipher_Strength under [FTP_Service]. These have been replaced with the OpenSSL_Ciphers option. *** New features: 9) New ioFTPD.ini option (Deny_Port_Host_ under [FTP_Service]). Active mode data transfers require the server to create connections to a user specified IP/Port. For security reasons the server should be prevented from initiating connections to the server box or any other machine behind your firewall if you have one. By default the server will now block access to the following non-routable private IP ranges: 10.*, 172.16.*, 192.168.*, and the loopback interface 127.*. To disable this feature entirely just specify 0.0.0.0 as the host to block. Alternatively, you may specify your own custom list of IP addresses/ranges to block. 10) The 'Restart_On_Deadlock' feature under [Threads] has been re-done. It was originally designed to handle the DLL loader lock getting stuck. When that happened the server would be unable to exit and so it would signal the ioFTPD-Watch process to forcefully terminate the server. That part remains the same, but the ioFTPD-Watch process now requires the server to signal it's alive at least once every minute else it will assume something bad happened and forcefully terminate it. On top of that, the server will now attempt to connect to all active services every minute and if that fails 3 times in a row it will try to exit gracefully. If it can't exit the ioFTPD-Watch process will time it out after another minute or so and forcefully terminate it. You can view actions or error messages by the ioFTPD-Watch.exe process in the new "Watch.log" file stored in the server's log directory (\$Log_Files). 11) New ioFTPD.ini option (Create_Certificate under [FTP_Service]). If 'True' this feature will create a new SSL certificate right after the server is started if no certificate was found for the service. 12) New ioFTPD.ini option (OpenSSL_Options under [FTP_Service]). You can now specify any v1.0 OpenSSL option flag to modify the encryption library's behavior. Arguments are separated by "|" and the "SSL_OP_" prefix should be left off. The complete list of options is available at: http://www.openssl.org/docs/ssl/SSL_CTX_set_options.html The 2 suggested options are: ALL - enable all compatibility options to work around broken SSL implementations. NO_TICKET - Disable RFC4507bis tickets for stateless session resumption. FlashFXP disabled this because of issues with some Java SSL implementations so I figure we should do the same. The default is no options. 13) New ioFTPD.ini option (OpenSSL_Ciphers under [FTP_Service]). You can now control exactly which ciphers are available and in what order they should be chosen. Documentation is available at: http://www.openssl.org/docs/apps/ciphers.html The default, if undefined, is "DEFAULT:!LOW:!EXPORT" which excludes anything under 128 bits. This affects both control and data connections. 14) New site command (site ciphers [-all]). This command displays available ciphers in the order they are chosen for the service you are connected to. A bit of information about each is also included by the OpenSSL library. [Note: the columns are generated by OpenSSL itself and aren't aligned, but pulling the data out of private structures subject to change just to align the columns didn't seem worth it - I may fix OpenSSL itself later though!] If you supply the "-all" argument then it will display the complete list of ciphers supported by OpenSSL itself. 15) The server now supports elliptic-curve and Diffie-Huffman based ephemeral key algorithms for one-time use ciphers which means the server now uses the most secure algorithms available to OpenSSL. You will need to re-generate the certificate to enable DH based algorithms. See #7 above. The "parameters" for generating ephemeral keys are stored in a new file ".dhp". 16) The server will now display the descriptive error messages returned by the OpenSSL library in a number of cases including data transfer errors to help users understand what the problem is and report problems better. 17) New ioFTPD.ini option (Log_OpenSSL_Transfer_Errors under [Network]). You can have the server automatically log OpenSSL error messages to the Debug.log file. The default is not to. 18) New ioFTPD.ini option (Data_Timeout under [FTP]). ioFTPD used to have a very liberal 10 minute timeout between receive/send calls of a data transfer before automatically aborting the transfer. The new default is 2 minutes, however you can use this setting to make it whatever you want. Be aware that exponential backoff for ethernet re-transmission can be 30 seconds across a LAN and that internet routing hiccups can lose or delay packets for several minutes. 19) New ioFTPD.ini option (Chmod_Check under [FTP]). You can now modify the behavior of the 'site chmod' command by choosing one of 3 settings. Master accounts can do anything under any setting so are not affected. Default : Require +w to parent of item being modified, and non-VFS admins must own the item being modified. WriteOnly: Require +w to parent of item being modified (no owner check). NoChecks : Can modify anything provided you can see it. The 'Default' setting is the original behavior and the default. 20) Modified ioFTPD.ini option (VFS under [VFS_PreLoad]). You can now completely disable the directory cache preloading feature by specifing the name of the VFS file to use as 'DISABLE'. 21) New ioFTPD.ini option (Site_Name under [FTP]). You can now define a custom name for your ftp server that will be used to customize the output of some site commands and help output. The default is 'ioFTPD'. 22) New ioFTPD.ini options (Site_Box_Header and Site_Box_Footer under [FTP]). This is the string to display in the top and bottom of site commands that contain bounding boxes or borders (- and |) around them. The string is fully processed by the message cookie parser and the defaults use the new formatting super cookies below so you can use custom themes to get colors and you don't have to worry about the box aligning correctly. 23) New ioFTPD.ini options (Help_Box_Header and Help_Box_Footer under [FTP]). Essentially the same as the Site_Box versions but used for 'help' and 'site help' output. 24) New super cookies (%[SiteName], %[SiteBoxHeader], %[SiteBoxFooter], %[HelpBoxHeader], %[HelpBoxFooter]). Returns the value of the associated option under the [FTP] section in the .ini configuration file. 25) New super cookie (%[SiteCmd]). Returns the name of the current site command being executed. 26) New super cookies (%[Mark], %[Fill(,)], %[Pad(,)]). These 3 cookies provide a powerful means of aligning data. First you use the Mark cookie to record the current position on a line, and then you use the Fill cookie to guarantee that there are at least characters from the marked position by appending data as needed. The default is to fill with spaces, however you can provide an arbitrary string and characters from it will be used one at a time, in order with looping, until the field is exactly characters wide not counting any ANSI control codes such as color specifiers. The Pad cookie works the same way except instead of appending to the end of the original string to reach the desired width it inserts characters at the beginning (marked position) so the original string will be right aligned. There probably should be a Center cookie as well, let me know if you need that. NOTE: Mark/Fill or Mark/Pad cannot span lines. 27) New super cookies (%[Save] and %[Restore]). The Save cookie will record the current theme/subtheme, and the current text settings of colors, etc. You can then change these however you want and at a later time revert to the saved settings via the Restore cookie. 28) Exported functions Config_Get, Config_GetInt, Config_GetBool, Config_GetPath changed to support multiple .ini files again such as v7.1 supported. This requires use of nxMyDB v2.1.0+, however only v2.1.1+ should be used as that is the first release to support the required handle locking and should be used with the custom libmysql.dll as well. *** Bug Fixes: 29) Fixed a serious bug where the server was passing the port re-use flag to the bind() function when processing the PASV command. This could result in server giving the same port # to 2 or more users at the same time and if the connections to that port didn't arrive in the order they were handed out the uploaded files would get swapped! This bug goes all the way back to before v5.8.5 so it isn't new and in most configurations with a decent range of passive ports must have been relatively rare but it's a big bug. 30) Fixed a serious bug introduced in v7.4 that would cause the server to crash if the number of outstanding requests to a particular disk was higher than the Device_Concurrency setting in the .ini file. 31) Fixed a bug where an already closed handle for a socket could be referenced by the server. When a TCP connection is timed out by the server or an ABOR command is issued the server forces closed the socket handle which results in any active overlapped I/O operations being aborted and invalidates the handle. However, if the server was just about to issue a new send or receive request it could use the just closed handle. I've added additional locking to prevent this from happening now. 32) Fixed a bug where the server failed to mark the cached directory as stale after creating the new file for uploads. 33) Fixed a bug with marking directories as stale. The dirty/stale flag is set outside of any locks, but is tested later while holding the lock and cleared if set. The test/clear operations were not atomic and thus a rare race condition was possible and it could be marked as current instead of stale. A simple change to make this atomic fixes it. 34) Fixed a bug where a thread could make a change to a directory, mark the directory as stale, and then request information about a file or the directory itself and not see the change. This can happen because the server assumed that if someone was already updating the cached directory entry that it would be up to date when finished. This is obviously not true if the update started before the 2nd thread made it's change. The directory stale flag is now tested before making this assumption and if it is set updates the directory again. This does not guarantee that a directory returned is always perfectly up to date (which would be both hard to do and a really bad idea), but rather than any actions made by the calling thread will be visible. 35) Fixed a bug where the server wasn't setting the blocking thread flag early enough when sending non-buffered data (iputs -nobuffer or SendQuick()) that could cause a deadlock if all worker threads decided to do this all at the same time. 36) Fixed a bug where the TCL [waitobject wait] command wasn't setting the blocking thread flag which indicates it could block the worker thread indefinitely. 37) Fixed a bug where the server could reject an upload because too many were in progress by a user but no error description was provided. 38) Fixed a bug where the Default_VFS file was always processed even if a custom one was specified via the 'VFS' option of [VFS_PreLoad]. Even worse was the fact that Default_VFS wouldn't do this in parallel during startup. 39) Changed PORT failure response code to 501 from 550 to comply with RFC. 40) The Dark-Bright theme for 'site who' changed to use high intensity colors. *** Known Bugs: 41) ioFTPD is unable to handle key re-negotiation of an established SSL connection. The OpenSSL library provides support for this but currently the server cannot handle it. This is not a feature loss since ioFTPD also didn't do this with the MS encryption library. 42) ioFTPD does not provide support for 512 bit ephemeral keys for use with weak "export" grade ciphers. Those ciphers should not be used at all.
 05-18-2010, 03:47 PM #3 pion Senior Member   Join Date: Feb 2006 Posts: 138 500 'IDNT': Command failed. both using wildcards, and full ip (Using excactly the same config as for 7.4.5 which worked fails) Dirlisting appear to not work (even tho I know it's preloading some stuff, but still takes so much time that it in fact looks like server is hanging, both on PASV and STAT -l command.) This is with DELAY = TRUE in ioftpd.ini After that part is done, dir list is behaving properly. But it looks really strange with a dirlisting command that just hangs in client without getting reply for several minutes.. first assumption then is for pasv port to be blocked or hdd problems.. Last edited by pion; 05-18-2010 at 04:10 PM.
 05-18-2010, 06:01 PM #4 Yil Too much time... FlashFXP Beta TesterioFTPD Administrator   Join Date: May 2005 Posts: 1,163 Code: v7.5.1 Release Notes: 1) Files in \System: Changed : ioFTPD.[exe,pdb] - Version 7.5.1.0. *** Bug Fixes: 2) Fixed the IDNT comment I accidentally broke when adding the Deny_Port_Host feature.
 05-18-2010, 06:18 PM #5 Yil Too much time... FlashFXP Beta TesterioFTPD Administrator   Join Date: May 2005 Posts: 1,163 pion: Fixed the IDNT command which I broke... Check the VFS option under preloading. I fixed the bug in v7.5.0 with the preloading feature where it would process both the VFS file and the Default_VFS file instead of just the VFS file if it was specified. Is it possible the directory that you notice the difference for was in the default.vfs file and not the file pointed at by the VFS= file? That would be the most likely reason for seeing a change.
 05-19-2010, 03:42 AM #6 pion Senior Member   Join Date: Feb 2006 Posts: 138 Crashed after less than 12 hours uptime. Not a single transfer comes trough after a period of 'working' time. In addition this, I see the following in log: 05-19-2010 00:18:15 ------------------------------------------------------------ 05-19-2010 00:37:17 ------------------------------------------------------------ 05-19-2010 01:02:48 ------------------------------------------------------------ Which in my case means that io has restarted 3 times, due to I wiped logs after first run. Debug.log is filled with: 05-19-2010 05:01:01 Accepted port 12345 and some places: 05-19-2010 05:01:06 AsyncSelectCancel flags: 10 Uploading dumpfile created with windbg, in this state: (10:25:09) [glftpd] 200 PORT command successful. (10:25:09) [io750] STOR myfile.r22 (10:25:09) [io750] 150 Opening BINARY mode data connection for myfile.r22 using SSL/TLS. (10:25:09) [glftpd] RETR myfile.r22 (10:25:09) [glftpd] 150 Opening BINARY mode data connection for myfile.r22 (12345 bytes) using SSL/TLS. (10:25:29) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_accept()), disconnected: Connection reset by peer. (10:25:29) [glftpd] Reversed FXP started (10:25:29) [io750] 426 Connection closed: Connection timed out. (10:25:29) [io750] CPSV (10:25:29) [io750] 226 ABOR command successful. (10:25:29) [io750] 227 Entering Passive Mode (1,1,1,1,65,194) (10:25:29) [glftpd] PORT 1,1,1,1,65,194 (10:25:29) [glftpd] 200 PORT command successful. (10:25:29) [io750] STOR myfile.r22 (10:25:29) [io750] 150 Opening BINARY mode data connection for myfile.r22 using SSL/TLS. (10:25:29) [glftpd] RETR myfile.r22 (10:25:29) [glftpd] 150 Opening BINARY mode data connection for myfile.r22 (12345 bytes) using SSL/TLS. (10:25:49) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_accept()), disconnected: Connection reset by peer. (10:25:49) [glftpd] Reversed FXP started (10:25:49) [io750] 426 Connection closed: Connection timed out. (10:25:49) [io750] CPSV (10:25:49) [io750] 226 ABOR command successful. (10:25:49) [io750] 227 Entering Passive Mode (1,1,1,1,61,11) (10:25:49) [glftpd] PORT 1,1,1,1,61,11 (10:25:49) [glftpd] 200 PORT command successful. (10:25:49) [io750] STOR myfile.r22 (10:25:49) [io750] 150 Opening BINARY mode data connection for myfile.r22 using SSL/TLS. (10:25:49) [glftpd] RETR myfile.r22 (10:25:49) [glftpd] 150 Opening BINARY mode data connection for myfile.r22 (12345 bytes) using SSL/TLS. (10:26:09) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_accept()), disconnected: Connection reset by peer. (10:26:09) [glftpd] Reversed FXP started (10:26:09) [io750] 426 Connection closed: Connection timed out. (10:31:04) [glftpd] CPSV (10:31:04) [glftpd] 227 Entering Passive Mode (2,2,2,2,46,107) (10:31:04) [io750] PORT 2,2,2,2,46,107 (10:31:04) [io750] 200 PORT command successful. (10:31:04) [io750] STOR file.r21 (10:31:04) [io750] 150 Opening BINARY mode data connection for file.r21 using SSL/TLS. (10:31:04) [glftpd] RETR file.r21 (10:31:04) [glftpd] 150 Opening BINARY mode data connection for file.r21 (12345 bytes) using SSL/TLS. (10:31:24) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer. (10:31:24) [glftpd] CPSV (10:31:24) [glftpd] 227 Entering Passive Mode (2,2,2,2,46,140) (10:31:24) [io750] 426 Connection closed: Connection timed out. (10:31:24) [io750] PORT 2,2,2,2,46,140 (10:31:24) [io750] 226 ABOR command successful. (10:31:24) [io750] 200 PORT command successful. (10:31:24) [io750] STOR file.r21 (10:31:24) [io750] 150 Opening BINARY mode data connection for file.r21 using SSL/TLS. (10:31:24) [glftpd] RETR file.r21 (10:31:24) [glftpd] 150 Opening BINARY mode data connection for file.r21 (12345 bytes) using SSL/TLS. (10:31:33) [i] (User Abort) (10:31:33) [glftpd] ABOR (10:31:33) [io750] ABOR (10:31:33) [glftpd] 435 Failed TLS negotiation on data channel (using SSL_connect()), disconnected: Connection reset by peer. (10:31:33) [glftpd] 225 ABOR command successful.
 05-19-2010, 04:03 AM #7 pion Senior Member   Join Date: Feb 2006 Posts: 138 My preload settings: [VFS_PreLoad] VFS = ../etc/sections.vfs DELAY = TRUE (10:54:10) [io750] PWD (10:54:10) [io750] 257 "/" is current directory. (10:54:10) [io750] STAT -l (10:57:30) [io750] Timeout, Connection closed This looks very much broken from a clients perspective.
 05-19-2010, 06:06 AM #8 Zer0Racer Senior Member ioFTPD Scripter   Join Date: Oct 2002 Posts: 702 Yil, is it correct that [VFS_PreLoad] uses slashes in path instead of backslash like in other places in ioFTPD.ini? Ie. Default_Vfs = ..\etc\default.vfs
 05-19-2010, 11:10 AM #9 Yil Too much time... FlashFXP Beta TesterioFTPD Administrator   Join Date: May 2005 Posts: 1,163 zero: The VFS= line to specify a .vfs file is an OS path so something like VFS=..\etc\default.vfs would be the correct form for that. The rest of the 2 = /XVID type lines are FTP paths and thus are forwards (/) slash separated. pion: Hmm, guess I forgot to remove a debug line, you shouldn't be seeing that 'Accepted Port' stuff. Evidently whatever is causing issues on your sites appears to still be there... If you see it restarted 3 times was that automatically restarted as opposed to you doing it manually? Can you check logs\Watch.log and see what it says? If it's auto-restarting on whatever the lockup problem you are having is then it's at least some progress was made as it doesn't require manual intervention now... pion: Did you check to see that the directory you are timing out on was listed in the sections.vfs file? What does the PRELOAD: and START: lines look like in ioFTPD.log and what is their order?
 05-19-2010, 12:25 PM #10 pion Senior Member   Join Date: Feb 2006 Posts: 138 Naah, watch.log doesn't kick in. However, the daemon is accepting connections at all times now. So that's some sort of progress I suppose. But in any case, it's even worse off now, because now there's now way to know if it's crashed unless you start a transfer..
 05-19-2010, 03:08 PM #11 Yil Too much time... FlashFXP Beta TesterioFTPD Administrator   Join Date: May 2005 Posts: 1,163 Wait a sec pion. Does this new build always accept control connections now? Even when it isn't accepting file transfers or data channel listings? Can you try to exhaust the 10 pre-allocated control connections by just logging in/out 12 times? It looks like ioFTPD isn't finding problems connecting to itself via it's internal testing since nothing is showing up in the watch logfile. Remember it takes 3 failures in a row and there is a minute between tests so at least a 3 minute detection window on the control channel is necessary, but if you can always connect then this obviously won't fail... If it's just data connections then that's real progress. The PORT/PASV (and ident check) logic uses an async handler callback and it's possible that something is getting screwed up there. When entirely different logic used for new control connections was also broken it seemed unlikely that that was the problem, but perhaps there was more than 1 issue... In fact I can think of 1 potential issue right off the bat that I'll look into.
 05-19-2010, 05:03 PM #12 Yil Too much time... FlashFXP Beta TesterioFTPD Administrator   Join Date: May 2005 Posts: 1,163 One other thing I just though of... That leftover "Accepted port" stuff. Can you check to see if that line occurs after things look broken? I'm guessing you won't, but as long as it's still in there might as well get another piece of debugging info...
 05-20-2010, 04:06 AM #13 pion Senior Member   Join Date: Feb 2006 Posts: 138 Control connections are accepted, yes. This was also the case when I disabled nxmydb before. But now I have nxmydb enabled, and control connection is always accepted.
 05-20-2010, 07:11 AM #14 pion Senior Member   Join Date: Feb 2006 Posts: 138 Entering with 12 new connections worked fine while in crashed state. 05-20-2010 12:53:15 Accepted port 15073 These messages also stopped appearing in debug.log
 05-20-2010, 11:12 PM #15 Yil Too much time... FlashFXP Beta TesterioFTPD Administrator   Join Date: May 2005 Posts: 1,163 7.5.2 Changelog Code: v7.5.2 Release Notes: 1) Files in \System: Changed : ioFTPD.[exe,pdb] - Version 7.5.2.0. *** Bug Fixes: 2) Fixed a bug where the built-in default Port_Deny_Host settings were improperly setup and not selected automatically if the .ini setting is missing completely. 3) Fixed a bug where changes to Device options in the .ini file might not be loaded immediately after a site rehash because of an internal check to make sure that isn't done within 30 seconds of the last update. That check is intented to prevent looking up the values over and over again if more than one service shares the same Device. That check is still used but the rehash counter must not have been incremented. 4) Added some Debug.log lines to catch interesting async socket related events.