Go Back   FlashFXP Forums > > > >

ioFTPD General New releases, comments, questions regarding the latest version of ioFTPD.

Reply
 
Thread Tools Rate Thread Display Modes
Old 09-03-2011, 07:36 AM   #106
Dahlia
Member
 
Join Date: Sep 2008
Posts: 33
Default

Error.log
09-03-2011 01:58:49 System detected loader lock compromised! Terminating!
Watch.log
09-03-2011 01:58:50 Received deadlock signal from ioFTPD, killing it.

Running ioftpd on fully updated win2003sp2/intel dual core cpu. What can i improve or change so this will not happen again ? Might be some hw or software compatibility problem ?

Ioftpd as service automatically restarted. But again need to cache all the directories in deep VFS structure, which is annoying :/

Found in Debug.log this:
09-03-2011 00:12:03 Worker exit: total=5, free=5, blocking=0, initial=5.
But since the time is differnet its probably somthing else.
Dahlia is offline   Reply With Quote
Old 09-03-2011, 08:08 PM   #107
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

Dahlia: Unfortunately that's the dreaded lockup bug. There isn't much you can do unless you're running an older version of ioFTPD. If it's still happening then you'll just have to live with the rare auto-restart because my next attempt at specifically trying to eliminate it won't happen before v8.1. The good news as you pointed out is it does detect this problem now and it will restart itself if run as a service...

Oh, the worker thread message just means at some point you needed more than the default 5 worker threads so one was dynamically created and then it exited. It probably can't hurt to adjust the minimum number of Worker_Threads under [Threads] to like 8 or something and see if you get any more of those debug messages.
Yil is offline   Reply With Quote
Old 09-04-2011, 11:07 AM   #108
Dahlia
Member
 
Join Date: Sep 2008
Posts: 33
Default

Thank you for reply.
Dahlia is offline   Reply With Quote
Old 09-11-2011, 06:57 AM   #109
Dahlia
Member
 
Join Date: Sep 2008
Posts: 33
Default

Found something weird with the timestamps(date/time) of subdirs in daydirs. Maybe its just some bad settings in my configs, but i dont know what to change.

Time/Date od DAYDIRs is not changing when i upload new dir into it. The Time/Date of the DAYDIR change only after i create a new DAYDIR. Well its little more complicated to explain so i did some screens.

In VFS structure is: private/_mp3/%DAYDIR (tried with mp3/%DAYDIR) its not working there either.

[1]
I created some sample dirs in DAYDIR: .../0913 like test03,test05,... (just mkdir, no del or anything)

Time/Date of DAYDIR is not changing.

[2]
but after i create a new DAYDIR, the previous dirs got updated (something wrong with timestamp cache there?)

I tried to even delete .ioftpd from those dirs but no effect on timestamps.

And why i want to have a real date there ? Its good for sorting so i see the dirs with new stuff first/last.


I have ioftpd 7.7.3, ioninja, nxtools.
Dahlia is offline   Reply With Quote
Old 09-11-2011, 03:16 PM   #110
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

Dahlia: I can confirm that if you create a simple directory that the parent of that directory isn't being marked stale in the cache. Isteana pointed out a very similar problem with plain files and I fixed that in 7.7.3, but looks like the same thing is happening with dirs as well...

I should point out that if you are using a zipscript the problem is mitigated because the creation of the incomplete symlinks in the parent dir after the first file is uploaded usually forces the cache to be updated...
Yil is offline   Reply With Quote
Old 09-12-2011, 09:59 AM   #111
Dahlia
Member
 
Join Date: Sep 2008
Posts: 33
Default

Is it possible to do some 'force' manual update of that cache ? Or ill crontab some mkd/del dir for now
Dahlia is offline   Reply With Quote
Old 09-12-2011, 05:46 PM   #112
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

Actually there are several ways to force cache updates.

You can use 'site refresh' or 'site refresh <dir>' to invalidate a cache entry at any time which is probably all you need. You can use the TCL command [vfs flush <dir>] in a script which you could tie to a POST MKD event, or there is an exe script (I forget the name though) which you can call to do the same thing.
Yil is offline   Reply With Quote
Old 09-19-2011, 03:26 PM   #113
pion
Senior Member
 
Join Date: Feb 2006
Posts: 138
Default

Another case of lockup bug in ioFTPD 7.7.2

Error.log

09-19-2011 21:16:26 System detected loader lock compromised! Terminating!

Watch.log

08-18-2011 18:33:30 Received deadlock signal from ioFTPD, killing it.
08-18-2011 20:56:39 Received deadlock signal from ioFTPD, killing it.
08-19-2011 13:30:24 Received deadlock signal from ioFTPD, killing it.
08-20-2011 01:48:05 Received deadlock signal from ioFTPD, killing it.
08-20-2011 03:15:59 Received deadlock signal from ioFTPD, killing it.
08-20-2011 10:21:13 Received deadlock signal from ioFTPD, killing it.
08-20-2011 11:03:56 Received deadlock signal from ioFTPD, killing it.
08-20-2011 14:16:11 Received deadlock signal from ioFTPD, killing it.
08-20-2011 14:59:21 Received deadlock signal from ioFTPD, killing it.
09-19-2011 21:16:26 Received deadlock signal from ioFTPD, killing it.
09-19-2011 22:03:48 Received deadlock signal from ioFTPD, killing it.
pion is offline   Reply With Quote
Old 09-19-2011, 04:06 PM   #114
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

pion: Interesting... Especially to see a whole bunch grouped together and then nothing for a month.

I do have one idea based upon Dahlia's info. See if the 'Log_Exiting_Workers_Threads' under [Threads] is enabled in the .ini file and then see if you can find any lines like 'Worker exit: total=5, free=5, blocking=0, initial=5.' in logs/Debug.log.

A long time ago I fixed a number of side effects of worker threads exiting. This included things like timers and async events being canceled. I'm not aware of any remaining issues like this, BUT whenever a thread is created/destroyed all the dll's are offered a chance to run. Most don't do anything, but the important point here is the lock that protects the dll's while they do this is what is getting stuck.

In my opinion removing the Microsoft encryption library really helped stability because it would acquire/release that lock every time an SSL connection was made. The MS encryption library dynamically reference counted the underlying encryption modules it was using. This greatly increased the odds of something going wrong it looks like!

My guess is during those times when you saw a problem the server was really busy and perhaps worker threads were being created / destroyed. Either this causes the problem because ioFTPD is doing something wrong (an uncaught side effect) or because some other weird thing is happening and the odds go up...

Anyway, it's a thought and perhaps looking at the logs we might be able to tell one way or the other.
Yil is offline   Reply With Quote
Old 09-19-2011, 04:17 PM   #115
paja1
Member
ioFTPD Registered User
 
Join Date: Jul 2005
Posts: 43
Default

I have strange "error" as well.
I simply can't see some files. There are fine, i can copy them see them in console, total commander and explorer. But their are unvisible in ioFTPD. All are quite big MKV files. Any idea?

Is there anything that can help you to find why?

Thanks!
paja1 is offline   Reply With Quote
Old 09-19-2011, 05:25 PM   #116
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

paja1: ioFTPD is pretty good about checking the timestamp on a directory to detect changes so it can invalidate its cache whenever it lists a directory. The only problem I'm aware of is with zipscripts dealing with files uploaded onto FAT32 filesystems because the last modified timestamp isn't accurate enough to catch changes. NTFS works fine and if you have large files you're using NTFS.

You can test to see if it's a caching issue by going to the dir and using 'site refresh' and re-listing the dir. That should force it to go grab a new snapshot of the directory. On the other hand you also need to rule out that the FTP client isn't at fault. It's standard fare for clients to cache directory listings and you need to make sure you have a current copy. Try re-starting the FTP client (make sure ALL copies of it are closed first!) and see if that makes a difference.

If none of that works then there might be something funny about the file. ioFTPD will refuse to show "hidden" or "system" files and directories. This prevents it from exporting special protected things like the recycle bin, the page file, etc. Open up the properties tab for the file and see if it's special in any way. I don't think access permissions on an individual file can prevent it from showing up in a directory listing if other files are showing up in that dir (which means the dir's permissions must be ok).

See if any of that helps. If all else fails, try deleting/renaming the .ioFTPD file just in case it's corrupted. I'm not sure how that could prevent some files from showing up, but it's something to try. You'll need to 'site refresh' the dir or even restart the server after touching the .ioFTPD file. I guess restarting the server is another option to try in some of the above cases as well.

If you have an active server and restarting is painful let me know and I'll show you how to setup a local test server you can run side by side with your regular one.
Yil is offline   Reply With Quote
Old 09-20-2011, 03:28 AM   #117
paja1
Member
ioFTPD Registered User
 
Join Date: Jul 2005
Posts: 43
Default

Yil,
thanks for answer, but I've restarted ioFTPD several times, all attributes are set only to "a" archive, and security is set to r/w for all users.

Other files are visible normally in the folder, if I create new e.g. "foo.txt" it appears immediately nut no this one big MKV file.

Yes, i did tried to delete all .ioFTPD files, no change

Even if i rename it to *.txt or anything else, still invisible ;(

And I've tried about 5 different version from 5.xx to 7.xx of ioFTPD, still the same.

Any more hints?

Thanks!
paja1 is offline   Reply With Quote
Old 09-22-2011, 06:18 AM   #118
pion
Senior Member
 
Join Date: Feb 2006
Posts: 138
Default

I might add that the lockup is happening quite frequently when under load, and more so on lower spec'ed machines.
pion is offline   Reply With Quote
Old 09-22-2011, 03:41 PM   #119
paja1
Member
ioFTPD Registered User
 
Join Date: Jul 2005
Posts: 43
Unhappy

Quote:
Originally Posted by paja1 View Post
Yes, rebooted several times, install all updates and hot fixes, still the same.
Only turning off monitoring of applications activity helped. Even switching it into check only incoming data helped as well.
Why it affected only OpenSSL version, dunno.

Pavel

Unfortunately my speed issues are still there. Is there anybody else having similar issue?
That version 6.x is much faster than 7.x?

I thought it was because of MS Security Essentials, but i was wrong.

I had to roll back back to 6.6. But 7.7.3 is more stable and have no issue with refreshing of folders behind NTFS Symlinks. So i would like to go back to 7.7. but i have to choose between features and stability vs. ultimate speed.

I have to clue what to do at this point. Additionally i still have the issue with "invisible" files. Any help? Anyone? Please!
paja1 is offline   Reply With Quote
Old 09-22-2011, 05:45 PM   #120
Yil
Too much time...
FlashFXP Beta Tester
ioFTPD Administrator
 
Join Date: May 2005
Posts: 1,194
Default

paja1: I tried looking at some large .mkv files (10gig or so) locally and I don't have any problems. If the directory doesn't have a .ioFTPD file I can't see any reason the file wouldn't be visible.

Do you use FlashFXP? If so check out Directory->View Raw Directory and see what it says. ioFTPD outputs data in alphabetical order with Folders before Files so if you can see the entry above and below where it should then it's missing from the listing and not a client issue. Perhaps similar options exist in other FTP clients. I just want to rule out a client having problems with >4gig files since that's the limit of a 32bit file size.

Also, pay attention to the size of the directory (the ".") entry or the size in the parent's listing for the dir. Can you tell from the size if the file must be present? I..e a dir with small files is somehow 10gig because of the missing .mkv?

If you manually create a .txt file and then refresh the dir listing do you see the new file? What about after a 'site refresh'? If you see the new file then it isn't a stale cache problem.

If you are using a path that involved hard links try to find one that doesn't just to see if that changes anything.


As far as your speed issue goes... Does this happen if you are only transferring one file? What's the CPU load on the machine? # of processors? Try increasing the number of io_threads from the number of cores on your machine to double that or at least 4-5 if only a single core. That might make a difference.
Yil is offline   Reply With Quote
Reply

Tags
bug, code, directory, release, user

Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 08:52 AM.

Parts of this site powered by vBulletin Mods & Addons from DragonByte Technologies Ltd. (Details)