Custom Search
|
Date: August 30, 2005
From: Steven French <sfrench@xxxxxxxxxx>
I was looking at cifs to windows ME interop again last night. Working through read test cases to it I noticed that cifs_readpages needs work, not just minor restructuring to make the code clearer and easier to understand, and not just work to enable async dispatching of read request (like what James Roper had started working on previously) but to change a fundamental assumption of that code. There is an assumption in the cifs_readpages code that the server can at least support reading one page (which is not an assumption in cifs_readpage/cifs_read) which is not the case for windows me (which supports just over 2k read sizes). This is not a problem to any halfway recent server nor is a big problem in the short term since we could force mounts to these servers to use cifs_readpage by taking out the function pointer to readpages for inodes off that mount, but I think it is time to rewrite this code (presumably much useful code in real life gets rewritten multiple times).
Since the one the local filesystems use - block readpages (mpage_readpages) is not a very good example (although it is perhaps the only example), this requires some care - so any help would be appreciated and I will be happy to take suggestions on the rewrite of this section.
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 28, 2005
From: Asser Femø <asser@xxxxxxx>
Just stumbled over this one as I was making a list of loose ends: I haven't been able to find a cifs server that actually uses file_notify_information in a NT_NOTIFY response. Samba seems to use STATUS_ENUM_DIR and Windows XP uses STATUS_SUCCESS. Do anybody know of an implementation that uses the file_notify_information structs? -Asser
pgpHVKNyFjrDd.pgp
Description: PGP signature
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 26, 2005
From: Steven French <sfrench@xxxxxxxxxx>
In-reply-to:
<20050826214856.GD9787@jeremy2
>
References:
<20050826214856.GD9787@jeremy2
>
You can't use continue from last in any case since the client frequently has to back up as the kernel filldir routine can not take arbitrary numbers of entries and hits its max at different boundaries than the cifs findfirst boundary.
There was of course a cifs search bug that was fixed in 1.35 with moderately long file names (you could see it if the resume name were more than about 200 bytes, and there more than one SMB full of resonse data) that you do want to pick up.
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
Jeremy Allison <jra@xxxxxxxxx>
08/26/2005 04:48 PM
|
|
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 26, 2005
From: Jeremy Allison <jra@xxxxxxxxx>
In-reply-to:
<4E022DDAB8F45741914ACD6EDFE2309B0143A21D@xxxxxxxxxxxxxxxxxxxx>
References:
<4E022DDAB8F45741914ACD6EDFE2309B0143A21D@xxxxxxxxxxxxxxxxxxxx>
On Fri, Aug 26, 2005 at 02:25:00PM -0700, Naidu Bollineni wrote: > Is there an issue with FindNext2 call? > > It doesn't add the bit CIFS_SEARCH_CONTINUE_FROM_LAST to the searchFlags > value, and thus, terminating prematurely in at least one of my tests. > > If I add it, then it lists all the files in the directory. I see this > issue exist in 1.34 release too. > > Smbfs does have this bit in it's call on the wire. You can't depend on the SEARCH_CONTINUE_FROM_LAST being there. You need to look at resume keys/resume names etc. Jeremy.
Date: August 26, 2005
From: "Naidu Bollineni" <naidu@xxxxxxxxxx>
|
Is there an issue with FindNext2 call? It doesn’t add the bit CIFS_SEARCH_CONTINUE_FROM_LAST
to the searchFlags value, and thus, terminating prematurely in at least one of
my tests. If I add it, then it lists all the files
in the directory. I see this issue exist in 1.34 release too. Smbfs does have this bit in it’s
call on the wire. From: linux-cifs-client-bounces+naidu=kazeon.com@xxxxxxxxxxxxxxx
[mailto:linux-cifs-client-bounces+naidu=kazeon.com@xxxxxxxxxxxxxxx] On Behalf Of Steven French What is
the kernel version? |
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 26, 2005
From: Steve French <smfltc@xxxxxxxxxx>
In-reply-to:
<1125060807.9256.4.camel@xxxxxxxxxxxxxxxxxxxxxxx>
References:
<1124937768.9705.11.camel@xxxxxxxxxxxxxxxxxxxxxxx> <430E8913.20105@xxxxxxxxxxxxx> <1125060807.9256.4.camel@xxxxxxxxxxxxxxxxxxxxxxx>
[CIFS] Fix oops in fs/locks.c on close of file with pending locks
The recent change to locks_remove_flock code in fs/locks.c changes how
byte range locks are removed from closing files, which shows up a bug in
cifs. The assumption in the cifs code was that the close call sent to
the server would remove any pending locks on the server on this file,
but that is no longer safe as the fs/locks.c code on the client wants
unlock of 0 to PATH_MAX to remove all locks (at least from this client,
it is not possible AFAIK to remove all locks from other clients made to
the server copy of the file). Note that cifs locks are different from
posix locks - and it is not possible to map posix locks perfectly on the
wire yet, due to restrictions of the cifs network protocol, even to
Samba without adding a new request type to the network protocol (which
we plan to do for Samba 3.0.21 within a few months), but the local
client will have the correct, posix view, of the lock in most cases.
The correct fix for cifs for this would involve a bigger change than I
would like to do this late in the 2.6.13-rc cycle - and would involve
cifs keeping track of all unmerged (uncoalesced) byte range locks for
each remote inode and scanning that list to remove locks that intersect
or fall wholly within the range - locks that intersect may have to be
reaquired with the smaller, remaining range.
Signed-off-by: Steve French <sfrench@xxxxxxxxxx>
Signed-off-by: Dave Kleikamp <shaggy@xxxxxxxxxxxxxx>
diff -Naur linux-2.6.13-rc7/fs/cifs/file.c linux/fs/cifs/file.c
--- linux-2.6.13-rc7/fs/cifs/file.c 2005-06-17 14:48:29.000000000 -0500
+++ linux/fs/cifs/file.c 2005-08-26 07:40:37.000000000 -0500
@@ -643,7 +643,7 @@
netfid, length,
pfLock->fl_start, numUnlock, numLock, lockType,
wait_flag);
- if (rc == 0 && (pfLock->fl_flags & FL_POSIX))
+ if (pfLock->fl_flags & FL_POSIX)
posix_lock_file_wait(file, pfLock);
FreeXid(xid);
return rc;
Date: August 26, 2005
From: Dave Kleikamp <shaggy@xxxxxxxxxxxxxx>
In-reply-to:
<430E8913.20105@xxxxxxxxxxxxx>
References:
<1124937768.9705.11.camel@xxxxxxxxxxxxxxxxxxxxxxx> <430E8913.20105@xxxxxxxxxxxxx>
The patch didn't apply without ignoring whitespace. Here it is again
with tabs intact.
--------------------- Cut Here ----------------------------------------
The recent change to locks_remove_flock code in fs/locks.c changes how
byte range locks are removed from closing files, which shows up a bug in
cifs. The assumption in the cifs code was that the close call sent to
the server would remove any pending locks on the server on this file,
but that is no longer safe as the fs/locks.c code on the client wants
unlock of 0 to PATH_MAX to remove all locks (at least from this client,
it is not possible AFAIK to remove all locks from other clients made to
the server copy of the file). Note that cifs locks are different from
posix locks - and it is not possible to map posix locks perfectly on the
wire yet, due to restrictions of the cifs network protocol, even to
Samba without adding a new request type to the network protocol (which
we plan to do for Samba 3.0.21 within a few months), but the local
client will have the correct, posix view, of the lock in most cases.
The correct fix for cifs for this would involve a bigger change than I
would like to do this late in the 2.6.13-rc cycle - and would involve
cifs keeping track of all unmerged (uncoalesced) byte range locks for
each remote inode and scanning that list to remove locks that intersect
or fall wholly within the range - locks that intersect may have to be
reaquired with the smaller, remaining range.
The immediate need though is for the following fix to get into 2.6.13 to
at least avoid the oops in the vfs.
[CIFS] Fix oops in fs/locks.c on close of file with pending locks
Signed-off-by: Steve French <sfrench@xxxxxxxxxx>
Signed-off-by: Dave Kleikamp <shaggy@xxxxxxxxxxxxxx>
diff -Naur linux-2.6.13-rc7/fs/cifs/file.c linux/fs/cifs/file.c
--- linux-2.6.13-rc7/fs/cifs/file.c 2005-06-17 14:48:29.000000000 -0500
+++ linux/fs/cifs/file.c 2005-08-26 07:40:37.000000000 -0500
@@ -643,7 +643,7 @@
netfid, length,
pfLock->fl_start, numUnlock, numLock, lockType,
wait_flag);
- if (rc == 0 && (pfLock->fl_flags & FL_POSIX))
+ if (pfLock->fl_flags & FL_POSIX)
posix_lock_file_wait(file, pfLock);
FreeXid(xid);
return rc;
Date: August 26, 2005
From: Steve French <smfrench@xxxxxxxxxxxxx>
In-reply-to:
<1124937768.9705.11.camel@xxxxxxxxxxxxxxxxxxxxxxx>
References:
<1124937768.9705.11.camel@xxxxxxxxxxxxxxxxxxxxxxx>
The immediate need though is for the following fix to get into 2.6.13 to at least avoid the oops in the vfs.
[CIFS] Fix oops in fs/locks.c on close of file with pending locks
Signed-off-by: Steve French <sfrench@xxxxxxxxxx>
diff -Naur old/fs/file.c new/fs/file.c
--- old/fs/cifs/file.c 2005-08-25 21:53:47.000000000 -0500
+++ new/fs/cifs/file.c 2005-08-25 21:54:56.000000000 -0500
@@ -643,7 +643,7 @@
netfid, length,
pfLock->fl_start, numUnlock, numLock, lockType,
wait_flag);
- if (rc == 0 && (pfLock->fl_flags & FL_POSIX))
+ if (pfLock->fl_flags & FL_POSIX)
posix_lock_file_wait(file, pfLock);
FreeXid(xid);
return rc;
The original problem report follows. Thanks to Shaggy for the initial
analysis.
Dave Kleikamp wrote:
Running the connectathon lock tests, I hit this BUG: [ 3094.124950] ------------[ cut here ]------------ [ 3094.124959] kernel BUG at fs/locks.c:1920! [ 3094.124962] invalid operand: 0000 [#1][ 3094.124964] PREEMPT [ 3094.124966] Modules linked in: cifs ipt_TCPMSS iptable_filter ip_tables blowfish sha256 dummy radeon irda crc_ccitt airo e1000 pcmcia yenta_socket rsrc_nonstatic pcmcia_core ntfs jfs[ 3094.124981] CPU: 0 [ 3094.124982] EIP: 0060:[<c017630e>] Not tainted VLI[ 3094.124984] EFLAGS: 00010246 (2.6.13-rc7) [ 3094.124993] EIP is at locks_remove_flock+0x7e/0x140[ 3094.124997] eax: dc925b74 ebx: c66159f4 ecx: 00000001 edx: 00000001 [ 3094.125001] esi: c6615a8c edi: c66159f4 ebp: c50ffec0 esp: d0c27e78 [ 3094.125004] ds: 007b es: 007b ss: 0068 [ 3094.125008] Process tlocklfs (pid: 12264, threadinfo=d0c27000 task=c6b03570)[ 3094.125010] Stack: cb210ec0 d0c27e9c 00000000 10c27000 00000001 00000000 00000000 00000000 [ 3094.125017] 80000000 00000023 cb210ec0 d0c27f1c 00000000 d69cc3c0 e1d96b1a e1d911ca [ 3094.125025] 00fe08bf d69cc3c0 00001f2f 00000000 80000000 00000000 00000000 00000001 [ 3094.125032] Call Trace:[ 3094.125038] [<e1d96b1a>] _FreeXid+0x1a/0x30 [cifs] [ 3094.125058] [<e1d911ca>] cifs_lock+0x17a/0x530 [cifs] [ 3094.125074] [<c0176281>] locks_remove_posix+0x131/0x140 [ 3094.125080] [<c0188740>] inotify_dentry_parent_queue_event+0xa0/0xd0 [ 3094.125089] [<c015d967>] __fput+0xa7/0x200 [ 3094.125098] [<c015bd2d>] filp_close+0x4d/0x80 [ 3094.125103] [<c015bdcb>] sys_close+0x6b/0xa0 [ 3094.125108] [<c0103345>] syscall_call+0x7/0xb[ 3094.125115] Code: 74 1b 89 c6 8b 06 85 c0 75 f3 e8 3e 39 33 00 81 c4 cc 00 00 00 5b 5e 5f 5d c3 8d 76 00 0f b6 50 28 f6 c2 02 75 22 f6 c2 20 75 0a <0f> 0b 80 07 2c f7 4d c0 eb cd 89 34 24 bf 02 00 00 00 89 7c 24 [ 3094.125147]I believe it is caused by this patch (stale POSIX lock handling): http://www.kernel.org/git/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=c293621bbf678a3d85e3ed721c3921c8a670610d The bit responsible is: @@ -1888,12 +1908,7 @@ void locks_remove_flock(struct file *fil while ((fl = *before) != NULL) { if (fl->fl_file == filp) { - /* - * We might have a POSIX lock that was created at the same time - * the filp was closed for the last time. Just remove that too, - * regardless of ownership, since nobody can own it. - */ - if (IS_FLOCK(fl) || IS_POSIX(fl)) { + if (IS_FLOCK(fl)) { locks_delete_lock(before); continue; } Leaving this: if (fl->fl_file == filp) { if (IS_FLOCK(fl)) { locks_delete_lock(before); continue; } if (IS_LEASE(fl)) { lease_modify(before, F_UNLCK); continue; } /* What? */ BUG(); }
Date: August 25, 2005
From: Steven French <sfrench@xxxxxxxxxx>
In-reply-to:
<4E022DDAB8F45741914ACD6EDFE2309B013CAFA6@xxxxxxxxxxxxxxxxxxxx>
References:
<4E022DDAB8F45741914ACD6EDFE2309B013CAFA6@xxxxxxxxxxxxxxxxxxxx>
OK - will take a look.
I am fairly certain that I have done this (mounting multiple users from same client ip address to windows 2003) many times earlier in the year and it was fixed in cifs 1.32.
Are you sure you needed this (your fix) when signing was off?
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 25, 2005
From: Steven French <sfrench@xxxxxxxxxx>
In-reply-to:
<430E25EE.5090306@xxxxxxxxxx>
References:
<430E25EE.5090306@xxxxxxxxxx>
The CIFS behavior (similar to NTFS/OS2/DOS and a few other OS in this regard) is to release each of the active locks
- either explicitly (by the application explicitly unlocking the same range it previously locked)
or
- if the file closes before the app releases the lock, the close file cifs network operation will release the locks on the server.
Of course, other things can cause the server to throw away all locks - e.g. killing the tcp session to the server will cause the
corresponding server process to release all state - includuing releasing the server side of the locks.
On the client, I should probably simply keep the list of all byte range locks (uncoalesced ie unmerged) and walk them manually if the server gives an error on the
unlock as I expect happened in your case - and do what posix would have done if the server were posix (ie lock all locks intersecting that range, and relock the edges as appropriate).
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
Peter Staubach <staubach@xxxxxxxxxx>
08/25/2005 03:11 PM |
|
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 25, 2005
From: Peter Staubach <staubach@xxxxxxxxxx>
In-reply-to:
<OFCC7770EF.1D9A3B21-ON87257068.006C6750-86257068.006D3119@xxxxxxxxxx>
References:
<OFCC7770EF.1D9A3B21-ON87257068.006C6750-86257068.006D3119@xxxxxxxxxx>
Steven French wrote:
Seems to me that the problem is that since the VFS is not requesting an unlock of the locked range (rather a total file unlock from 0 to MAX_OFFSET), the call on the server to unlock (via cifs) will fail so that will cause the posix_lock_file_wait call to be skipped on the last unlock - so that means that the remove_flock code will still see flocks hanging off the inode (since the unlock in effect failed).This is tricky to handle in CIFS since these really aren't posix locks but they are somewhat close ... and like NFS, some apps will require a "nolock" (or as cifs calls it "nobrl") mount option to skip all the remote locking entirely in order to work. We should not oops of courseAt a minimum removing the "if rc==0" check on about line 646 of fs/cifs/file.c is needed so the posix unlock call will actually occur on the next line even if the server has no idea what lock we are trying to unlock (the server/network protocol does not recognize 0 to MAX_OFFSET which is presumably being done here as "unlock all locks" - there may be a remote network ioctl or fcntl I can use that does this at least for the case of NTFS - I will have to poke on that more - but the close on the server will release the locks in any case so it is not as critical as it sounds for the server to understand the "unlock all" concept). For the special case of 0 to MAX_OFFSET - I probably need to find code to walk all byte range locks and unlock them all (remember cifs servers do not coalesce locks so I will need to keep each of the underlying locks - which the local kernel may have merged)
Yes, the intention of the intention of the code in locks_remove_posix()
is to release and free all locks held by the specified owner.
So, does this imply that in the past, there were locks outstanding on the
server, unless the process itself, had released all of its locks manually?
Thanx...
ps
Date: August 25, 2005
From: "Naidu Bollineni" <naidu@xxxxxxxxxx>
|
What we observed is that Windows 2003
server did not let multiple user sessions over the same TCP connection. So,
when I changed a line in cifs_mount that does “if (existingSrvTcp)”
to “if (existingCifsSes)”, it allowed it to go further. But, when
the second session request gets the response, there was TCP Reset coming for
the first TCP connection immediately behind the second SessResponse. So, when I fixed the VCnumber to be
different, all things were normal. Since the VCnumber being just a 16bit
quantity, I thought I could add a bitmap logic like how Max_anon does in fs/super.c,
so any wrap around won’t stomp on existing session’s VCnumber. Here is the support code: static u32
VCs_used[Max_VCs/(8*sizeof(u32))]; spinlock_t cifs_global_vclock =
SPIN_LOCK_UNLOCKED; void cifs_release_vc(struct cifsSesInfo *ses) {
spin_lock(&cifs_global_vclock);
clear_bit(ses->VCnum, VCs_used);
spin_unlock(&cifs_global_vclock); } Here is the code change in CIFSSessSetup()
:
pSMB->req_no_secext.AndXCommand = 0xFF;
pSMB->req_no_secext.SessionKey = *(u32 *)ses->server->sessid;
spin_lock(&cifs_global_vclock);
VCnum = find_first_zero_bit(VCs_used, Max_VCs);
if (VCnum == Max_VCs) {
spin_unlock(&cifs_global_vclock);
cifs_buf_release(smb_buffer);
printk("No empty Session Slot to initiate session request\n");
return -EAGAIN;
} set_bit(VCnum,
VCs_used);
spin_unlock(&cifs_global_vclock);
ses->VCnum = VCnum;
pSMB->req_no_secext.VcNumber = VCnum; And to release the busy bit in case of
errors in CifsSessSetup(), at the end:
if (rc) {
cifs_release_vc(ses);
}
return rc; In normal unmount cases, sesInfoFree()
should call cifs_release_vc(). So, the changes in misc.c: extern void cifs_release_vc(struct
cifsSesInfo *ses);
atomic_dec(&sesInfoAllocCount);
cifs_release_vc(buf_to_free); <<<<<<<<<ß-----------------------------
added line.
if (buf_to_free->serverOS)
kfree(buf_to_free->serverOS); From: Steven French
[mailto:sfrench@xxxxxxxxxx] Yes - I will take a look at that.
From:
linux-cifs-client-bounces+naidu=kazeon.com@xxxxxxxxxxxxxxx [mailto:linux-cifs-client-bounces+naidu=kazeon.com@xxxxxxxxxxxxxxx]
On Behalf Of Steven French I should
have a version of the cifs client that can mount and do the most important
operations (open/read/write/close) to Windows 9x/ME servers available for
testing before Friday. I am just finishing up legacy getattr support. |
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 25, 2005
From: Steven French <sfrench@xxxxxxxxxx>
Seems to me that the problem is that since the VFS is not requesting an unlock of the locked range (rather a total file unlock from 0 to MAX_OFFSET), the call on the server to unlock (via cifs) will fail so that will cause the posix_lock_file_wait call to be skipped on the last unlock - so that means that the remove_flock code will still see flocks hanging off the inode (since the unlock in effect failed).
This is tricky to handle in CIFS since these really aren't posix locks but they are somewhat close ... and like NFS, some apps will require a "nolock" (or as cifs calls it "nobrl") mount option to skip all the remote locking entirely in order to work. We should not oops of course
At a minimum removing the "if rc==0" check on about line 646 of fs/cifs/file.c is needed so the posix unlock call will actually occur on the next line even if the server has no idea what lock we are trying to unlock (the server/network protocol does not recognize 0 to MAX_OFFSET which is presumably being done here as "unlock all locks" - there may be a remote network ioctl or fcntl I can use that does this at least for the case of NTFS - I will have to poke on that more - but the close on the server will release the locks in any case so it is not as critical as it sounds for the server to understand the "unlock all" concept). For the special case of 0 to MAX_OFFSET - I probably need to find code to walk all byte range locks and unlock them all (remember cifs servers do not coalesce locks so I will need to keep each of the underlying locks - which the local kernel may have merged)
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 25, 2005
From: Ian Campbell <ijc@xxxxxxxxxxxxxx>
In-reply-to:
<20050825145750.GA6658@xxxxxxxxxxxxxxxxxx>
References:
<20050823130023.GB8305@xxxxxxx> <20050823152331.GA10486@xxxxxxxxxxxxxxxxxx> <1124973618.17190.9.camel@icampbell-debian
> <20050825145750.GA6658@xxxxxxxxxxxxxxxxxx>
"Naidu
Bollineni" <naidu@xxxxxxxxxx>
On Thu, 2005-08-25 at 15:57 +0100, Jamie Lokier wrote:
> Ian Campbell wrote:
> > On Tue, 2005-08-23 at 16:23 +0100, Jamie Lokier wrote:
> > > <receive some request>...
> > > if (any_dnotify_or_inotify_events_pending) {
> > > read_dnotify_or_inotify_events();
> > > if (any_events_related_to(file)) {
> > > store_in_userspace_stat_cache(file, stat(file));
> > > }
> > > }
> > > stat_info = lookup_userspace_stat_cache(file);
> > >
> > > Now that's a silly way to save one system call in the fast path by itself.
> >
> > I'm not that familiar with inotify internals but doesn't
> > read_dnotify_or_inotify_events() or
> > any_dnotify_or_inotify_events_pending() involve a syscall?
>
> The fast path is just any_dnotify_or_inotify_events_pending: there
> aren't any relevant events pending in the fast path.
[snip]
> As I explained in the previous mail, all this is absolutely pointless
> to save one system call. It's a lot of work for negligable gain.
>
> The point is when it saves lots of calls and userspace logic together,
> for things like web page templates and compiled programs, which depend
> on many files which can be revalidated in a small number of operations.
Thanks for the explaination.
Ian.
--
Ian Campbell
Current Noise: Iron Maiden - Prodigal Son
Fay: The British police force used to be run by men of integrity.
Truscott: That is a mistake which has been rectified.
-- Joe Orton, "Loot"
Date: August 25, 2005
From: Jamie Lokier <jamie@xxxxxxxxxxxxx>
In-reply-to:
<1124973618.17190.9.camel@icampbell-debian
>
References:
<20050823130023.GB8305@xxxxxxx> <20050823152331.GA10486@xxxxxxxxxxxxxxxxxx> <1124973618.17190.9.camel@icampbell-debian
>
Ian Campbell wrote:
> On Tue, 2005-08-23 at 16:23 +0100, Jamie Lokier wrote:
> > <receive some request>...
> > if (any_dnotify_or_inotify_events_pending) {
> > read_dnotify_or_inotify_events();
> > if (any_events_related_to(file)) {
> > store_in_userspace_stat_cache(file, stat(file));
> > }
> > }
> > stat_info = lookup_userspace_stat_cache(file);
> >
> > Now that's a silly way to save one system call in the fast path by itself.
>
> I'm not that familiar with inotify internals but doesn't
> read_dnotify_or_inotify_events() or
> any_dnotify_or_inotify_events_pending() involve a syscall?
The fast path is just any_dnotify_or_inotify_events_pending: there
aren't any relevant events pending in the fast path.
There's a few methods of doing this for free per individual stat cache check.
1. Signal handler.
dnotify: Check a variable set by the signal handler.
[ There's a small time window between dnotify sending the
signal and the receiving thread noticing on an SMP system due
to the IPI, during which the sending task might have another
way to signal the recieving task that it's finished some
operation, so this method of using dnotify to invalidate a
stat cache only has the correct ordering properties on UP systems. ]
This works because dnotify signals are thread-specific,
so the checking thread will definitely have received the
signal after the time another process modifies the file.
inotify: Disappointingly inotify doesn't support SIGIO readiness :(
2. If you have to mix the test with a poll/select/epoll/rtsig fd waiting
for some other purpose. For example: a file/web/local server, where the
constraint is only that each stat() to revalidate a cached response
appears to happen any time after the beginning of receiving the
network request is known.
dnotify: It's free if you were using sigtimedwait anyway for I/O events,
provided you completely read the queue, or get the signal
priority right.
inotify: It's free if you were using poll/select/epoll anyway for I/O
events, provided in the case of epoll that you completely
read the queue, or use a two-level queue.
3. Amortising the test over many stat cache checks.
Even if you must use a system call to check for any pending events,
for revalidating an object which depends on multiple files, only
one call is needed for all of the stat cache checks.
More generally (this is more flexible), you can separate the notion
of "cache time checkpoint" from "cache validation". It's enough to
know that a stat result was valid any time between the checkpoint
time, and the current time. That's how I'm implementing the
file/web/local server case described above in step 2. Then the
events only need to be checked once during that time interval, no
matter how many complex objects are being revalidated.
It gets slightly more efficient when you have multiple, overlapping
checkpoint->validation time intervals due to multiple outstanding
requests being processed concurrently.
As I explained in the previous mail, all this is absolutely pointless
to save one system call. It's a lot of work for negligable gain.
The point is when it saves lots of calls and userspace logic together,
for things like web page templates and compiled programs, which depend
on many files which can be revalidated in a small number of operations.
-- Jamie
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Date: August 25, 2005
From: Ian Campbell <ijc@xxxxxxxxxxxxxx>
In-reply-to:
<20050823152331.GA10486@xxxxxxxxxxxxxxxxxx>
References:
<20050823130023.GB8305@xxxxxxx> <20050823152331.GA10486@xxxxxxxxxxxxxxxxxx>
On Tue, 2005-08-23 at 16:23 +0100, Jamie Lokier wrote:
> <receive some request>...
> if (any_dnotify_or_inotify_events_pending) {
> read_dnotify_or_inotify_events();
> if (any_events_related_to(file)) {
> store_in_userspace_stat_cache(file, stat(file));
> }
> }
> stat_info = lookup_userspace_stat_cache(file);
>
> Now that's a silly way to save one system call in the fast path by itself.
I'm not that familiar with inotify internals but doesn't
read_dnotify_or_inotify_events() or
any_dnotify_or_inotify_events_pending() involve a syscall?
Ian.
--
Ian Campbell
Current Noise: Primordial - The Coffin Ships
Today is the first day of the rest of the mess.
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Date: August 24, 2005
From: Steven French <sfrench@xxxxxxxxxx>
In-reply-to:
<BBBDF0DA-0132-49A1-B2D2-61F7D488FBD3@xxxxxxxxxxxxxx>
References:
<BBBDF0DA-0132-49A1-B2D2-61F7D488FBD3@xxxxxxxxxxxxxx>
My opinion is that a maintainer for smbfs should not be a high priority,
but any hot smb bugs (e.g. oops or hang in kernel) among the bugs listed in
samba or kernel bugzillas could be analyzed/fixed as people have time or
interest, and I would be happy to review them.
My reasoning why smbfs is becoming a lower priority:
1) There are only two remaining smbfs capabilities which cifs does not have
equivalent support for and these are coming soon:
a) CIFS will have some Windows ME support within a week (Windows 95,
Windows 98 will also likely work
but I have not tested them - I will need help from those that still
have older boxes or vmware images)
b) Kerberos support - CIFS will have Kerberos/SPNEGO support (via
upcall to ntlm_auth) by the 4th quarter.
2) cifs is more functional, more posix compliant in most or all cases, and
much better supports newer Samba and Windows servers (its initial target)
3) We do not have enough filesystem or Samba developers (especially with
Samba 3 maitaining and Samba 4 new development going on at the same time)
to afford to split the testing and development work for the kernel piece
much
4) We are working on formalizing the documentation for the "CIFS POSIX
Extensions" (with the help of Jeremy Allison and others, and feedback from
others outside the Samba team as well) which have been implemented in cifs
(smbfs partially implemented the older Unix extensions).
We (Samba team and kernel filesystem community) do need help though...
I would be glad to help anyone get started with working on GUIs, new tools,
new loosely related kernel libraries, or items off the cifs vfs list. We
have a couple people working cifs items at the moment (that I know of -
DNOTIFY support through the summer of code, and another doing
cifs_writepages work), and there have been others (I would love to James
Roper rework his promising cifs async readpages readahead code).
1) A maintainer would need some system programming experience (e.g. kernel
or device driver work or willingness to dive in and learn e.g. from the
O'Reilly Device Driver book), and prove themselves by demonstrating some
accepted kernel patches.
2) Initial few months of their patches for the subsystem should be
carefully reviewed (for smbfs by me or the old maintainer and it would not
hurt to have linux-fsdevel and samba-technical reviews)
The easiest path might be to take an item from the TODO list of one of the
filesystems and implement it, carving a recognized area that can standalone
to some extent (so patches collide less often with other work).
For Samba kernel work there is an obvious need to extend (or perhaps
reimplement something similar) Tim Potter's Samba LSM module for example,
and the cifs vfs has about 10 items on the todo list. Across multiple
filesystems is one of the more interesting problems. - ACL mapping (NFSv4
to CIFS, CIFS to NFSv4, POSIX to CIFS, NFSv4 to POSIX, CIFS to unix mode,
unix mode bits to NFSv4 etc.) - an obvious case for a library in kernel.
Another hot recent area is credential caching/management in kernel for cifs
and nfsv4 (and ecryptfs etc.) for which a new mailing list was just setup
by Trond (hosted on the linux-nfs server).
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
Hal Wigoda
<hwigoda@mindspri
ng.com> To
Sent by: linux-kernel@xxxxxxxxxxxxxxx,
linux-cifs-client samba-technical@xxxxxxxxxxxxxxx,
-bounces+sfrench= linux-cifs-client@xxxxxxxxxxxxxxx
us.ibm.com@lists. cc
samba.org
Subject
[linux-cifs-client] Re: New
08/23/2005 11:19 maintainer needed for the Linux smb
PM filesystem
is a maintainer still needed?
and if so, what are the qualifications?
hal wigoda
chicago
_______________________________________________
linux-cifs-client mailing list
linux-cifs-client@xxxxxxxxxxxxxxx
https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 24, 2005
From: Steven French <sfrench@xxxxxxxxxx>
In-reply-to:
<4E022DDAB8F45741914ACD6EDFE2309B013CAC0E@xxxxxxxxxxxxxxxxxxxx>
References:
<4E022DDAB8F45741914ACD6EDFE2309B013CAC0E@xxxxxxxxxxxxxxxxxxxx>
Yes - I will take a look at that.
I would not expect vcnumber to need to be unique unless there were different tcp sessions from the same client to the same server ... are you sure vcnumber was the issue - you should be able to have two smb uids over one session.
We did have a signing problem with Windows 2003 with multiple users mounting at the same time (Windows 2003 requires signing when a member of a domain, by default, so this shows up most often with Windows 2003) - that has long been fixed though. Are you sure that you have that fix?
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
"Naidu Bollineni" <naidu@xxxxxxxxxx>
08/24/2005 12:35 AM |
|
I should have a version of the cifs client that can mount and do the most important operations (open/read/write/close) to Windows 9x/ME servers available for testing before Friday. I am just finishing up legacy getattr support.
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 24, 2005
From: "Naidu Bollineni" <naidu@xxxxxxxxxx>
|
Hi Steve, Can you take care of the VcNumber and
SessionKey fields to be filled properly in SessionSetup call? I had issues with ntap, and Win2K3 boxes
failing attempts by multiple users connecting to the same server from same
client simultaneously. When I fixed the VcNumber field with a global variable
having monotonically increasing value, it worked fine against both servers. Thanks, From:
linux-cifs-client-bounces+naidu=kazeon.com@xxxxxxxxxxxxxxx
[mailto:linux-cifs-client-bounces+naidu=kazeon.com@xxxxxxxxxxxxxxx] On Behalf Of Steven French I should
have a version of the cifs client that can mount and do the most important
operations (open/read/write/close) to Windows 9x/ME servers available for
testing before Friday. I am just finishing up legacy getattr support. |
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 24, 2005
From: Hal Wigoda <hwigoda@xxxxxxxxxxxxxx>
In-reply-to:
<430BEBEA.60704@xxxxxxxxxxxxx>
References:
<430BEBEA.60704@xxxxxxxxxxxxx>
is a maintainer still needed? and if so, what are the qualifications? hal wigoda chicago
Date: August 24, 2005
From: Steve French <smfrench@xxxxxxxxxxxxx>
CIFS (in the cifs.git tree) can now handle not just mounts to Windows ME (and probably Windows 9x), but readdir and enough of lookup. Finishing up the remainder should go fast (OpenX instead of NTCreateX is the main piece left).
Of course finding Windows 95, Windows 98, and OS/2 servers is a little harder than it sounds...although scripting a subset of the functional tests that should work should be pretty easy.
I will also put a version of the source that will compile at least as far back as 2.6.9 up on the project page within a few days.
Date: August 23, 2005
From: cifsbal.8.mtf022004@xxxxxxxxxxxx
> When I smbmount a Samba share on a station running Ubuntu 5.04 (Samba
> 3.0.14a) symbolic links located on the share are not followed (they
> appear as symbolic links).
Thank you for your answer Steven.
I also began to think about the kernel. Mine is 2.6.10.
But I've just found the solution: with "newer" smbmount you have to add:
unix extensions = no
in the [global] section of smb.conf.
Martin
Date: August 23, 2005
From: Steven French <sfrench@xxxxxxxxxx>
I should have a version of the cifs client that can mount and do the most important operations (open/read/write/close) to Windows 9x/ME servers available for testing before Friday. I am just finishing up legacy getattr support.
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 23, 2005
From: Steven French <sfrench@xxxxxxxxxx>
In-reply-to:
<430B7A33.3090808@xxxxxxxxxxx>
References:
<430B7A33.3090808@xxxxxxxxxxx>
What is the kernel version?
If 2.6.9 or later kernel have you tried using cifs vfs instead?
Steve French
Senior Software Engineer
Linux Technology Center - IBM Austin
phone: 512-838-2294
email: sfrench at-sign us dot ibm dot com
_______________________________________________ linux-cifs-client mailing list linux-cifs-client@xxxxxxxxxxxxxxx https://lists.samba.org/mailman/listinfo/linux-cifs-client
Date: August 23, 2005
From: cifsbal.8.mtf022004@xxxxxxxxxxxx
Hello, When I smbmount a Samba share on a station running Ubuntu 5.04 (Samba 3.0.14a) symbolic links located on the share are not followed (they appear as symbolic links). smbmount on Mandrake 7.2 and 9.2 stations work normally. smbclient works normally, with symbolic links resolution. The Samba client in Nautilus works normally. Windows clients follow the symbolic links normally. The Samba server is version 3.0.14a runnning under Mandrake 9.2. I browsed the web to no avail. I tried to compile and install various Samba versions on my Ubuntu 5.04 station to no avail (Samba 3.0.14a, Samba 2.2.12). Thank you for your help! Sincerely, Martin
Date: August 23, 2005
From: cifsbal.8.mtf022004@xxxxxxxxxxxx
Hello, When I smbmount a Samba share on a station running Ubuntu 5.04 (Samba 3.0.14a) symbolic links located on the share are not followed (they appear as symbolic links). smbmount on Mandrake 7.2 and 9.2 stations work normally. smbclient works normally, with symbolic links resolution. The Samba client in Nautilus works normally. Windows clients follow the symbolic links normally. The Samba server is version 3.0.14a runnning under Mandrake 9.2. I browsed the web to no avail. I tried to compile and install various Samba versions on my Ubuntu 5.04 station to no avail (Samba 3.0.14a, Samba 2.2.12). Thank you for your help! Sincerely, Martin
Date: August 23, 2005
From: cifsbal.8.mtf022004@xxxxxxxxxxxx
Hello, When I smbmount a Samba share on a station running Ubuntu 5.04 (Samba 3.0.14a) symbolic links located on the share are not followed (they appear as symbolic links). smbmount on Mandrake 7.2 and 9.2 stations work normally. smbclient works normally, with symbolic links resolution. The Samba client in Nautilus works normally. Windows clients follow the symbolic links normally. The Samba server is version 3.0.14a runnning under Mandrake 9.2. I browsed the web to no avail. I tried to compile and install various Samba versions on my Ubuntu 5.04 station to no avail (Samba 3.0.14a, Samba 2.2.12). Thank you for your help! Sincerely, Martin
Date: August 23, 2005
From: Jamie Lokier <jamie@xxxxxxxxxxxxx>
In-reply-to:
<20050823130023.GB8305@xxxxxxx>
References:
<20050823130023.GB8305@xxxxxxx>
Asser Femø wrote:
> According to the fcntl manual you can cancel a notification by doing
> fcntl(fd, F_NOTIFY, 0) (ie. sending 0 as the notification mask), but
> looking in the kernel code fcntl_dirnotify() immediately calls
> dnotify_flush() with neither telling the vfs module about it. Is there a
> reason for this? Otherwise I'd propose calling
> filp->f_op->dir_notify(filp, 0) at some point in this scenario.
>
> Regarding inotify, inotify_add_watch doesn't seem to pass on the request
> either, which works fine for local filesystem operations as they call
> fsnotify_* functions every time, but that isn't really feasible for
> filesystems like cifs because we'd have to request change notification
> on everything. Is there plans for implementing a mechanism to let vfs
> modules get watch requests too?
On a related note:
dnotify and inotify on local filesystems appear to be synchronous, in
the following rather useful sense:
If you have previously registered for inotify/dnotify events that will
catch a change to a file, and called stat() on the file, then the
following operation:
<receive some request>...
stat_info = stat(file)
may be replaced in userspace code with:
<receive some request>...
if (any_dnotify_or_inotify_events_pending) {
read_dnotify_or_inotify_events();
if (any_events_related_to(file)) {
store_in_userspace_stat_cache(file, stat(file));
}
}
stat_info = lookup_userspace_stat_cache(file);
Now that's a silly way to save one system call in the fast path by itself.
But when the stat_info is a prerequisite for validating cached data --
such as the contents of a file parsed into a data structure -- it can
save a lot of system calls and logical work.
For example, an Apache-style path walk which checks for .htaccess, or
a Samba-style path walk which is checking for unsafe symbolic links,
can be reduced from say 20 system calls to zero using this method.
Pre-compiled or pre-parsed programs/scripts/templates/config-files
where all the source files used are prerequisites for invalidating a
cached compiled form, reduces from say 40 system calls to stat() all
the source files, to zero.... that's quite a saving.
It's not just reducing system calls. The logical tests in userspace
are also skipped, if coded properly, facilitating very quick decisions
about things that depend on files which mostly don't change.
(Cascading structured cache prerequisites...mmm).
Remote dnotify/inotify doesn't _necessarily_ have this synchronous
property. It may do in some cases, depending on the implementation
(this is subtle...).
So, it would be nice if there was a way to query this... rather than
the tedious method of testing the filesystem type and having a table
of "known local filesystem types" where it's safe to depend on this
property. Alternatively, a way to specify at dnotify/inotify creation
type that synchronous notifications are required, and have the request
rejected if those