FltCreateFile and top device.

FltCreateFile calls IoCreateFileEx with IO_DRIVER_CREATE_CONTEXT.DeviceObjectHint pointing to the Filter Manager's filter object and then calls the lower registered filters. That allows the created file object to have TopDeviceObjectHint pointing to the Filter Manager's object.

 # Child-SP          RetAddr           Call Site
00 ffffe101`3fffebf8 fffff801`92125200 FLTMGR!FltpCreate
01 ffffe101`3fffec00 fffff801`9213058b nt!IopParseDevice+0x7f0
02 ffffe101`3fffedd0 fffff801`921340c0 nt!ObpLookupObjectName+0x46b
03 ffffe101`3fffefa0 fffff801`9213803a nt!ObOpenObjectByNameEx+0x1e0
04 ffffe101`3ffff0e0 fffff801`920b0eb4 nt!IopCreateFile+0x3aa
05 ffffe101`3ffff180 fffff808`485240d5 nt!IoCreateFileEx+0x124
06 ffffe101`3ffff210 fffff808`4853d32d FLTMGR!FltpCreateFile+0x1cd
07 ffffe101`3ffff310 fffff808`4b6a79f8 FLTMGR!FltCreateFile+0x8d
08 ffffe101`3ffff3a0 fffff808`484f4b4c avscan!AvPreCreate+0x378 [d:\work\avscan\filter\avscan.c @ 2106]
09 ffffe101`3ffff4b0 fffff808`484f46ec FLTMGR!FltpPerformPreCallbacks+0x2ec
0a ffffe101`3ffff5d0 fffff808`48526117 FLTMGR!FltpPassThroughInternal+0x8c
0b ffffe101`3ffff600 fffff801`92125200 FLTMGR!FltpCreate+0x2d7
0c ffffe101`3ffff6b0 fffff801`9213058b nt!IopParseDevice+0x7f0
0d ffffe101`3ffff880 fffff801`921340c0 nt!ObpLookupObjectName+0x46b
0e ffffe101`3ffffa50 fffff801`920c9e90 nt!ObOpenObjectByNameEx+0x1e0

0: kd> dt nt!_FILE_OBJECT ffff948c621a3330
   +0x000 Type             : 0n5
   +0x002 Size             : 0n216
   +0x008 DeviceObject     : 0xffff948c`60a3bc80 _DEVICE_OBJECT
   +0x010 Vpb              : 0xffff948c`60a556e0 _VPB
   +0x018 FsContext        : 0xffff948c`6111a740 Void
   +0x020 FsContext2       : 0xffff8483`76cf78f0 Void
   +0x028 SectionObjectPointer : (null) 
   +0x030 PrivateCacheMap  : (null) 
   +0x038 FinalStatus      : 0n0
   +0x040 RelatedFileObject : (null) 
   +0x048 LockOperation    : 0 ''
   +0x049 DeletePending    : 0 ''
   +0x04a ReadAccess       : 0x1 ''
   +0x04b WriteAccess      : 0 ''
   +0x04c DeleteAccess     : 0 ''
   +0x04d SharedRead       : 0x1 ''
   +0x04e SharedWrite      : 0x1 ''
   +0x04f SharedDelete     : 0x1 ''
   +0x050 Flags            : 0x40000
   +0x058 FileName         : _UNICODE_STRING "\"
   +0x068 CurrentByteOffset : _LARGE_INTEGER 0x0
   +0x070 Waiters          : 0
   +0x074 Busy             : 0
   +0x078 LastLock         : (null) 
   +0x080 Lock             : _KEVENT
   +0x098 Event            : _KEVENT
   +0x0b0 CompletionContext : (null) 
   +0x0b8 IrpListLock      : 0
   +0x0c0 IrpList          : _LIST_ENTRY [ 0xffff948c`621a33f0 - 0xffff948c`621a33f0 ]
   +0x0d0 FileObjectExtension : 0xffff948c`6226f1b0 Void
   
0: kd> dq 0xffff948c`6226f1b0
ffff948c`6226f1b0  00000000`00000000 00000000`00000000
ffff948c`6226f1c0  ffff948c`60dff0d0 00000000`00000000
ffff948c`6226f1d0  ffff948c`6243f2c0 00000000`00000000
ffff948c`6226f1e0  00000000`00000000 00000000`00000000
ffff948c`6226f1f0  00000000`00000000 00000000`00000000
ffff948c`6226f200  61436d4d`02120006 00000000`0000034c
ffff948c`6226f210  ffff8483`7535ed10 ffff948c`62161e28
ffff948c`6226f220  ffff948c`6221da78 00000000`00000000

0: kd> dq ffff948c`60dff0d0
ffff948c`60dff0d0  ffff948c`610734a0 00000000`00000000
ffff948c`60dff0e0  00000000`00000000 00000000`00000000
ffff948c`60dff0f0  65536d4d`02060003 6c8da38a`069a7123
ffff948c`60dff100  00000000`00000000 0000024e`49c8000a
ffff948c`60dff110  0000024e`49c80fff 00000000`00000000
ffff948c`60dff120  00000000`00000000 00000000`00000000
ffff948c`60dff130  00000000`00000000 00000000`00000000
ffff948c`60dff140  00000000`00000002 00000000`00000000

0: kd> !object ffff948c`610734a0
Object: ffff948c610734a0  Type: (ffff948c5e34eb00) Device
    ObjectHeader: ffff948c61073470 (new version)
    HandleCount: 0  PointerCount: 1
    
0: kd> !devstack ffff948c610734a0
  !DevObj           !DrvObj            !DevExt           ObjectName
> ffff948c610734a0  \FileSystem\FltMgr ffff948c610735f0  
  ffff948c61048060  \FileSystem\fastfatffff948c610481b0 
  
0: kd> !vpb 0xffff948c`60a556e0
Vpb at 0xffff948c60a556e0
Flags: 0x1 mounted 
DeviceObject: 0xffff948c61048060
RealDevice:   0xffff948c60a3bc80
RefCount: 8
Volume Label: 

Current process when closing a kernel handle.

If you call PsGetCurrentProcess() in a filter or driver when processing IRP_MJ_CLEANUP for a kernel handle the system process is returned as NtClose() calls KeStackAttachProcess() if the handle belongs to a system process kernel table.

2: kd> !thread ffffc5006e6da080

THREAD ffffc5006e6da080  Cid 1588.05ec  Teb: 00000000002aa000 Win32Thread: 0000000000000000 WAIT: (WrResource) KernelMode Non-Alertable

    ffffc5006be8eb70  SynchronizationEvent

IRP List:

    ffffc5006e2ba140: (0006,04c0) Flags: 00000404  Mdl: 00000000

    ffffc50075b8aae0: (0006,0118) Flags: 00060000  Mdl: 00000000

Not impersonating

DeviceMap                 ffffd58256416bd0

Owning Process            ffffc500733ff080       Image:         XXXXXXXX

Attached Process          ffffc5006b8b66c0       Image:        System

Microsoft Security Essentials content scan callback to the service.

Below is a stack when a MSE file system filter(WdFilter.sys) called a service(MsMpEng.exe) to perform file content scan on file open.


00 nt!KiSwapContext

01 nt!KiSwapThread
02 nt!KiCommitThreadWait
03 nt!KeWaitForMultipleObjects
04 nt!FsRtlCancellableWaitForMultipleObjects
05 FLTMGR!FltSendMessage
06 WdFilter!MpScanFile
07 WdFilter!MpAmPostCreate
08 WdFilter!MpPostCreate
09 FLTMGR!FltpPerformPostCallbacks
0a FLTMGR!FltpPassThroughCompletionWorker
0b FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted
0c FLTMGR!FltpCreate
16 nt!IopParseDevice
17 nt!ObpLookupObjectName
18 nt!ObOpenObjectByNameEx
19 nt!IopCreateFile
1a nt!NtCreateFile
1b nt!KiSystemServiceCopyEnd
1c ntdll!NtCreateFile

In response the service sent an IOCTL to the filter to create a section( i.e. a mapped file) for data scan

0b mup!MupStateMachine
0c mup!MupFsControl
0d FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted
0e FLTMGR!FltPerformSynchronousIo
0f FLTMGR!IssueControlOperation
10 FLTMGR!FltFsControlFile
11 FLTMGR!FltpSetPurgeFailureMode
12 FLTMGR!FltCreateSectionForDataScan
13 WdFilter!MpCreateSection
14 WdFilter!MpMessage
15 FLTMGR!FltpFilterMessage
16 FLTMGR!FltpMsgDispatch
17 FLTMGR!FltpDispatch
21 nt!IopSynchronousServiceTail
22 nt!IopXxxControlFile
23 nt!NtDeviceIoControlFile
24 nt!KiSystemServiceCopyEnd
25 ntdll!NtDeviceIoControlFile

VFS structures have changed in Sierra

After Sierra(10.12) was released I received two tickets for my Mac OS X file system filter project on GitHub MacOSX-FileSystem-Filter . It happened that vnode structure has changed. The offset for v_op field has changed from 0xC8 to 0xD0. Instead of hardcoding a new definition I activated the dynamic inference for structure layouts that has been in the project for a long time but had not been used as structures had not changed for years.

Update: Apple released 10.12 source code on 25 November 2016. To download xnu/Darwin source code follow the link https://opensource.apple.com/release/os-x-1012.html 

Waiting for concurrent page fault completion

An interesting call stack when a thread waits in a page fault for another thread completing paging data from a file

00 nt!KiSwapContext
01 nt!KiSwapThread
02 nt!KiCommitThreadWait
03 nt!KeWaitForSingleObject
04 nt!MiWaitForCollidedFaultComplete
05 nt!MiResolveTransitionFault
06 nt!MiResolveProtoPteFault
07 nt!MiDispatchFault
08 nt!MmAccessFault
09 nt!KiPageFault
0a nt!memcpy
0b nt!CcCopyBytesToUserBuffer
0c nt!CcMapAndCopyFromCache
0d nt!CcCopyReadEx
0e nt!CcCopyRead
0f nt!FsRtlCopyRead
10 ***
11 ***
12 ***
13 nt!NtReadFile
14 nt!KiSystemServiceCopyEnd

FileObjects and SectionObjectPointer in Windows.

Just for the record.

FileObject->SectionObjectPointer is allocated and set by a file system driver but the structure is managed by the Memory Manager (Mm). SectionObjectPointer is shared between all file objects for the same data stream.

FileObject->SectionObjectPointer->DataSectionObject and FileObject->SectionObjectPointer->ImageSectionObject contain address of ControlArea for data and image.

ControlArea deletion is synchronized by ControlArea->WaitingForDeletion and ControlArea->u.Flags.BeingDeleted. WaitingForDeletion points to a structure with notification event and a reference counter.

All functions that might destroy control area take SectionObjectPointer as a parameter. These functions acquire a global lock then check that ControlArea is not NULL. If control area exists ControlArea->u.Flags.BeingDeleted is checked and if it is set a function waits on WaitingForDeletion event with incremented reference counter so the event is deleted when the last waiting thread exit from a waiting state and the reference counter drops to zero. A call to MiCleanSection set SectionObjectPointer->DataSectionObject  and  SectionObjectPointer->ImageSectionObject  to NULL. This call is synchronized with ControlArea->u.Flags.BeingDeleted.

The functions that might delete control area include MmFlushImageSection and CcPurgeCacheSection. That means that it is safe to provide SectionObjectPointer to these functions without synchronizing with file objects deletion. It is even possible to call this functions with a SectionObjectPointer when all related file objects have been deleted or have IopDeleteFile being called for them which might happen in IRP_MJ_PNP processing path.

File mapping and FILE_OBJECT in Windows

There is a WinDBG command !ca that shows file mapping related information. I will show how to get this file mapping information for a file object ( FILE_OBJECT type) by a direct access to structures.

The core of file mapping( and file data caching that uses file mapping ) is SEGMENT object and CONTROL_AREA structures. SEGMENT object contains a pointer to an array of Prototype PTEs ( ProtoPTE ) of _MMPTE_PROTOTYPE type. Each ProtoPTE points to a related physical page if the page is valid. When a file mapping is created the related  virtual memory range PTEs( Page Table Entries ) have the invalid bit set and point to Prototype PTEs. When a corresponding virtual address is accessed a page fault happens, the page fault handler follows a link to ProtoPTE and fixes process PTE to point to a real page. That allows all processes to share the same physical pages for the same file memory mapping. The physical page might need to be allocated and data read in from a file if this has not been done before, after that the page is shared between all processes mapping the file.

FILE_OBJECT has SectionObjectPointer field which is set by a file system driver (FSD) but all its fields are initialized by Memory Manager(CC) and Cache Manager(CC). SectionObjectPointer is of _SECTION_OBJECT_POINTERS type with DataSectionObject field pointing to a CONTROL_AREA structure that in turn points to a SEGMENT object. CONTROL_AREA has a _SUBSECTION structure following it at the tail, all subsequent _SUBSECTION structures are linked by NextSubsection  pointer. Each _SUBSECTION has SubsectionBase field that points to a related ProtoPTEs array.

Below all these structures for a real file object are printed from WinDBG.

0: kd> ??FileObject

struct _FILE_OBJECT * 0x8750ef80

   +0x000 Type             : 0n5

   +0x002 Size             : 0n128

   +0x004 DeviceObject     : 0x879c9030 _DEVICE_OBJECT

   +0x008 Vpb              : 0x879d5888 _VPB

   +0x00c FsContext        : 0x87bdde68 Void

   +0x010 FsContext2       : 0x863cc188 Void

   +0x014 SectionObjectPointer : 0x87bddea8 _SECTION_OBJECT_POINTERS

   +0x018 PrivateCacheMap  : 0x869acf90 Void

   +0x01c FinalStatus      : 0n0

   +0x020 RelatedFileObject : (null) 

   +0x024 LockOperation    : 0 ''

   +0x025 DeletePending    : 0 ''

   +0x026 ReadAccess       : 0 ''

   +0x027 WriteAccess      : 0 ''

   +0x028 DeleteAccess     : 0 ''

   +0x029 SharedRead       : 0 ''

   +0x02a SharedWrite      : 0 ''

   +0x02b SharedDelete     : 0 ''

   +0x02c Flags            : 0xc0012

   +0x030 FileName         : _UNICODE_STRING "\Sample Pictures\Chrysanthemum.jpg"

   +0x038 CurrentByteOffset : _LARGE_INTEGER 0x11000

   +0x040 Waiters          : 0

   +0x044 Busy             : 1

   +0x048 LastLock         : (null) 

   +0x04c Lock             : _KEVENT

   +0x05c Event            : _KEVENT

   +0x06c CompletionContext : (null) 

   +0x070 IrpListLock      : 0

   +0x074 IrpList          : _LIST_ENTRY [ 0x8750eff4 - 0x8750eff4 ]

   +0x07c FileObjectExtension : 0x8774e950 Void

0: kd> ??FileObject->SectionObjectPointer

struct _SECTION_OBJECT_POINTERS * 0x87bddea8

   +0x000 DataSectionObject : 0x863c1758 Void

   +0x004 SharedCacheMap   : 0x869acea0 Void

   +0x008 ImageSectionObject : (null) 

0: kd> dt nt!_CONTROL_AREA 0x863c1758 

   +0x000 Segment          : 0xaeb311a8 _SEGMENT

   +0x004 DereferenceList  : _LIST_ENTRY [ 0x0 - 0x0 ]

   +0x00c NumberOfSectionReferences : 1

   +0x010 NumberOfPfnReferences : 0x40

   +0x014 NumberOfMappedViews : 1

   +0x018 NumberOfUserReferences : 0

   +0x01c u                : <unnamed-tag>

   +0x020 FlushInProgressCount : 0

   +0x024 FilePointer      : _EX_FAST_REF

   +0x028 ControlAreaLock  : 0n0

   +0x02c ModifiedWriteCount : 0

   +0x02c StartingFrame    : 0

   +0x030 WaitingForDeletion : (null) 

   +0x034 u2               : <unnamed-tag>

   +0x040 LockedPages      : 0n1

   +0x048 ViewList         : _LIST_ENTRY [ 0x86a3a898 - 0x86a3a898 ]

0: kd> ??sizeof(nt!_CONTROL_AREA)

unsigned int 0x50

0: kd> dt nt!_SUBSECTION 0x863c1758+0x50

   +0x000 ControlArea      : 0x863c1758 _CONTROL_AREA

   +0x004 SubsectionBase   : 0xa8e9b008 _MMPTE

   +0x008 NextSubsection   : (null) 

   +0x00c PtesInSubsection : 0x40

   +0x010 UnusedPtes       : 0

   +0x010 GlobalPerSessionHead : (null) 

   +0x014 u                : <unnamed-tag>

   +0x018 StartingSector   : 0

   +0x01c NumberOfFullSectors : 0x40

0: kd> dt nt!_SEGMENT  0xaeb311a8 

   +0x000 ControlArea      : 0x863c1758 _CONTROL_AREA

   +0x004 TotalNumberOfPtes : 0x40

   +0x008 SegmentFlags     : _SEGMENT_FLAGS

   +0x00c NumberOfCommittedPages : 0

   +0x010 SizeOfSegment    : 0x40000

   +0x018 ExtendInfo       : (null) 

   +0x018 BasedAddress     : (null) 

   +0x01c SegmentLock      : _EX_PUSH_LOCK

   +0x020 u1               : <unnamed-tag>

   +0x024 u2               : <unnamed-tag>

   +0x028 PrototypePte     : 0xa8fe97e8 _MMPTE

   +0x030 ThePtes          : [1] _MMPTE

1: kd> dt nt!_MMPTE .
   +0x000 u                :
      +0x000 Long             : Uint8B
      +0x000 VolatileLong     : Uint8B
      +0x000 HighLow          : _MMPTE_HIGHLOW
      +0x000 Flush            : _HARDWARE_PTE
      +0x000 Hard             : _MMPTE_HARDWARE
      +0x000 Proto            : _MMPTE_PROTOTYPE
      +0x000 Soft             : _MMPTE_SOFTWARE
      +0x000 TimeStamp        : _MMPTE_TIMESTAMP
      +0x000 Trans            : _MMPTE_TRANSITION
      +0x000 Subsect          : _MMPTE_SUBSECTION
      +0x000 List             : _MMPTE_LIST

1: kd> dt nt!_MMPTE_PROTOTYPE
   +0x000 Valid            : Pos 0, 1 Bit
   +0x000 Unused0          : Pos 1, 7 Bits
   +0x000 ReadOnly         : Pos 8, 1 Bit
   +0x000 Unused1          : Pos 9, 1 Bit
   +0x000 Prototype        : Pos 10, 1 Bit
   +0x000 Protection       : Pos 11, 5 Bits
   +0x000 Unused           : Pos 16, 16 Bits
   +0x000 ProtoAddress     : Pos 32, 32 Bits

Caching and file object reference in Windows.

This is how the last reference to a file object backing cached file data is being released by the kernel. In that case this was a network filesystem

19 nt!IofCallDriver
1a mup!MupiCallUncProvider
1b mup!MupStateMachine
1c mup!MupClose
1d nt!IofCallDriver
1e nt!IopDeleteFilec
1f nt!ObpRemoveObjectRoutine
20 nt!ObfDereferenceObjectWithTag
21 nt!ObfDereferenceObject
22 nt!CcDeleteSharedCacheMap
23 nt!CcWriteBehind
24 nt!CcWorkerThread
25 nt!ExpWorkerThread
26 nt!PspSystemThreadStartup
27 nt!KiThreadStartup

Mac OS X file system filtering via hooking technique.

 Mac OS X doesn't support a full fledged file system filtering like Windows as Apple believes that BSD stackable file system doesn't fit Mac OS X. The available kernel authorization subsystem ( kauth ) allows only filtering open requests and a limited number of operations on file/directory. It doesn't allow filtering read and write operations and provides a limited control over file system operations as a vnode is already created at the moment of kauth callback invoking. In Windows parlance a kauth callback is a postoperation callback for create/open request ( IRP_MJ_CREATE for Windows ), that is all you have for Mac OS X. Not much really.

 The filtering can also be implemented by registering MAC ( Mandatory Access Control ) layer. But MAC has limited functionality and not consistent in relation to file system filtering as it was not designed as a file system filter layer. Instead of being called by VFS layer MAC registered callbacks are scattered through the kernel code. They are called from system calls and other kernel subsystems, so MAC doesn't provide a consistent interface for VFS filter and if I remember correctly MAC was declared by Apple as deprecated for usage by third party developers.

 To summarize the above, there is no official support from Apple if you need to filter read or/and write requests, filter and modify VFS vnode operation.

 The lack of a stackable file system support by Mac OS X VFS required to find a way to place a filter between VFS invoking vnode operations via VNOP_* functions and a file system driver implementing these vnode operations. I developed a hooking technique for VFS layer that emulates a stackable file system by replacing vnode operations in array. This technique allows to place a filter between VFS and file system driver and supports sophisticated filtering such as isolation file system filter when a filter creates vnodes instead of a file system driver thus gaining a full control over file data. I used this technique in two projects to implement filtering for lookup, create, read and write requests and to implement an isolation file system.

 The code for a skeleton file system filter is in my GitHub repository https://github.com/slavaim/MacOSX-FileSystem-Filter it was extracted from a bigger project MacOSX-Kernel-Filter in the same repository.

 FltFakeFSD.h and FltFakeFSD.cpp are optional files that helps to infer the vnode and vnodeop_desc structures layout that are not declared in SDK.  This is achieved by registering a dummy file system, creating a vnode and inspecting it to find required offsets. All you need to do is to call FltGetVnodeLayout() in a filter initialization code.

 Alternatively you can extract declarations from XNU code at opensource.apple.com. An alternative implementation without using the fake FSD can be found in VersionDependent.h and VersionDependent.cpp files, where struct vnodeop_desc_Yosemite was borrowed from Apple's open source code for Yosemite(10.10), it happened that vnode and vnodeop_desc structures haven't change in all latest kernel versions so this code works for Mavericks(10.9) and El Capitan(10.11).

 The filter registers a kauth callback FltIOKitKAuthVnodeGate::VnodeAuthorizeCallback which in turn calls FltHookVnodeVopAndParent that hooks vnode related operations. The kauth callback is used only to trigger filtering for a file system. When an application opens a file FileX in a directory DirX a vnode for a DirX is provided as a parameter for VnodeAuthorizeCallback so the following calls to file system's lookup or create operations for FileX will be visible for file system filter. When an original lookup or create returns a filter calls FltHookVnodeVopAndParent for a returned vnode or does this in a kauth callback which is nearly the same as kauth callback is a postoperation callback called after VNOP_LOOKUP or VNOP_CREATE. In case of lookup vnode operation do not forget about name cache, so lookup is not always called when an application calls open( FileX ), but if you need to see all lookup operations it is possible to disable name caching for a vnode. You can devise your own way of finding vnodes to hook depending on your requirements, for my purposes a kauth callback worked well as I was filtering vnodes of VREG type so I can use VDIR vnode as an anchor that started filtering for VREG vnodes. Also in most file systems vnode operations array is shared between all vnodes of VREG and VDIR type.

 The filter registers operations it wants to intercept in gFltVnodeVopHookEntries array. The skeleton filter immediately calls an original function from a hook but if you want to see a filter in action look at DldVNodeHook.cpp which is a more advanced implementation at MacOSX-Kernel-Filter that also implements an isolation filter for read and write operations on a file by redirecting read and write to a sparse file on another volume. The isolation filter is implemented at DldCoveringVnode.cpp and a sparse file at DldSparseFile.cpp. 

FYI a set of call stacks when hooks are active

• Lookup hook

  frame #0: 0xffffff7f903b7a11 FsdFilter`FltVnopLookupHook(ap=0xffffff80a7053a18) + 17 at VFSHooks.cpp:65
  

  frame #1: 0xffffff800dd3f2f8 kernellookup(ndp=0xffffff80a7053d58) + 968 at kpi_vfs.c:2783

  frame #2: 0xffffff800dd3ea95 kernel`namei(ndp=0xffffff80a7053d58) + 1941 at vfs_lookup.c:371

  frame #3: 0xffffff800dd52005 kernel`nameiat(ndp=0xffffff80a7053d58, dirfd=<unavailable>) + 133 at vfs_syscalls.c:2920

  frame #4: 0xffffff800dd5fcc7 kernel`fstatat_internal(segflg=<unavailable>, ctx=<unavailable>, path=<unavailable>, ub=<unavailable>, xsecurity=<unavailable>, xsecurity_size=<unavailable>, isstat64=<unavailable>, fd=<unavailable>, flag=<unavailable>) + 231 at vfs_syscalls.c:5268

  frame #5: 0xffffff800dd54f0a kernel`stat64(p=<unavailable>, uap=0xffffff801e3f1380, retval=<unavailable>) + 58 at vfs_syscalls.c:5413

  frame #6: 0xffffff800e04dcb2 kernel`unix_syscall64(state=0xffffff801dfc5540) + 610 at systemcalls.c:366
• Pagein hook
  frame #0: 0xffffff7f903b7b91 FsdFilterFlt`VnopPageinHook(ap=0xffffff80a76aba20) + 17 at VFSHooks.cpp:229
frame #1: 0xffffff800e046372 kernel`vnode_pagein(vp=0xffffff801904db40, upl=0x0000000000000000, upl_offset=<unavailable>, f_offset=73416704, size=<unavailable>, flags=0, errorp=0xffffff80a76abae8) + 402 at kpi_vfs.c:4980
frame #2: 0xffffff800db952a8 kernelv`node_pager_cluster_read(vnode_object=0xffffff80a76abb10, base_offset=73416704, offset=<unavailable>, io_streaming=<unavailable>, cnt=<unavailable>) + 72 at bsd_vm.c:1045
frame #3: 0xffffff800db943f3 kernel`vnode_pager_data_request(mem_obj=0xffffff8018fb5e38, offset=73433088, length=<unavailable>, desired_access=<unavailable>, fault_info=<unavailable>) + 99 at bsd_vm.c:826
frame #4: 0xffffff800db9f75b kernel`vm_fault_page(first_object=0x0000000000000000, first_offset=73433088, fault_type=1, must_be_resident=0, caller_lookup=0, protection=0xffffff80a76abe84, result_page=<unavailable>, top_page=<unavailable>, type_of_fault=<unavailable>, error_code=<unavailable>, no_zero_fill=<unavailable>, data_supply=0, fault_info=0xffffff80a76abbe0) + 3051 at memory_object.c:2178
frame #5: 0xffffff800dba3902 kernel`vm_fault_internal(map=0xffffff8010f761e0, vaddr=140735436247040, fault_type=1, change_wiring=0, interruptible=2, caller_pmap=0x0000000000000000, caller_pmap_addr=<unavailable>, physpage_p=0x0000000000000000) + 3042 at vm_fault.c:4423
frame #6: 0xffffff800dc1ec9c kernel`user_trap(saved_state=<unavailable>) + 732 at vm_fault.c:3229

• Close hook
  frame #0: 0xffffff7f903b7a91 FsdFilter`FltVnopCloseHook(ap=0xffffff80a78cbd28) + 17 at VFSHooks.cpp:119
frame #1: 0xffffff800dd734b7 kernel`VNOP_CLOSE(vp=0xffffff801c7144b0, fflag=<unavailable>, ctx=<unavailable>) + 215 at kpi_vfs.c:3047
frame #2: 0xffffff800dd671d1 kernel`vn_close(vp=0xffffff801c7144b0, flags=<unavailable>, ctx=0xffffff80a78cbe60) + 225 at vfs_vnops.c:723
frame #3: 0xffffff800dd6850f kernel`vn_closefile(fg=<unavailable>, ctx=0xffffff80a78cbe60) + 159 at vfs_vnops.c:1494
frame #4: 0xffffff800dfb1680 kernel`closef_locked [inlined] fo_close(fg=0xffffff801ba367e0, ctx=0xffffff801bcb42a0) + 14 at kern_descrip.c:5711
frame #5: 0xffffff800dfb1672 kernel`closef_locked(fp=<unavailable>, fg=0xffffff801ba367e0, p=0xffffff8019600650) + 354 at kern_descrip.c:4982
frame #6: 0xffffff800dfad88e kernelclose_internal_locked(p=0xffffff8019600650, fd=<unavailable>, fp=0xffffff801fc113d8, flags=<unavailable>) + 542 at kern_descrip.c:2765
frame #7: 0xffffff800dfb13d6 kernel`close_nocancel(p=0xffffff8019600650, uap=<unavailable>, retval=<unavailable>) + 342 at kern_descrip.c:2666
frame #8: 0xffffff800e04dcb2 kernel`unix_syscall64(state=0xffffff801bc20b20) + 610 at systemcalls.c:366
• Inactive hook
  frame #0: 0xffffff7fa9dc6ac0 FsdFilter`FltVnopInactiveHook(ap=0xffffff803c5198cc) at VFSHooks.cpp:140
frame #1: 0xffffff8027775e06 kernelVNOP_INACTIVE(vp=0xffffff803c519870, ctx=<unavailable>) + 70 at kpi_vfs.c:4756
frame #2: 0xffffff8027745543 kernel`vnode_rele_internal(vp=0xffffff80bfecbd90, fmode=<unavailable>, dont_reenter=<unavailable>, locked=<unavailable>) + 435 at vfs_subr.c:1929
frame #3: 0xffffff80279c5359 kernel`proc_exit(p=0xffffff80338eb000) + 2841 at vfs_subr.c:1845
frame #4: 0xffffff802756447a kernel`thread_terminate_self + 474 at thread.c:466
frame #5: 0xffffff802756878b kernel`special_handler(rh=<unavailable>, thread=<unavailable>) + 219 at thread_act.c:891
frame #6: 0xffffff802756855a kernel`act_execute_returnhandlers + 202 at thread_act.c:801
frame #7: 0xffffff8027536e76 kernel`ast_taken(reasons=<unavailable>, enable=<unavailable>) + 278 at ast.c:177
frame #8: 0xffffff802761eeae kernel`i386_astintr(preemption=<unavailable>) + 46 at trap.c:1171

The module has been made unloadable by taking a reference to IOKit com_FsdFilter class in com_FsdFilter::start routine. The reason is that unload for a hooking file system filter driver is a dangerous operation as it is hard to implement it correctly to avoid race conditions and preserve data consistency. To process unload correctly you need to process the following cases

• unhook all vnode tables
• check that there is no return to the hooking code in any call stack or else unloading a module results in a panic as control returns to unloaded code
• check that there is no hooking code being called, i.e. there is no a processor with IP register pointing to any function in a driver
• ensure that data changed by a filter will not crash the system or damage user or system data, for example if the driver implements any data modification it should guarantee data consistency on unload

The only feasible way to implement this is dividing a driver in two parts - one part can be unloaded and another part is never unloaded. Unloading complicates the code as you have to synchronize the code with unloading procedure, this increases complexity and as a result of this complexity and nondeterminism in the system state it's never implemented correctly.

A case of successful registration of an incorrectly defined file system minifilter.

 An interesting observation. If you forget to terminate FLT_OPERATION_REGISTRATION array with IRP_MJ_OPERATION_END then no instances will be attached but a minifilter is successfully registered and InstanceSetup callback is called. No any error is reported. Just yet another case of a closed source Microsoft subsystem with inconsistent behavior when you can spent hours chasing a bug by trial and error approach instead of looking at source code.

Unsafe use of FltGetFileNameInformationUnsafe

  FltGetFileNameInformationUnsafe name is spot on. It is very unsafe and dangerous function but not in the way mentioned in WDK. The documentation misses yet another dangerous case when it should not be used. This case is related to file system isolation filters( for more information on such filters follow this link https://www.osronline.com/article.cfm?article=560 ). I would state this additional condition as

  - A filter driver must not use FltGetFileNameInformationUnsafe for file objects it never observed. A filter observed a file object if it was called at least once with this file object for any operation and it never received IRP_MJ_CLOSE for this file object or observed a failed IRP_MJ_CREATE. This condition guaranties that a file object is initialized by a filter of file system driver that is beneath a filter calling FltGetFileNameInformationUnsafe . 

  When the above condition is violated by any file system filter a filter might call an underlying file system with a file object this file system never initialized as the file object was initialized by a file system filter that is above a filter that has called FltGetFileNameInformationUnsafe . At first glance it seems that in that case an upper filter must isolate the underlying filters from such file object. This is true, but there is a degenerate case when a filter can gain access to file objects that it should never see. This is a case of a process creation callback when a file object for executable file is provided as a parameter to a callback and this file object might have been initialized by a file system filter above one that registered this callback. Below is a call stack from the system where Microsoft Windows Defender file system minifilter used FltGetFileNameInformationUnsafe from a process creation callback to query a file name for a file object initialized by a file system filter attached to MiniFilter Manager ( aka fltmgr.sys ). 

Ntfs!NtfsCommonQueryInformation+0xa7

Ntfs!NtfsFsdDispatchSwitch+0xd3

Ntfs!NtfsFsdDispatchWait+0x47

nt!IovCallDriver+0x3cd

fltmgr!FltpQueryInformationFile+0x10e

fltmgr!QueryStandardLinkInformation+0x4d

fltmgr!FltpSetStreamListStandardInformationFlags+0x7a

fltmgr!FltpGetFileNameInformation+0x796

fltmgr!FltGetFileNameInformationUnsafe+0x71

WdFilter!MpGetImageNormalizedName+0x51

WdFilter!MpCreateProcessNotifyRoutineEx+0x77

nt!PspInsertThread+0x7a7

nt!NtCreateUserProcess+0x806

nt!KiSystemServiceCopyEnd+0x13

As a result NTFS crashed when it received an IRP with a file object initialized by a file system filter attached to the top of the drivers stack. The obvious solution is to use the old good ObQueryNameString as it has been done for the last 25 years by the kernel itself, for example in NtQueryVirtualMemory when processing a memory region supported by a mapped file. In that case an IRP is created and sent from the very top of a drivers stack so an isolation filter has a chance to intercept this IRP and complete it.

On importance of FileObject->Vpb pointer

 An easily overlooked issue in FS(file systems) and FS filters development for Windows is Vpb pointer in FILE_OBJECT structure, i.e. Volume Parameters Block ( struct VPB ). It is responsibility of FS or FS filter ( in case of stacked FS, i.e. FSD-over-FSD ) to initialize this pointer to a correct value. It is especially important in case of FSD over FSD implementations as some underlying FSs check the VPB->ReferenceCount pointer on their PnP path and both upper FSD and lower FSD share the same VPB pointer. The kernel references Vpb in IopParseDevice routine that calls IRP_MJ_CREATE dispatch routine, but doesn't set Vpb pointer for FileObject sent with create Irp, and dereferences Vpb in IopDeleteFile just after IRP_MJ_CLOSE completes but IopDeleteFile fetches the pointer from FileObject->Vpb that is set by FS when processing IRP_MJ_CREATE. If FS or FS filter fails to set Vpb pointer PnP safe remove for some file systems would be impossible as Vpb->ReferenceCount never drops below a checked value.

 I encountered this problem when working on my own implementation for FSD-over-FSD FS filter. The problem manifested itself on FAT32 , NTFS was clean from this issue.

 Below are two call stacks for 32 bit Windows 8, the first is for a case when Vpb->ReferenceCount is bumped and the second for a case when it is decremented.


nt!IopCheckVpbMounted+0x81
nt!IopParseDevice+0x48b
nt!ObpLookupObjectName+0x6ef
nt!ObOpenObjectByName+0x1e3
nt!IopCreateFile+0x372
nt!NtCreateFile+0x78
nt!KiSystemServiceCopyEnd+0x13
ntdll!NtCreateFile+0xa


nt!IopDecrementVpbRefCount+0x5b
nt!IopDeleteFile+0xf5
nt!ObpRemoveObjectRoutine+0x64
nt!ObfDereferenceObjectWithTag+0x8f
nt!NtClose+0x210
nt!KiSystemServiceCopyEnd+0x13