pnfs.txt 2.3 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556
  1. Reference counting in pnfs:
  2. ==========================
  3. The are several inter-related caches. We have layouts which can
  4. reference multiple devices, each of which can reference multiple data servers.
  5. Each data server can be referenced by multiple devices. Each device
  6. can be referenced by multiple layouts. To keep all of this straight,
  7. we need to reference count.
  8. struct pnfs_layout_hdr
  9. ----------------------
  10. The on-the-wire command LAYOUTGET corresponds to struct
  11. pnfs_layout_segment, usually referred to by the variable name lseg.
  12. Each nfs_inode may hold a pointer to a cache of of these layout
  13. segments in nfsi->layout, of type struct pnfs_layout_hdr.
  14. We reference the header for the inode pointing to it, across each
  15. outstanding RPC call that references it (LAYOUTGET, LAYOUTRETURN,
  16. LAYOUTCOMMIT), and for each lseg held within.
  17. Each header is also (when non-empty) put on a list associated with
  18. struct nfs_client (cl_layouts). Being put on this list does not bump
  19. the reference count, as the layout is kept around by the lseg that
  20. keeps it in the list.
  21. deviceid_cache
  22. --------------
  23. lsegs reference device ids, which are resolved per nfs_client and
  24. layout driver type. The device ids are held in a RCU cache (struct
  25. nfs4_deviceid_cache). The cache itself is referenced across each
  26. mount. The entries (struct nfs4_deviceid) themselves are held across
  27. the lifetime of each lseg referencing them.
  28. RCU is used because the deviceid is basically a write once, read many
  29. data structure. The hlist size of 32 buckets needs better
  30. justification, but seems reasonable given that we can have multiple
  31. deviceid's per filesystem, and multiple filesystems per nfs_client.
  32. The hash code is copied from the nfsd code base. A discussion of
  33. hashing and variations of this algorithm can be found at:
  34. http://groups.google.com/group/comp.lang.c/browse_thread/thread/9522965e2b8d3809
  35. data server cache
  36. -----------------
  37. file driver devices refer to data servers, which are kept in a module
  38. level cache. Its reference is held over the lifetime of the deviceid
  39. pointing to it.
  40. lseg
  41. ----
  42. lseg maintains an extra reference corresponding to the NFS_LSEG_VALID
  43. bit which holds it in the pnfs_layout_hdr's list. When the final lseg
  44. is removed from the pnfs_layout_hdr's list, the NFS_LAYOUT_DESTROYED
  45. bit is set, preventing any new lsegs from being added.