Posts RSS Comments RSS 249 Posts and 391 Comments till now

Using the LDAP stats control with Powershell

At TEC this year I did a session with Darren the "GPO Guy" regarding using S.DS.Protocols to get back query stats from Active Directory. I mentioned my stats control code several times and I promised I would post the code so here it is:

What: The stats control is a LDAP control that you can pass that will tell the server to return its internal stats on a query.

Why: The stats control is a great way to see what the Domain Controller does with your filter. Like what indexes it hits, how many entries it had to visit, how much time the DC spent, and entries visited. It is very useful in creating the most efficient LDAP Query possible.

How: I Used System.DirectoryServices.Protocols.DirectoryControl to pass the LDAP control to the Server and I used System.DirectoryServices.Protocols.BERConverter along with the protocol spec here: LDAP_SERVER_GET_STATS_OID: 1.2.840.113556.1.4.970 to decode the Byte Array that was returned.

Here is what is Returned: For 2000
- threadCount: Number of threads that were processing LDAP requests on the DC at the time the search operation was performed.
- coreTime: The time in milliseconds which core logic in the DC spent processing the request.
- callTime: The overall time in milliseconds that the DC spent processing the request.
- searchSubOperations: The number of individual operations which the DC performed in processing the request.

For 2003/2008
- threadCount: Number of threads that were processing LDAP requests on the DC at the time the search operation was performed.
- callTime: The overall time in milliseconds that the DC spent processing the request
- entriesReturned: The number of objects returned in the search result.
- entriesVisited: The number of objects that the DC considered for inclusion in the search result.
- filter: String which represents the optimized form of the search filter used by the DC to perform a search. This very well may be different than the filter that was passed.
- index: String which indicates which database indexes were used by the DC to perform the search.

For 2008 Only
- pagesReferenced: The number of database pages referenced by the DC in processing the search.
- pagesRead: The number of database pages read from disk.
- pagesPreread: The number of database pages preread from disk by the DC in processing the search.
- pagesDirtied: The number of clean database pages modified by the DC in processing the search.
- pagesRedirtied: The number of previously modified database pages that were modified by the DC in processing the search.
- logRecordCount: The number of database log records generated by the DC in processing the search.
- logRecordBytes: The size in bytes of database log records generated by the DC in processing the search.

Note:
- Must have SE_DEBUG_PRIVILEGE
- I did NOT implement SO_EXTENDED_FMT flag.
- I did NOT test 2000.
- The functions that decodes Byte Array actually return objects, but for this test I just outputed the test to mimic ADFind.exe
- Special thanks to Robin Caron, joe Richards, and Dmitri Gavrilov for help with the decoding.
- Here is GREAT Doc on the Controls (and everything else AD) [MS-ADTS]: Active Directory Technical Specification

Code:
  1. Param(
  2.         $filter = "(objectclass=*)",
  3.         $base,
  4.         $Server,
  5.         [int]$pageSize = 1000,
  6.         [string[]]$props = @("1.1"),
  7.         [switch]$StatsOnly,
  8.         [switch]$Verbose
  9.     )
  10. function CreateStatsObject2008{
  11.     Param($StatsArray)
  12.     $DecodedArray = [System.DirectoryServices.Protocols.BerConverter]::Decode("{iiiiiiiiiaiaiiiiiiiiiiiiii}",$StatsArray) # Win2008
  13.     $myStatsObject = New-Object System.Object
  14.     $myStatsObject | Add-Member -Name "ThreadCount"     -Value $DecodedArray[1]  -MemberType "NoteProperty"
  15.     $myStatsObject | Add-Member -Name "CallTime"        -Value $DecodedArray[3]  -MemberType "NoteProperty"
  16.     $myStatsObject | Add-Member -Name "EntriesReturned" -Value $DecodedArray[5]  -MemberType "NoteProperty"
  17.     $myStatsObject | Add-Member -Name "EntriesVisited"  -Value $DecodedArray[7]  -MemberType "NoteProperty"
  18.     $myStatsObject | Add-Member -Name "Filter"          -Value $DecodedArray[9]  -MemberType "NoteProperty"
  19.     $myStatsObject | Add-Member -Name "Index"           -Value $DecodedArray[11] -MemberType "NoteProperty"
  20.     $myStatsObject | Add-Member -Name "PagesReferenced" -Value $DecodedArray[13] -MemberType "NoteProperty"
  21.     $myStatsObject | Add-Member -Name "PagesRead"       -Value $DecodedArray[15] -MemberType "NoteProperty"
  22.     $myStatsObject | Add-Member -Name "PagesPreread"    -Value $DecodedArray[17] -MemberType "NoteProperty"
  23.     $myStatsObject | Add-Member -Name "PagesDirtied"    -Value $DecodedArray[19] -MemberType "NoteProperty"
  24.     $myStatsObject | Add-Member -Name "PagesRedirtied"  -Value $DecodedArray[21] -MemberType "NoteProperty"
  25.     $myStatsObject | Add-Member -Name "LogRecordCount"  -Value $DecodedArray[23] -MemberType "NoteProperty"
  26.     $myStatsObject | Add-Member -Name "LogRecordBytes"  -Value $DecodedArray[25] -MemberType "NoteProperty"
  27.     $myStatsObject
  28. }
  29. function CreateStatsObject2003{
  30.     Param($StatsArray)
  31.     $DecodedArray = [System.DirectoryServices.Protocols.BerConverter]::Decode("{iiiiiiiiiaia}",$StatsArray) # Win2003
  32.     $myStatsObject = New-Object System.Object
  33.     $myStatsObject | Add-Member -Name "ThreadCount"     -Value $DecodedArray[1]  -MemberType "NoteProperty"
  34.     $myStatsObject | Add-Member -Name "CallTime"        -Value $DecodedArray[3]  -MemberType "NoteProperty"
  35.     $myStatsObject | Add-Member -Name "EntriesReturned" -Value $DecodedArray[5]  -MemberType "NoteProperty"
  36.     $myStatsObject | Add-Member -Name "EntriesVisited"  -Value $DecodedArray[7]  -MemberType "NoteProperty"
  37.     $myStatsObject | Add-Member -Name "Filter"          -Value $DecodedArray[9]  -MemberType "NoteProperty"
  38.     $myStatsObject | Add-Member -Name "Index"           -Value $DecodedArray[11] -MemberType "NoteProperty"
  39.     $myStatsObject
  40. }
  41. function CreateStatsObject2000{
  42.     Param($StatsArray)
  43.     $DecodedArray = [System.DirectoryServices.Protocols.BerConverter]::Decode("{iiiiiiii}",$StatsArray) # Win2000
  44.     $myStatsObject = New-Object System.Object
  45.     $myStatsObject | Add-Member -Name "ThreadCount"          -Value $DecodedArray[1]  -MemberType "NoteProperty"
  46.     $myStatsObject | Add-Member -Name "CoreTime"             -Value $DecodedArray[3]  -MemberType "NoteProperty"
  47.     $myStatsObject | Add-Member -Name "CallTime"             -Value $DecodedArray[5]  -MemberType "NoteProperty"
  48.     $myStatsObject | Add-Member -Name "searchSubOperations"  -Value $DecodedArray[7]  -MemberType "NoteProperty"
  49.     $myStatsObject
  50. }
  51.  
  52. if($Verbose){$VerbosePreference\3 = "Continue"}
  53.  
  54. Write-Verbose " - Loading System.DirectoryServices.Protocols"
  55. [VOID][System.Reflection.Assembly]::LoadWithPartialName("System.DirectoryServices.Protocols") 
  56.  
  57. [int]$pageCount = 0
  58. [int]$objcount = 0
  59.  
  60. if(!$Server)
  61. {
  62.     $rootDSE = [ADSI]"LDAP://rootDSE"
  63.     $Server = $rootDSE.dnsHostName
  64.     if(!$base){$base = $rootDSE.defaultNamingContext}
  65.     switch ($rootDSE.domainControllerFunctionality)
  66.     {
  67.         0 {$expression = 'CreateStatsObject2000 $stats'}
  68.         2 {$expression = 'CreateStatsObject2003 $stats'}
  69.         3 {$expression = 'CreateStatsObject2008 $stats'}
  70.     }
  71. }
  72.  
  73. Write-Verbose " - Using Server:  [$Server]"
  74. Write-Verbose " - Using Base:    [$base]"
  75. Write-Verbose " - Using Filter:  [$filter]"
  76. Write-Verbose " - Page Size:     [$PageSize]"
  77. Write-Verbose " - Returning:     [$props]"
  78. Write-Verbose " - CSV:           [$csv]"
  79. Write-Verbose " - NoHeaders:     [$noHeader]"
  80. Write-Verbose " - Count:         [$Count]"
  81. Write-Verbose " - StatsOnly:     [$StatsOnly]"
  82. Write-Verbose " - Expression:    [$expression]"
  83.  
  84. Write-Verbose " - Creating LDAP connection Object"
  85. $connection = New-Object System.DirectoryServices.Protocols.LdapConnection($Server)  
  86. $Subtree = [System.DirectoryServices.Protocols.SearchScope]"Subtree"
  87.  
  88. Write-Verbose " + Creating SearchRequest Object"
  89. $SearchRequest = New-Object System.DirectoryServices.Protocols.SearchRequest($base,$filter,$Subtree,$props)
  90.  
  91. Write-Verbose "   - Creating System.DirectoryServices.Protocols.PageResultRequestControl Object"
  92. $PagedRequest  = New-Object System.DirectoryServices.Protocols.PageResultRequestControl($pageSize)
  93.  
  94. Write-Verbose "   - Creating System.DirectoryServices.Protocols.SearchOptionsControl Object"
  95. $SearchOptions = New-Object System.DirectoryServices.Protocols.SearchOptionsControl([System.DirectoryServices.Protocols.SearchOption]::DomainScope)
  96.  
  97. Write-Verbose "   - Creating System.DirectoryServices.Protocols.DirectoryControl Control for OID: [1.2.840.113556.1.4.970]"
  98. $oid = "1.2.840.113556.1.4.970"
  99. $StatsControl = New-Object System.DirectoryServices.Protocols.DirectoryControl($oid,$null,$false,$true)
  100.  
  101. Write-Verbose "   - Adding Controls"
  102. [void]$SearchRequest.Controls.add($pagedRequest)
  103. [void]$SearchRequest.Controls.Add($searchOptions)
  104. [void]$SearchRequest.Controls.Add($StatsControl)
  105.  
  106. $start = Get-Date
  107. while ($True)
  108. {
  109.     # Increment the pageCount by 1
  110.     $pageCount++
  111.  
  112.     # Cast the directory response into a SearchResponse object
  113.     Write-Verbose " - Cast the directory response into a SearchResponse object"
  114.     $searchResponse = $connection.SendRequest($searchRequest)
  115.  
  116.     # Display the retrieved page number and the number of directory entries in the retrieved page
  117.     Write-Verbose (" - Page:{0} Contains {1} response entries" -f $pageCount,$searchResponse.entries.count)
  118.  
  119.  
  120.     Write-Verbose " - Returning Stats for Page:$PageCount"
  121.     $stats = $searchResponse.Controls[0].GetValue()
  122.     $ResultStats = invoke-Expression $expression
  123.     if($pageCount -eq 1)
  124.     {
  125.         $StatsFilter = $ResultStats.Filter
  126.         $StatsIndex = $ResultStats.Index
  127.         Write-Verbose "   + Setting Filter to [$StatsFilter]"
  128.         Write-Verbose "   + Setting Index  to [$StatsIndex]"
  129.     }
  130.  
  131.     # If Cookie Length is 0, there are no more pages to request"
  132.     if ($searchResponse.Controls[1].Cookie.Length -eq 0)
  133.     {
  134.         if($count){$objcount}
  135.         "`nStatistics"
  136.         "================================="
  137.         "Elapsed Time: {0} (ms)" -f ((Get-Date).Subtract($start).TotalMilliseconds)
  138.         "Returned {0} entries of {1} visited - ({2})`n" -f $ResultStats.EntriesReturned,$ResultStats.EntriesVisited,($ResultStats.EntriesReturned/$ResultStats.EntriesVisited).ToString('p')
  139.         "Used Filter:"
  140.         "- {0}`n" -f $StatsFilter
  141.         "Used Indices:"
  142.         "- {0}`n" -f $StatsIndex
  143.         break
  144.     }
  145.  
  146.     # Set the cookie of the pageRequest equal to the cookie of the pageResponse to request the next 
  147.     # page of data in the send request and cast the directory control into a PageResultResponseControl object
  148.     Write-Verbose " - Setting Cookie on SearchResponse to the PageReQuest"
  149.     $pagedRequest.Cookie = $searchResponse.Controls[1].Cookie
  150. }

tshell :: Apr.09.2009 :: Active Directory, All :: No Comments »

Trackback this post | Feed on Comments to this post

Leave a Reply