Wednesday, October 11, 2017

SQL Server – Script to Rebuild / Reorganize Indexes based on Fragmentation and No of Pages

Heavily fragmented indexes can degrade query performance and cause applications to respond slowly. Reorganizing or rebuilding of indexes will fix this issue. However, this is a very resource intensive process. Therefore identify and only rebuild the indexes that need to be rebuilt is important.

Following criteria can be used to determine which indexes to rebuild, which indexes to reorganize, and which indexes to leave alone.
  • REBUILD index : if fragmentation is > 30% and number of pages > 1000
  • REORGANIZE index : if fragmentation is > 10 % but < 30% and number of pages > 1000

We can use system function sys.dm_db_index_physical_stats to find out fragmentation information of indexes like:
  • avg_fragmentation_in_percent: The percent of logical fragmentation (out-of-order pages in the index).
  • fragment_count: The number of fragments (physically consecutive leaf pages) in the index.
  • avg_fragment_size_in_pages: Average number of pages in one fragment in an index.

Following is a complete SQL script copied from http://www.sqlmusings.com/2009/03/15/a-more-effective-selective-index-rebuildreorganize-strategy/ for the above purpose. Setting “report_only = 1” of the script will only analyze the database without updating.

-- Ensure a USE <databasename> statement has been executed first.
SET NOCOUNT ON

-- adapted from "Rebuild or reorganize indexes (with configuration)" from MSDN Books Online
-- (http://msdn.microsoft.com/en-us/library/ms188917.aspx)

-- =======================================================
-- || Configuration variables:
-- || - 10 is an arbitrary decision point at which to reorganize indexes.
-- || - 30 is an arbitrary decision point at which to
-- || switch from reorganizing, to rebuilding.
-- || - 0 is the default fill factor. Set this to a
-- || a value from 1 to 99, if needed.
-- =======================================================
DECLARE @reorg_frag_thresh   float       SET @reorg_frag_thresh   = 10.0
DECLARE @rebuild_frag_thresh float       SET @rebuild_frag_thresh = 30.0
DECLARE @fill_factor         tinyint     SET @fill_factor         = 80
DECLARE @report_only         bit         SET @report_only         = 1

-- added (DS) : page_count_thresh is used to check how many pages the current table uses
DECLARE @page_count_thresh smallint     SET @page_count_thresh   = 1000

-- Variables required for processing.
DECLARE @objectid       int
DECLARE @indexid        int
DECLARE @partitioncount bigint
DECLARE @schemaname     nvarchar(130)
DECLARE @objectname     nvarchar(130)
DECLARE @indexname      nvarchar(130)
DECLARE @partitionnum   bigint
DECLARE @partitions     bigint
DECLARE @frag           float
DECLARE @page_count     int
DECLARE @command        nvarchar(4000)
DECLARE @intentions     nvarchar(4000)
DECLARE @table_var      TABLE(
                          objectid     int,
                          indexid      int,
                          partitionnum int,
                          frag         float,
                          page_count   int
                        )

-- Conditionally select tables and indexes from the
-- sys.dm_db_index_physical_stats function and
-- convert object and index IDs to names.
INSERT INTO
    @table_var
SELECT
    [object_id]                    AS objectid,
    [index_id]                     AS indexid,
    [partition_number]             AS partitionnum,
    [avg_fragmentation_in_percent] AS frag,
       [page_count]                         AS page_count
FROM
    sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL , NULL, 'LIMITED')
WHERE
    [avg_fragmentation_in_percent] > @reorg_frag_thresh
       AND
       page_count > @page_count_thresh
       AND
    index_id > 0
      

-- Declare the cursor for the list of partitions to be processed.
DECLARE partitions CURSOR FOR
    SELECT * FROM @table_var

-- Open the cursor.
OPEN partitions

-- Loop through the partitions.
WHILE (1=1) BEGIN
    FETCH NEXT
        FROM partitions
        INTO @objectid, @indexid, @partitionnum, @frag, @page_count

    IF @@FETCH_STATUS < 0 BREAK

    SELECT
        @objectname = QUOTENAME(o.[name]),
        @schemaname = QUOTENAME(s.[name])
    FROM
        sys.objects AS o WITH (NOLOCK)
        JOIN sys.schemas as s WITH (NOLOCK)
        ON s.[schema_id] = o.[schema_id]
    WHERE
        o.[object_id] = @objectid

    SELECT
        @indexname = QUOTENAME([name])
    FROM
        sys.indexes WITH (NOLOCK)
    WHERE
        [object_id] = @objectid AND
        [index_id] = @indexid

    SELECT
        @partitioncount = count (*)
    FROM
        sys.partitions WITH (NOLOCK)
    WHERE
        [object_id] = @objectid AND
        [index_id] = @indexid

    -- Build the required statement dynamically based on options and index stats.
    SET @intentions =
        @schemaname + N'.' +
        @objectname + N'.' +
        @indexname + N':' + CHAR(13) + CHAR(10)
    SET @intentions =
        REPLACE(SPACE(LEN(@intentions)), ' ', '=') + CHAR(13) + CHAR(10) +
        @intentions
    SET @intentions = @intentions +
        N' FRAGMENTATION: ' + CAST(@frag AS nvarchar) + N'%' + CHAR(13) + CHAR(10) +
        N' PAGE COUNT: '    + CAST(@page_count AS nvarchar) + CHAR(13) + CHAR(10)

    IF @frag < @rebuild_frag_thresh BEGIN
        SET @intentions = @intentions +
            N' OPERATION: REORGANIZE' + CHAR(13) + CHAR(10)
        SET @command =
            N'ALTER INDEX ' + @indexname +
            N' ON ' + @schemaname + N'.' + @objectname +
            N' REORGANIZE; ' +
            N' UPDATE STATISTICS ' + @schemaname + N'.' + @objectname +
            N' ' + @indexname + ';'

    END
    IF @frag >= @rebuild_frag_thresh BEGIN
        SET @intentions = @intentions +
            N' OPERATION: REBUILD' + CHAR(13) + CHAR(10)
        SET @command =
            N'ALTER INDEX ' + @indexname +
            N' ON ' + @schemaname + N'.' +     @objectname +
            N' REBUILD'
    END
    IF @partitioncount > 1 BEGIN
        SET @intentions = @intentions +
            N' PARTITION: ' + CAST(@partitionnum AS nvarchar(10)) + CHAR(13) + CHAR(10)
        SET @command = @command +
            N' PARTITION=' + CAST(@partitionnum AS nvarchar(10))
    END
    IF @frag >= @rebuild_frag_thresh AND @fill_factor > 0 AND @fill_factor < 100 BEGIN
        SET @intentions = @intentions +
            N' FILL FACTOR: ' + CAST(@fill_factor AS nvarchar) + CHAR(13) + CHAR(10)
        SET @command = @command +
            N' WITH (FILLFACTOR = ' + CAST(@fill_factor AS nvarchar) + ')'
    END

    -- Execute determined operation, or report intentions
    IF @report_only = 0 BEGIN
        SET @intentions = @intentions + N' EXECUTING: ' + @command
        PRINT @intentions     
        EXEC (@command)
    END ELSE BEGIN
        PRINT @intentions
    END
       PRINT @command

END

-- Close and deallocate the cursor.
CLOSE partitions
DEALLOCATE partitions

GO

Rebuilding an index can be executed online or offline. Reorganizing an index is always executed online: https://docs.microsoft.com/en-us/sql/relational-databases/indexes/reorganize-and-rebuild-indexes

Friday, April 24, 2015

Cross Domain Access to HTTP Handler using jQuery Ajax

We can access Http handlers directly from a regular AJAX call, if they are in the same domain. However it is not the case if handler is hosted in a different domain. One solution is to use JSONP (the “padding” around the pure JSON).

JSONP pass the response from the server in to a user specified function, return the response as JSON, but also wrap the response in the requested call back.

In order to use JSONP, server has to support JSONP. The client tells the server the callback function for the response. The server then return the response as JSON, but also wraps the response in this callback function.

HttpHandler sending the response in the format expected by JSONP.

public class AtomHandler : IHttpHandler
{
  ///<summary>
  /// Enables processing of HTTP Web requests by a custom HttpHandler that implements the System.Web.IHttpHandler interface.
  ///</summary>
  /// <param name="context"> A System.Web.HttpContext object that provides references to the intrinsic server objects (for example, Request, Response, Session, and Server) used to service HTTP requests .< /param>
  public voidProcessRequest( HttpContext context)
  {
   string jsonString= "{'Title':'What Will Brand Engagement Look Like in 2020?','Summary':'When it comes to brand engagement, 5 years will seem like a lifetime.','UpdatedDate':'4/23/2015 4:27:46 AM +00:00','FeedType':'MBD - Marketing'}" ;
 
   string jsonp=context.Request["callback"];
 
   if (! String.IsNullOrEmpty (jsonp))
   {
    jsonString = jsonp + "("+jsonString+")";
   }
 
   context.Response.ContentType = "application/json";
   context.Response.Write(jsonString);
  }
 
  ///<summary>
  /// Gets a value indicating whether another request can use this instance.
  ///</summary>
  public bool IsReusable
  {
   get
   {
    return false;
   }
  }
}


jQuery JSONP request to get data from handler.

var loadAtomData=function() {
  try {
    $.ajax({
      url: "http://<server>/AtomTest/AtomHandler.ashx",               
      type: "GET",
      dataType: "jsonp",
      jsonp: "callback",
      contentType: "application/json; charset=utf-8" ,
 
      success: function (responseData, txtStatus, jqXHR) {
        console.log( "The response is" , responseData);
      },
      error: function (responseData, txtStatus, errThrown) {
        console.warn(responseData, txtStatus, errThrown);
        alert( 'JSONP failed - ' + txtStatus);
      }
    });
  } catch(e) {
    alert(e)
  }
}
 
$(document).ready( function() {
loadAtomData();
});

Web browsers let scripts from a webpage to access data from a second page, if both have the same origin (URI scheme, hostname and port). JSONP works since browsers do not enforce the same-origin policy on <script> tags.

Tuesday, April 7, 2015

How to Create a Custom View for SharePoint 2010 Survey

As you all know creating a custom view for SharePoint in-built survey list is not there by default. But this is a required feature in some cases. For example if you want to filter survey responses by an answer to a particular question, then having a view with that question is handy. 

So here is a workaround:

01. First of all find the survey ID. We can do that by navigating to survey settings and Find query string parameter “List” from the browser URL.

http://<server>/_layouts/survedit.aspx?List=%7B630DCEEC%2D45DD%2D4C35%2D9EBF%2D7B2C607333AE%7D

02. You won’t find ‘Create View” option in the ribbon or anywhere, but can access that page by directly accessing from the URL. So visit “Create View” page using the URL:

http://<server>/_layouts/ViewType.aspx?List=<List ID from previous step>

03. In order to create a new View, click on “All Responses” link from “Start from an existing view” section.



04. In the Create View page, provide a “View Name”, select Columns and set any other settings. Click OK.

That is it! View can be accessed from the URL: http://<server>/Lists/<survey name>/<view name>.aspx

“Detailed” is the custom view that I created and shown in the above screenshot.

Wednesday, February 11, 2015

xrm.CreateQuery() - Invalid 'where' Condition



“Invalid ‘where’ condition. An entity member is invoking an invalid property or method.”

I came across above error, while I was working on a LINQ query to retrieve a set of records from a CRM 2013 custom entity. It is a CRM 2013 online environment and I was using XrmServiceContext to retrieve data.


Even though it appears to be that I have specified an invalid property, I found out below line is causing the issue.

Where(d => d.nav_expirationdate.HasValue && ...

It turns out that LINQ to CRM implementation expects two values in each condition and CRM Property must be on the left hand side. Otherwise where clause will simply crash.

Code that will Crash with Invalid 'where' condition exception:

this.xrm.CreateQuery<nav_myrequest>().Where(d => d.nav_expirationdate.HasValue && ...

Code that will Work without an error:

this.xrm.CreateQuery<nav_myrequest>().Where(d => d.nav_expirationdate != null && ...

Wednesday, September 3, 2014

CRM 2013 – Plugin Registration Error

Error: Plug-in assembly does not contain the required types or assembly content cannot be updated.

This error was occurring when I try to update a Plugins assembly using Update Assembly window of the CRM SDK PluginRegistration.exe. I was able to register this same assembly couple of hours ago and now without major changes I’m unable to register it.

Problem turned out to be that another developer has updated the same plugin assembly with extra plugin classes added to the assembly. I was getting this error; since I did not have those extra plugins in the assembly I was trying register.

This is something to note about when registering plugins for the testing purpose in a shared environment without check-in the code.

Full error I got:
Unhandled Exception:
System.ServiceModel.FaultException`1[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=6.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]: Plug-in assembly does not contain the required types or assembly content cannot be updated.
Detail: <OrganizationServiceFault xmlns="http://schemas.microsoft.com/xrm/2011/Contracts" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <ErrorCode>-2147204725</ErrorCode>
  <ErrorDetails xmlns:a="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
  <Message>Plug-in assembly does not contain the required types or assembly content cannot be updated.</Message>
  <Timestamp>2014-09-03T08:17:30.7702502Z</Timestamp>
  <InnerFault>
    <ErrorCode>-2147204725</ErrorCode>
    <ErrorDetails xmlns:a="http://schemas.datacontract.org/2004/07/System.Collections.Generic" />
    <Message>Plug-in assembly does not contain the required types or assembly content cannot be updated.</Message>
    <Timestamp>2014-09-03T08:17:30.7702502Z</Timestamp>
    <InnerFault i:nil="true" />
    <TraceText i:nil="true" />
  </InnerFault>
  <TraceText i:nil="true" />
</OrganizationServiceFault>

Server stack trace:
   at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.Xrm.Sdk.IOrganizationService.Update(Entity entity)
   at Microsoft.Xrm.Sdk.Client.OrganizationServiceProxy.UpdateCore(Entity entity)
   at Microsoft.Crm.Tools.PluginRegistration.RegistrationHelper.UpdateAssembly(CrmOrganization org, String pathToAssembly, CrmPluginAssembly assembly, PluginType[] type)
   at Microsoft.Crm.Tools.PluginRegistration.PluginRegistrationForm.btnRegister_Click(Object sender, EventArgs e)