Friday 22 May 2015

HRESULT: 0x80131904 SharePoint Database Full


As most companies who have adopted SharePoint will know, a huge part of any SharePoint project is migration of content from other legacy systems and file shares.

Even with the most controlled migration plan, there is always the possibiltiy of a proactive user manually dumping huge amounts of content on their SharePoint site (especially once they discover windows explorer view!).

When this occurs you will see sudden SQL growth and especially within the DocStreams table within the content database.

When a file is uploaded to SharePoint 2013 shredded storage breaks an individual BLOB into shredded BLOBs to be stored in the DocStreams table.

For some more detailed information regarding shredded storage check out this technet blog.

Back to the point, which may or may not be related to a migration. If a large amount of content is uploaded to your content database this can result in SQL storage issues.

You may see errors on the SharePoint pages, including HRESULT: 0x80131904

In the ULS log I could see that various tables were growing rapidly and filling their primary file group, namely the EventCache and AuditData tables due to the high usage.

ULS log errors:

Database full error on SQL Server instance 'SQL-SharePoint-Prd' in database 'ContentDB_Intranet'.
Additional error information from SQL Server is included below.  Could not allocate space for object 'dbo.EventCache'.'
EventCache_Id' in database 'ContentDB_Intranet' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files,
dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.


Database full error on SQL Server instance 'SQL-SharePoint-Prd' in database 'ContentDB_Intranet'.
Additional error information from SQL Server is included below.  Could not allocate space for object 'dbo.AuditData'.
'AuditData_OnSiteOccurred' in database 'ContentDB_Intranet' because the 'PRIMARY' filegroup is full. Create disk space by deleting unneeded files,
dropping objects in the filegroup, adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

You will also see event viewer errors:


Event viewer error:

Database full error on SQL Server instance 'SQL-SharePoint-Prd' in database 'ContentDB_Intranet'. Additional error information from SQL Server is included below.

Could not allocate space for object 'dbo.DocStreams'.'DocStreams_CI' in database 'ContentDB_Intranet' because the 'PRIMARY'

filegroup is full. Create disk space by deleting unneeded files, dropping objects in the filegroup,

adding additional files to the filegroup, or setting autogrowth on for existing files in the filegroup.

You could do maintenance to shrink these DBs, but the likely fix for most people will be to add the required disk space to allow the required growth of the databases.

You should consider your autogrowth settings, have a read of this for more details.

To avoid issues like this in the future monitoring of the free space on the SQL disks is essential for your production environment and there are various tools out there to help you with this.

Thanks for reading,
Matt

Monday 18 May 2015

SharePoint and Kerberos Incorrect SPN Error, The HTTP request is unauthorized with client authentication scheme 'Negotiate'

This post is related to an issue I have faced a couple of times in recent months, and I thought I would share my experience for anyone troubleshooting a similar issue.

My SharePoint web applications are set to use Negotiate (Kerberos), so I am assuming you have been through the process of setting up Kerberos authentication.

A great guide linked here by the way.

We have strict change control for production farms so I have only faced this in two staging farms where less stringent change management has meant configuration changes have been made and 'broken' Kerberos authentication.

But this could easily happen in a less tightly controlled production environment, which unfortunately I have too much experience with in the past!

The symptoms could be in many forms, depending how you are interacting with SharePoint.

I was trying to activate some features on a site and was receiving an error in PowerShell:

The HTTP request is unauthorized with client authentication scheme 'Negotiate'. The authentication header received from the server was 'Negotiate'

I then tried to browse the site in internet explorer and was prompted for my credentials, which would not be accepted.

There was no failing back to NTLM so I was stuck at this point.



To enable the Kerberos events in event viewer you will need to check you have the below registry entry:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters

Registry Value: 
LogLevel

Value Type: 
REG_DWORD

Value Data: 
0x1

If the entry does not exist, create it.



Check the event viewer under system events you should begin to see Kerberos events when trying to authenticate to the problematic site.


Now I could see it was clearly a Kerberos authentication issue and checking the SPNs is the first logical step.

Check the application pool account running the web application, lets say as an example it is: domain\MattTestAccount

For Kerberos to work this account should have SPNs assigned to it.

in command prompt:

setspn -l domain\MattTestAccount

You should then see all SPNs for that service account listed.

If the SPN for your web application is not displayed you have found the issue, and this was the the problem in my case.

How could this have happened when it was previously working?

In my instance, at some point the application pool account running the IIS site had been changed, maybe to make the app pool accounts consistent, maybe to have individual accounts per web app, there are a multitude of reasons but the important thing is you have found the root cause of the issue.

Now you need to set assign the SPN to the correct service account and your Kerberos authentication issue will be resolved.

Guidance on setting SPNs and all other required configuration is in the guide linked at the start of this post.

Thanks for reading,

Matt