SharePoint 2013 - Central Admin site prompts for authentication over and over and fails to load from remote server

Noticed something on a recent 2013 farm build where Central Administration site opened correctly from the server it was running on, but when you tried to open it from another server in the farm or a workstation you were prompted for authentication three times and it never succeeded.

This seemed like a behavior I have seen in the past when working with Kerberos based web applications that did not have the SPN's configured correctly. I checked the authentication provider on Central Admin and found it was set to Kerberos instead of NTLM. Since the client is not utilizing Kerberos I changed to NTLM, saved changes, and after this could access CA from other servers in the farm.

I looked again at my script and realized that I had not set the authentication provider switch correctly so this is why it had been set to Kerberos

Microsoft SQL Server 2012, 2008R2, 2008, 2005, 2000 and 7.0 Builds

Ran across this site and it has all the needed SP build numbers for multiple versions of the product. Also has some other helpful links on how to get build numbers, etc.

Microsoft SQL Server 2012, 2008R2, 2008, 2005, 2000 and 7.0 Builds
http://sqlserverbuilds.blogspot.com/

March and April CU for 2013

It seems that the CU's for 2013 are coming quicker than they did for 2010. The March CU was launched and now has to be installed in order to apply any future updates.

April 2013 CU also released and looking through the details for SP Server it appears that most of the issues are around sites and service applications that were migrated from 2010.

March CU
http://blogs.technet.com/b/stefan_gossner/archive/2013/03/21/march-public-update-for-sharepoint-2013-available-and-mandatory.aspx

April CU
http://blogs.technet.com/b/stefan_gossner/archive/2013/04/27/april-2013-cu-for-sharepoint-2013-has-been-released.aspx

Hopefully we don't see the same issues that we did with 2010 where CU's are released, pulled, and then released again.

Passed 70-668 for MCITP

I finally sat the 70-668 SharePoint 2010 Administration test this past Thursday. I have been working with 2010 for a couple years now, but was still nervous about sitting the test as I am just not a good test guy. I was very pleased when It was all over and I passed with an acceptable score.

Now its time to start working toward the first dev test.

My Install of SharePoint 2013 on Windows Server 2012 VM

I decided to dive into Windows Server 2012  and SharePoint 2013 this weekend as I had downloaded the software ofr each a while back and wanted to get started. I am using Vmware workstation 8 for this.


Windows Server 2012 Installation

I created a new VM, selected Windows 7 x64 in Vmware, set the ISO to my OS ISO and powered up the VM.

I ran into an issue almost immediately when booting up from the iso.The error was

Windows Cannot Read The ProductKey Setting From The Unattend answer file

When you click OK, the VM would just reboot.

I did a bit of googling and found the solution here

The fix was easy, just uncheck the power on, or turn off the floppy settings in the VM. I chose to just turn off the floppy drive. After this the installation routing ran as I would have expected.

The installation time was quick, seems like it only took 30 or 40 min to complete the installation.

I had not gone through any 2012 docs prior to this install, so when I logged on for the first time and loaded up the GUI it was a bit of a surprise. No start menu, icons, or anything else that looked familiar. I poked around server manager a bit and back to google I went.

I found that in order to get to some desktop type icons that another feature had to be installed "Windows Desktop Experience", so I installed this feature and after reboot had some tiles on the screen for things like control panel, IE, etc. I then found an article here that goes over the various keyboard shortcuts for Windows 8 and they seem to work in 2012 as well.

So at this point I had at least a base install up to begin digging into and to try and install SharePoint 2013 on.

Share Point 2013 Installation

I started with the prerequisiteinstaller package and everything installed and correctly, after a couple reboots it was all wrapped up.

I performed a standalone installation as all I want to do at this point is go through the installer and have a look at some of the interface and feature differences.

The installation of the binaries went fine, but it did error out when it kicked off psconfig with the error below

ErrorCode:SubStatus:
Service running under Network Service account in workgroup environment is not supported.

Back to google and found the solution to the issue here, I only had to change the app fabric service account and did not encounter the TCP port in use error. I then ran

PS C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\BIN>
.\PSCONFIG.EXE -cmd Configdb create SkipRegisterAsDistributedCacheHost

and this completed with no errors.

When I then proceeded to run psconfig from the GUI as the article mentions, it kicks off, but then fails at the end with an error on installing "sample data". I looked this up and found that this does not stop the completion of the configuration, so for now i just ignored it.

I launched Central Admin and the new CA interface loaded up. Yet again the look and feel have changed further and I see some new options that are not in 2010.

Even with the few errors encountered I had my new VM built in under 2 a couple hours and up and running.

April 2012 Cumulative Update WOES - (Update Pulled)

If you have already downloaded the April CU, STOP, do not install it. If you haven’t yet, well don’t worry because it was pulled

Overview:

I have been working with Microsoft over the last two weeks on issues we were experiencing after installing the April 2012 CU in all of our production environments.

We started receiving reports the Monday after installing the update that document libraries and lists were no longer present on some sites. There were also reports of icons changing for document libraries to link icons.

At first with this type of request you immediately think that this has to be some type of user problem. So I investigate the report and find that the item they are complaining about is in the recycle bin.

So I figure the user just deleted the list on accident. So I try to restore the list and receive an error that the list already exists. I go looking through the site via gui and cannot find this list anywhere. I dropped to power shell and could not find the list that way either.

The next twist is we pulled an audit report for this list in question and find that users were making changes in the list on Friday. The recycle bin shows this list was deleted on Tuesday. So how does a user update content in a list that has been deleted.

By this time we are now having multiple reports coming in from different farms and site collections of the same behavior. Only now we are seeing this behavior reported on lists, document libraries, and even sub sites.

So we are seeing the following behavior on our farms

Missing Items - Document libraries, Lists, Sub sites disappear from view on sites. Cannot view them in gui or powershell

Icons Change - On link lists the icons would change to a link icon, when you click on the link you receive a 404 or 503 error and content fails to load. When I go to the list/library settings page for that list the URL for the item is incorrect

Investigation:

I confirmed that reported sub sites just threw a 404 error like they have been deleted. We find on some items a 404 is thrown, some throw http 500, and then some throw an unexpected error with correlation id.

I dug a bit further into the issue and started finding that the items affected by these issues had been deleted from the site and then recreated with the same name. The deleted item was still in the 1^st stage recycle bin.

If I try to restore the item in the recycle bin it throws an error that the item already exists. Problem is that you cannot see the item to delete it. Trying to delete from powershell failed as well as it could not find the item.

There were no errors in the upgrade logs from the upgrades or any further evidence that I could find related to these missing items.

I contacted Microsoft on this issue as this clearly looks like an issue with the CU.

We spent the first couple days with different engineers going through the typical info gathering process.

We were not making much progress and it seemed it was trying to be directed at customizations we had in our environment.

My colleague remembered a strange issue in Sharepoint 2007 where permissions would get hosed and the item would not appear due to having no permissions assigned.

I looked into the broken lists and found that they did have broken inheritance on them. So to test this theory I took a backup of one of the broken sites prior to the update being applied and restored into one of our test environments.

The list that was missing in production was present on the restored site.

I then set inheritance on this item in question back to inherited, and ran the upgrade on the content database.

This time after the upgrade was complete the list in question was still visible on the site

Steps to Repro

· Take a copy of the affected database prior to the April CU

· Stand up the site in a test environment that has the April CU installed

· Attach your DB copy via central admin so upgrade process is not run

· Browse your site and you will see all data you expect

· Now perform the upgrade-spcontentdatabase on the content db, once this is done you will see that your document library is no longer on the site.

After working through the issue for several more days with various escalation engineers we finally were able to get them to admit a problem in the CU.

I worked with an engineer, Sojesh Sreelayam, who had seen some similar behavior before in an unrelated case. I name him specifically as he was the tech that figured out the issue we were having and came up with the solution.

So he started going down the path of looking at the scope id's and perms table for the affected items and confirmed this was in fact the same behavior he saw before..

The problem ended up being that the scope ID had been removed from the perms table in the content database for the affected items. To resolve they had to restore the scope ID to the perms table for each content database on the affected items. In order to do this we had to have the databases restored prior to the updates. Then the missing data was pulled from the database prior to it being broken.

There were some additional items found where items in a library that were not in the recycle bin were affected and were no longer visible on the site. These could not be repaired in the same way and required additional steps to be taken to correct the files.

Resolution:

In the end, to resolve this issue it required SQL scripts to be run against the production content databases.  All direct database edits were done by the MS tech and were approved form the product team to resolve the issues we had.

I have never been involved with this type of issue related to a CU or any other update for that matter. It required a large amount of time and a crit sit case with MS to resolve, but I will say that once we were able to get them to agree that it was caused by the CU things moved along much quicker and they resolved all identified issues.

Whether we have further items reported is yet to be seen, but as of today the case is closed and it’s on to the next item.

Interesting thing a business analyst sent me a few days ago, http://bjarnegram.wordpress.com/2012/05/29/sharepoint-2010-cu-april-2012-retracted/

The April CU has been pulled due to technical problems - http://support.microsoft.com/kb/2598151

Insufficient Database permissions – proc_GetProductVersions \ EventID 5214

We started seeing these strange event log entries on our web front ends.

“Insufficient SQL database permissions for user ‘Name: <user id> SID: <GUID> ImpersonationLevel: Impersonation’ in database ‘SharePoint_Config’ on SQL Server instance ‘<DB Server>’. Additional error information from SQL Server is included below.
The EXECUTE permission was denied on the object ‘proc_GetProductVersions’, database ‘SharePoint_Config’, schema ‘dbo’.”

I ran across this issue in our environment and found the info on what causes the error here, http://sharepointsoldiers.wordpress.com/2011/10/09/insufficient-database-permissions-proc_getproductversions-eventid-5214/#comment-30.

Thanks to the author for opening the case with MS and finding root cause.

I installed the April 2012 CU,  in one of our test environments in preparation for an upcoming deployment and found that this issue is resolved by this CU. Not sure if it was fixed in the prior Feb CU or not, but this one fixed it for us.