TFS Hosted Build Controller and .NET 4.7

If you want the TFS hosted build controller to run .NET 4.7 builds, make sure to change the default agent queue to the "Hosted VS2017" version. You can do this by editing the build definition and in the Tasks Process landing page, it is the second option, as shown below.

Getting all changesets associated to work items

I had a need today to pull a list of all check-ins that had been associated to a certain list of work items. This can be done easily using the TFS API assemblies, most of which are located in C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\ReferenceAssemblies\v2.0. The code below will write the list of all file changes across all changesets for the specified work item IDs.

using (TextWriter tmp = Console.Out)
            {
                using (var fs = new FileStream("Test.txt", FileMode.Create))
                {
                    using (var sw = new StreamWriter(fs))
                    {
                        Console.SetOut(sw);

                        var workItemIds = new int[] { 1,2,3 };
                        var collectionUri = new Uri("http://yourtfs/server/");

                        try
                        {
                            using (var tpc = new TfsTeamProjectCollection(collectionUri))
                            {
                                var workItemStore = tpc.GetService<WorkItemStore>();
                                var teamProject = workItemStore.Projects["project-name"];
                                var versionControlServer = tpc.GetService<VersionControlServer>();
                                var artifactProvider = versionControlServer.ArtifactProvider;
                                var workItems = workItemStore.Query(workItemIds, "Select [System.Id], [System.Title] from WorkItems");
                                var allChangesets = new List<Changeset>();

                                foreach (WorkItem workItem in workItems)
                                {
                                    allChangesets.AddRange(
                                        workItem.Links.OfType<ExternalLink>().Select(link => artifactProvider.GetChangeset(new Uri(link.LinkedArtifactUri)))
                                        );
                                }

                                var orderedChangesets = allChangesets.OrderByDescending(c => c.CreationDate).ToArray();
                                foreach (var changeset in orderedChangesets)
                                {
                                    Console.WriteLine("{0} on {1:MM-dd-yyyy HH:mm} by {2} ({3} change(s))", changeset.ChangesetId, changeset.CreationDate, changeset.Owner, changeset.Changes.Length);
                                    foreach (var change in changeset.Changes)
                                    {
                                        Console.WriteLine("    [{0}] {1}", change.ChangeType, change.Item.ServerItem);
                                    }
                                    Console.WriteLine("-----");
                                    Console.WriteLine();
                                }
                            }
                        }
                        catch (Exception ex)
                        {
                            Console.WriteLine(ex.Message);
                        }

                        sw.Close();
                    }
                }
                Console.SetOut(tmp);
                Console.WriteLine("Press any key to exit...");
                Console.ReadKey();
            }

Branching

I had a conversation with a co-worker yesterday about when to create branches in TFS and the conversation reflected a confusion surrounding the use of branches. The coworker suggested that a new branch should be created each time code was being pushed to any environment because it’s the only way you can be completely sure the branch isn’t polluted.

To get the obvious out of the way, if a source control system “pollutes” a branch with no human intervention, get a new source control system. The system has failed at its most basic task. It is very likely this is not the case. It is probably how you are executing your branching and merging strategy.

Let’s take a typical branching structure: Dev > QA > Prod; Dev is the parent of QA, which is the parent of Prod. Changes are merged from Dev into QA, then from QA into Prod. You should never get merge conflicts when going from Dev to QA, or QA to Prod. Because of this, merging is clean and no “pollution” can happen.

How is this possible?

The only way a merge conflict happens is when a change has occurred in the target branch you are merging into which has not been integrated in the source branch. But - and this is the critical point - if you are following good merging practices, if a change must be made in QA or Prod, it is immediately merged into the parent branch(es). No exceptions! If I make a change in the QA branch, my next immediate check-in is a merge to the Dev branch from QA. I will resolve any merge conflicts with this merge, ensuring that my change is properly integrated in Dev. The next time Dev is merged into QA, it will already have this change, and so no merge conflicts will occur.

Crossing Domains without Password Pain

I work in consulting, and as such, my machine is never on the client domain.  This causes some headache because every time I want to connect to a client resource, I am prompted for my client username and password.  By default, the client’s resources are in the Internet zone, so nothing is trusted, nor are my saved passwords used to authenticate – I am asked to reenter my password each time I connect to any resource.  Fortunately, there is an easy solution – simply add the domain as a Local Intranet site in your Internet settings.  Under Internet Options, Security, Local Intranet, Sites, Advanced, you can enter the paths to the client resource that you commonly access.  Once this is done, after you check the “Save Password” option when you access a resource again, you won’t have to reenter the password again.

If the client has a TFS instance, you can save your username and password used to access it by going to the team web access site in Internet Explorer and saving your credentials when prompted.  When you launch Visual Studio and connect to that TFS instance, it will then use your saved credentials.

Team Foundation Server 2010 Security Practices

A coworker asked me to point him to the location where to find the documents on TFS Security Best Practices and I realized they don’t exist!  There are a number of blog posts with people describing how they have configured TFS, and there are some documents on MSDN that provide the permissions and roles that are available in TFS, but I didn’t see anything that provided a good overview and guidance on setting up and managing TFS security.  So, I will briefly cover the surface area of TFS security, a couple of tools that will help make security management a bit easier, and then some practical recommendations on configuring security.

TFS Security

Managing security in TFS involves managing security to three different components: TFS itself, WSS or Sharepoint, and Reporting Services.  Each of these has different access levels, roles, and permissions, so it isn’t as simple as a single selection.  Out-of-the-box, a team project will be compromised of the following roles within the different application components:

Team Project Roles	Sharepoint Roles	Reporting Services Roles
Project Administrators Contributors Readers Builders	Full Control Design Contribute Read	Content Manager Browser

Straight from the MSDN documentation on the Team Project roles:

• ProjectName\Project Administrators Members of this group can administer all aspects of the team project, although they cannot create projects.
• ProjectName\Contributors Members of this group can contribute to the project in multiple ways, such as adding, modifying, and deleting code and creating and modifying work items.
• ProjectName\Readers Members of this group can view the project but not modify it.
• ProjectName\Builders Members of this group have build permissions for the project. Members can manage test environments, create test runs, and manage builds.

Additionally, access to TFS can be granted at three primary levels – server, collection, and project.  You will very rarely assign permissions at the server and collection level, although the latter is a little more common.  Typically your TFS Admin account(s) and service accounts will be assigned at the server level, and your collection administrators will be assigned at the collection level.  The bulk of the role and permission assignments happen at the team project level.

TFS Security Tools

Fortunately, you don’t have to manage the permissions to all three locations separately.  TheTeam Foundation Server Administration Tool is a free, open-source, tool that consolidates this into a single interface.  The other tool that I have found extremely useful for viewing and managing user workspaces and content is the Team Foundation Sidekicks tool from Attrice(also free).  Note that both of these tools are specific to a team project; you can’t use either to manage the server or collection membership.

Configuring Security

Since TFS is built on top of and integrates nicely with Active Directory, it is strongly recommended to use Active Directory groups to manage access to TFS.  In fact, I’ll go so far as to say unless it is impossible to do so, manage all access to TFS through Active Directory groups.  I think you will find this can be done in almost all of the cases.  This will radically simplify the surface area of what must be managed.  As a practical note, typically a new hire will need to be added to certain A/D groups; why not include the appropriate TFS A/D groups in the “new hire ticket” as well?  In addition, by configuring access to an Active Directory group, granting users the same set of permissions involves simply adding them to that group, rather than duplicating the entire subset of permissions.

• Use Active Directory groups to manage TFS access

The other strong recommendation I would make is that access to TFS should reflect the structure of the project teams.  Since we are remaining with A/D groups, this means my A/D group structure that provides access to TFS should reflect the team and project structure.  I’ll also add that it will be very helpful to create a tree structure of groups so that permissions will roll up.  I will talk more of this as we walk through some scenarios.  My primary push for this is to encourage you to work with your Infrastructure people to get groups defined in such a way that reflects your teams and projects, if possible.  Usually they like simple and easy-to-manage too!

• A/D groups for TFS access should reflect team and project structure.

Let’s cover project collections and team projects.

When would I consider a new project collection?

The primary limitation to keep in mind with team project collections is that the artifacts are not accessible across collections. You cannot access work items, source, builds, etc. across collections. If your team or organization does not integrate with other teams and has very little if any dependencies, a collection may make more sense. However, keep in mind – moving a team project to a different collection is not supported. There are a variety of reasons for this – within each collection, work item IDs, changeset numbers, builds, etc. all start from 1, so you would have major clashes if you tried to consolidate collections. You might consider project collections at a department level – perhaps HR, Internal IT, and Products each have their own collection.

When would I consider a new team project?

A team project can provide a way to encapsulate security, code, and project management activities for a team or project. If you are leveraging Sharepoint and keep the option checked, TFS will create a new Sharepoint site when the project is created. Items across team projects are visible to other team projects within a collection, if security allows. This includes work items, builds, source control, and reports. You might consider team projects when you are creating a product, starting a new project that falls within existing department, or want to track activities that fall within a distinct bucket. Keep in mind that Areas and Iterations can be used within team projects to provide some segregation of activities, so you may not need a new team project if it is just an extension of an existing project or effort.

Now that we have the above two ideas defined and some collection versus team project recommendations, let’s walk through some practical scenarios.  These are based on several different organizations that I’ve been a part of and the work I did in managing TFS access for each organization.  My convention is to use a “TFS.” prefix on the A/D group name so that it is clear that it is for TFS access.  However, this can also be done via OUs in A/D, depending on your Infrastructure preferences.

Scenario 1: Multiple Departments, Multiple Products, Not-Shared within an IT organization

For our first scenario, let’s take an IT department that contains multiple teams, where each team works on multiple, independent internal company products.  It is very unusual for the teams to share resources and thus any sharing is considered a transfer to a new team.  Each team is cross-functional, containing QA, dev, BA, manager, and architect resources.  The access requirements for these resources are the following:

• Architects and possibly team leads should have Project Admin rights, and are the ones responsible for builds.
• Developers should be Contributors.
• QA, BA, and managers should have read-only rights to the source control, but be able to manage work items, iterations, areas, etc.
• The PMO office (executives) want to run reports and read access to source is fine.

We might create a group structure that looks like the following:

A/D Group Name	TFS Team Project Permission	Members
TFS.All	–	All TFS.* groups
TFS.ProjectName	–	All TFS.ProjectName.* groups
TFS.ProjectName.Admins	Project Administrator	Architects, maybe leads
TFS.ProjectName.Developers	Contributor	Developers
TFS.ProjectName.Editors	Work Item Editor	QA, BAs, Managers
TFS.ProjectName.PMO	Reader	PMO office

The Work Item Editor role is a new TFS security group that we will create for the project, with the permissions for that role granting view and edit on work items, iterations, and areas, but read only on source control.  Note that we create a group that contains the other groups – this makes it easy to do team level activity, like email everyone on the team, regardless of role.  If you need to separate the Editor group into separate QA, BA, and Manager groups, that can be easily done as well.

Scenario 2: Multiple Departments, Multiple Products, Shared within an IT organization

For our second scenario, let’s modify one thing: the developer and QA resources within the organization are shared across projects, but the team leads, managers, and BAs are dedicated consistently to a project.  In this case, we might do something like:

A/D Group Name	TFS Team Project Permission	Members
TFS.All	–	All TFS.* groups
TFS.ProjectName	–	All TFS.ProjectName.* groups
TFS.ProjectName.Admins	Project Administrator	Project leads
TFS.ProjectName.Editors	Work Item Editor	Managers and BAs
TFS.QA	Work Item Editor for all dev projects	QA
TFS.Developers	Contributor for all dev projects	Dev, Architects

Once again, you can separate the devs and architects, QA and BAs, if needed.

Scenario 3: Single Department, Multiple Products

For the third scenario, we just have a single department with multiple products (and thus I am assuming it is the same team that works the products).  You can create each product as a separate team project, and in some cases, this can make sense.  The big advantage to having the different products within the same team project is that you can roll everything up under a single project, so it makes resource availability, tracking, and iteration planning simpler.  Since this scenario assumes the same team, the groups don’t need any team or project designation:

A/D Group Name	TFS Team Project Permission	Members
TFS.All	–	All TFS.* groups
TFS.Admins	Project Administrator	Project leads
TFS.Managers	Work Item Editor	Managers
TFS.Editors	Work Item Editor for all dev projects	QA and BAs
TFS.Developers	Contributor for all dev projects	Dev, Architects

Scenario 4: Multiple Teams, Multiple Products, Some Common, Some Team-Specific

For the fourth scenario, let’s take the same idea of multiple teams and products, but vary it slightly by stating that each team manages multiple products, and there are some projects that are common across all the teams.  In this instance, we might create team specific A/D groups and then assign them to the respective product-specific team projects and any common team projects for which they need access.  I’ve found it is helpful to create a team project named Common or Shared that contains the shared artifacts across the teams, so that it is clear that it is shared.  If you want to segregate it further (have multiple shared team projects), consider naming the common team projects with a Common or Shared prefix, such as “Common-Architecture”, “Common-Services”, etc.

A/D Group Name	TFS Team Project Permission	Members
TFS.All	–	All TFS.* groups
TFS.TeamName	–	All TFS.TeamName.* groups
TFS.TeamName.Admins	Project Administrator for team-specific and common projects	Project leads
TFS.TeamName.Developers	Contributor for team-specific and common projects	Developers
TFS.TeamName.Editors	Work Item Editor for team-specific and common projects	Managers, QA, BAs

You can of course separate the common project access into its own security groups, if needed (TFS.Common.Admins, TFS.Common.Developers,…).

Hopefully this helps as you define your TFS security model.

Essential TFS Process Template Modifications

Out-of-the-box TFS 2010 ships with two very solid process templates, and you can install the Scrum 1.0 template and the Scrum for Team Systems 3.0 templates as well.  However, there are two template modifications in the Agile template that are very useful.  The first is to modify the Assigned To field to exclude some of the TFS service accounts and duplicate values.

<FIELD name="Assigned To" refname="System.AssignedTo" >
<ALLOWEDVALUES expanditems="true" filteritems="excludegroups">
<LISTITEM value="[Project]\Project Administrators"/>
<LISTITEM value="[Project]\Contributors"/>
<LISTITEM value="[Project]\Not Assigned"/>
</ALLOWEDVALUES>
<PROHIBITEDVALUES expanditems="true">
<LISTITEM value="Project Administrators"/>
<LISTITEM value="Contributors"/>
<LISTITEM value="Not Assigned"/>
</PROHIBITEDVALUES>
<DEFAULT from="value" value="Not Assigned"/>
<ALLOWEXISTINGVALUE/>
<HELPTEXT>..snip..</HELPTEXT>
</FIELD> 

The other two modifications are to create initial states in both the Task and Bug work item types.  This can be done by adding the states in the work item type editor, editing the workflow to add the states, and then republishing the work items back to the server.  The flow on a bug would be modified to Created > Active > Resolved > Closed and the Task flow would be modified to a flow of Created > Active > Closed.

TFS 2010, Build Definitions, and Powershell

I needed to get a list of build definitions from a TFS 2010 instance as we are planning to migrate to a new server and will not be moving the collection, but only the builds and source files inside the collection. The following Powershell scripts will list out the build definitions for a server:

clear-host

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.WorkItemTracking.Client")


$tfsCollectionUrl = "http://{server}:8080/tfs/{collection}"
$server = new-object Microsoft.TeamFoundation.Client.TfsTeamProjectCollection(New-Object Uri($tfsCollectionUrl))
$buildServer = $server.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$workStore = $server.GetService([Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore])

$workStore.Projects | ForEach-Object {
Write-Host ("Project: "+$_.Name)
$buildServer.QueryBuildDefinitions($_.Name) | select-object {$_.Name, $_.ContinuousIntegrationType} | format-table
}

BRD Lite Rangers HOL–Web deployment task failed

Going through the HOL Lab for the BRD Lite Team Build process template, I received the error “Web deployment task failed” after changing the connection string to remove the ‘sqlExpress’ instance.  Yet it still failed, even when I changed it to run under the Admin account.  After turning on the Diagnostic detail, it turned out to be a different project issue:

This was because I had to change the connection string in the web project properties under the Package/Publish SQL tab.  Once I removed the ‘sqlexpress’ value from the Data Source, my build succeeded.

TFS 2010 SP1 Installation Issue–TFS_SCHEMA_VERSION

I just completed applying TFS SP1 our production TFS server.  The installation experience was smooth, but I ran into one problem that caused some minor panic and about an hour of research.  The install succeeded, the server restarted, but the TFS Job would not start up, and when trying to connect to TFS via Visual Studio, it threw an error about a mismatch of TFS versions in the database.  In the error log, the following was being logged:

TF53010: The following error has occurred in a Team Foundation component or extension:
Date (UTC): 23/05/2011 23:59:18
Machine: --removed--
Application Domain: TfsJobAgent.exe
Assembly: Microsoft.TeamFoundation.Framework.Server, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; v2.0.50727
Service Host: 16ff6530-41b7-4803-8474-7ff80ca3c33f (--removed--)
Process Details:
  Process Name: TFSJobAgent
  Process Id: 2056
  Thread Id: 1584
  Account name: --removed--

Detailed Message: There was an error during job agent execution. The operation will be retried. Similar errors in the next five minutes may not be logged.
Exception Message: The requested schema property TFS_SCHEMA_VERSION did not match the expected value. The server requires the Microsoft Team Foundation Server 2010 (SP1) schema but the database currently implements Microsoft Team Foundation Server 2010 (RTM). (type DatabaseSchemaException)

Exception Stack Trace:    at Microsoft.TeamFoundation.Framework.Server.TeamFoundationDatabaseSettings.ValidateDatabase(TeamFoundationRequestContext requestContext, String connectionString, String expectedSchema)
   at Microsoft.TeamFoundation.Framework.Server.FrameworkSqlResourceComponent..ctor(TeamFoundationRequestContext requestContext, String databaseCategory)
   at Microsoft.TeamFoundation.Framework.Server.TeamFoundationJobService.AcquireJob(TeamFoundationRequestContext requestContext, Guid agentId, TeamFoundationJobQueueEntry jobToStart)
   at Microsoft.TeamFoundation.Framework.Server.JobRunner.AcquireJob(TeamFoundationRequestContext requestContext, TeamFoundationJobQueueEntry queuedJob)
   at Microsoft.TeamFoundation.Framework.Server.JobApplication.CheckJobQueue()
   at Microsoft.TeamFoundation.Framework.Server.JobApplication.ProcessJobQueueInternal()
   at Microsoft.TeamFoundation.Framework.Server.JobServiceUtil.RetryOperationsUntilSuccessful(RetryOperations operations)

Upon opening SQL Management Studio and right clicking the Properties of the appropriate TFS collection database, sure enough, the version was still contained the value “RTM”.  I changed the value to “SP1” (highlighted below), started the TFS Job service, and the collection servicing started immediately.  Within 5 minutes, the server was up and fully operational. 

I don’t know why this had to be done, but there it is.  Hope this helps others who might run into this problem.  Thanks to Stackoverflow for the answer to this.

Installing TFS 2010 on a SQL Instance

When installing Team Foundation Server 2010 on a SQL Instance today, I ran into a couple of issues that are related to the SQL Browser service.  If the SQL browser service is running, you only need to specify the server and instance name, as “sqlservername\instancename”.  If the browser service is not running, you should specify the server name and port number, as “sqlservername, portnumber”.  The value in the screenshot below did NOT work until I removed the “\tfs” value.

TFS Build Servers

By default TFS 2010 installs the build server with the fully qualified domain name as part of the path.  In our environment, this caused the build servers to be unreachable.  You can change this by going to TFS Admin Console on the build server, stop the build service, open the properties window, and remove the domain name from the path:

Configure Team Build 2010 to use a Proxy Server

If you want to set up the TFS 2010 Build server to use the TFS Proxy server, you’ll need to log in with the build account on the server and modify the registry with the following entries:

[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0\TeamFoundation\SourceControl\Proxy]

"Enabled"="True"

"Url"="http://proxyservername:8081"

These entries are set when you use the Visual Studio UI to configure proxy usage, so if you have Visual Studio installed on the build server, you can configure it through the VS settings dialog.

TFS and Reporting Services Access Issue

After installing and configuring Team Foundation Server 2010 with Reporting Services, I was unable to access the Reporting Services instance from any other box besides the server itself, regardless of the account used (even the setup account was denied).  I would hit the host/reports site and would be prompted for a username and password and even when putting valid credentials in, the site would still deny me. The solution ended up being to remove the Windows Negotiate authentication option from the RSReportServer.config file.  This file is located at %Program Files%\Microsoft SQL Server\INSTANCENAME\Reporting Services\ReportServer.  Remove the RSWindowsNegotiate node under Authentication/AuthenticationTypes.

Migrating TFS 2008 from SQL 2005 to SQL 2008 (R2)

I have been tasked with upgrading our TFS 2008 instance to TFS 2010.  However, our current environment is a TFS 2008 dual-server deployment running on a SQL 2005 database server, and TFS 2010 requires SQL Server 2008.  In order to upgrade to TFS 2010, our current TFS 2008 instance must be migrated over to run against a SQL 2008 database server.

After quite a bit of searching and reading a variety documentation, I came to the conclusion that there doesn’t seem to be an official Microsoft document that describes this process.  The closest thing to this was the endorsement of this article by several of the TFS folks.  I highly recommend reading this article, as it is about 99% of what you’ll need to do in order to have a successful migration.

There is one gotcha that is very important or the process will fail when you try to move the TFS databases.  You must install a TFS 2008 with SP1 integrated on a separate (or same) server configured to point to the new SQL 2008 database server to “prep” the database server and make sure all the necessary TFS components, jobs, and security roles are set up correctly.

Thus, before starting the process as described in the article, first create a new TFS 2008 instance with the database on the SQL 2008 instance you will be migrating too.  Then you can begin following the instructions in the referenced article.

In addition, I unprovisioned WSS from the application tier server and re-provisioned it (by running the Sharepoint Products and Technologies Configuration Wizard), and then created a new web application on port 80, and then restored the content database and configured it to use the restored content database.  I wanted to move all WSS databases (Admin, Config, and Content) and simply moving the Content was not sufficient.  Note if you do this, you’ll probably need to reinstall the WSS extensions for TFS as the last step.

One final comment – we used SQL Server 2008 R2 and the migration worked fine.  Just make sure your TFS install has SP1 integrated.

Other links I found helpful:

• Moving TFS from hardware configuration to another
• Moving Sharepoint Content databases
• Rebuilding the Team System Cube
• Integrating TFS 2008 and TFS 2008 SP1 installs

Creating TFS 2008 Build Server

The base install used for setting up a TFS 2008 Build server was Windows 2008 R2 Standard.

• Install Visual Studio 2008 Developer Edition, custom install, with only the Visual Web Developer and Team System Developer tools

• Install TFS 2008 Team Explorer
• Install Visual Studio 2008 Tester Edition, custom install, with only the Team System Tester Tools.

• Install VS 2008 SP1.
• Install the Web Deployment Setup.
• Install VS 2008 DB Pro GDR Update.
• Install Team Build.
• Install TFS SP1.

For .NET 4.0 Builds, I also installed:

• .NET 4.0 Framework
• Visual Studio 2010 Ultimate with the Web Development tools.