Why we are moving away from Visual Studio Online

About 9 months ago, I started experimenting with Visual Studio Online for my company’s source control system. Initial results were very promising – the system required little maintenance (mainly just adding/removing accounts and projects), and was full of great features. External collaboration with customers was a snap, as I just needed to add their Microsoft Accounts – no complicated ADFS or trust setups involved.

However, as a software consultancy working with many different clients (and having to adhere to strict confidentiality and non-disclosure agreements), several points of data leakage in the product made it unsuitable for use in this kind of an environment. We have now performed the migration from cloud back to on-prem using the time-limited export option in VSO. Following are some of the reasons for this move back.

Leaking User Information

The nail in the coffin for VSO was the leakage of User information across projects. Visual Studio Online only provides you with a single Team Foundation Project Collection (called “Default Collection”). Project Collections contain projects, users, and settings, and are a security and isolation boundary very similar to SharePoint Site Collections.

In TFS on-prem, you can create multiple project collections. If you work with multiple customers, you can have a collection per customer, and ensure that you have isolation such that only projects and users from that customer are in the collection.

In Visual Studio Online, all your users and projects are lumped into a single project collection, and this presents an opportunity for one customer to see the other customers you are working with.

I was very shocked to see that the Users tab, which is available for everyone in the collection, shows ALL the users in the collection. I assumed that the list of users would be filtered somehow to the users from the projects you had access rights to, but it is an all-up list, and completely possible for one customer to see the other customers in the system.

Users Tab

The Users tab, showing all collection users and email addresses.

In addition to the Users tab, the Assigned To fields on work items were another source of leakage. When typing in a name in the Assigned To field, it’s possible to assign users from any of the users in the project collection, not just members of the current project The autocomplete will happily resolve the names cross-project.

Once we found this out, this user leakage became a non-starter for us.

Cross-Project Work Item Linking

Since most of our consultants work on multiple client projects, they have access rights across the various projects. When working with Work Items, it is often desired to link work items via Parent-Child relationships. The Add Link dialog enables typing work item IDs, and then seeing a preview of the work item before creating the link. You can type in a work item ID from a different project and create cross-project links. In this respect, Visual Studio Online handles this better – it doesn’t allow you to see the work items from projects that you don’t have rights to access:

Cross Project Work Item Link.

However, it’s possible for our consultants to accidentally create a cross-project (cross-customer) link between work items, which can create a confusing experience for our customers. In the screen below, the linked work item shows as “getting details…”, but it never actually gets the details.

Work Item Getting Details.

It would be better if it told you that you didn’t have rights, like it does when you click into the item:

Work Item Details


While VSO is a fantastic platform, at this time it doesn’t work well as the sole repository for a software consultancy that works with multiple clients, and needs complete isolation between client information.




4 comments on “Why we are moving away from Visual Studio Online
    • Hi Buck. Thanks a lot for posting that link and commenting here.

      There are a couple issues for us with an account per client/customer approach:

      • Only one account can be created per each Microsoft ID. After each one of our developers uses up their account, we’ll have to start creating dummy Microsoft ID accounts just to open up a VSO account. Having these Microsoft IDs to manage doesn’t really cut it in an enterprise environment. We really need single sign on using our on-prem credentials and IdPs so that we don’t have to deal with managing Microsoft accounts manually as people come and go in the organization. We can’t really ask our customers to open these accounts – we need to be in control of the ownership of the tenant and the source code.
      • We will end up paying multiple times for the same users. We only have a certain number of MSDN accounts, which we use for our primary developers, and for the rest, including QA/Project Managers/Offshore resources, we would pay for Basic licenses. Most of these people would need to be on multiple client projects, and so we’d have to mirror these accounts in each tenant (management pain), and the Basic licenses couldn’t be reused across tenants.

      Ideally, if some of the data leakage was re-architected, the single tenant approach would work for us. I know the recommendations for TFS on-prem have been moving towards a single Project Collection anyway. We would love to move back – hopefully you are working towards solving these challenges and a better identity management story with SSO.

  1. Adam, I too had this concern early on, so for my company we set up alias/forwarding email accounts (i.e. client1@…, client2@…, etc.) that all forwarded to our TFS admin. These accounts served as the Owner accounts of our individual client Collections.

    Recently, VSO has allowed a single user to be the owner of *multiple* project collections allowing us, once again, to administer different clients via client1.visualstudio.com, client2.visualstudio.com, etc. Thus, giving us the desired isolation.

    Unfortunately, what’s still lacking is the dashboarding via SharePoint integration that you get with an on-prem of TFS. This feature has been requested since tfspreview.com (circa 2011) and has yet to be enabled. It wouldn’t be very hard. All Microsoft would need to do is install the TFS site templates into SharePoint online (like they already do with Project Server), then we can manually set the Project Portal to a site in O365 SharePoint.

Comments are closed.