To RSVP – http://chicagoalmug.org.

I think we had a lot of questions and a very good interactive discussion – and as a user group organizer, that’s the best part!

Thanks Ed (again) for coming by – we know it was a long day for you – but it didn’t show.

Microsoft has a tool that it acquired – called Team Explorer Everywhere – which allows non Microsoft focused developers, testers, and others the ability to use TFS.

Ed Thompson from Microsoft will be here on April 7th to talk about it. 

http://chicagoalmug.org is the place to RSVP.

The event is at Microsoft Chicago – 200 E Randolph, Chicago IL – 2nd Floor.  April 7th, at 6:30 PM.

This week – January 12, 2011 – Gary Pedretti will be talking about the Architecture/Modeling functionality within Visual Studio 2010.  The event is at 6:30 – 8:30PM at the AON Center (Microsoft) – 200 E Randolph, 2nd Floor.  Registration is closing on Tuesday morning, so if interested, please go to http://chicagoalmug.org and click on the following to sign up:

How Microsoft Developer Division [DevDiv] Does Planning

Traceability – Brian talked a great deal about how the process has improved at Microsoft’s DevDiv since the releases of 2005/2008.  They have a hierarchical set of requirements traceability from core principles down to deliverables.  The high level core principles were called “Pillars.”  With 2010 – the core principles were Quality in Development and Testing.  So you can imagine many of the features that we’ve heard about – lots of them roll up into those two high level things.  While discussing “Requirements Management” – Brian really talks about three key parts of requirements management process:

Elicitation – there is not much support within Microsoft TFS to handle that.  The teams use 3rd Party or other MS applications in this phase still.

Analysis/Definition – the act of translating what was elicited into a solution – of what the applications could be.  Some synthesis and analysis happens here to transform problems into solutions for sign off/buy-in by business stake-holders.

Management – understanding traceability and completeness of the development efforts as compared to the requirements that were documented.

Brian covered a lot of the product realignment and SKU changes – many of which are have been talked about a great deal in Microsoft’s published documentation.

Peanut Butter – In talking about the context of how the process has evolved at Microsoft over the past 10 years, he described an effect where by Microsoft would say – ‘let’s release 2003 or 2005’ and the various distributed teams in the division would go back to their teams and decide what they could get in there, and as a result – the release of a product would consist of a lot of incremental improvements, but there was no real key pillar of why someone should go out and buy the toolset.  I think this has something to do with the way that Microsoft as they’ve grown have evolved to include vision as a key component of the planning.  Many enterprise teams today struggle here as well – and determine what is a success criteria for why a project succeeds or fails.  With 2010 – they clearly took a big look back if you look at all the improvements and understand what are going to be a part of the key business value improvements of the product.  Peanut Butter is a great term to describe this.  i have been involved on a few larger teams where synthesizing product releases took an hour to boil down.  It wasn’t because of distributed walls, but it was due to a lack of message from the beginning in the release.

No Repro Bugs – In talking through the development process there, Brian took a case study of how they improved testing and development quality – by wanting to eliminate “No Repro” bugs.  Those are the bugs filed by testers that the developer just can’t reproduce.  The developer says so and it just goes away – time wasted.  Brian estimated 30% time-waste due to this factor within the Microsoft Dev Div team.  Some of the key features around removing this situation that Brian talked about:

• Video Recording – the tester as they are using Testing Professional [a tool designed explicitly for testers that is not based on Visual Studio] – a video is automatically attached to the newly filed Defect which gets sent to the developer.
• Automatic System Information – Also as defects are automatically filed, the environment [CPU, RAM, OS, Browser] information are all automatically attached to the work item when this happens.
• Intellitrace – While the tester is running through an application – the stack / heap is being recorded, and automatically attached to the bug – the way that Brian described was that this was better than an eventual stack dump/crash dump because it is dynamic and you can open up the intellitrace in an interactive debugger, essentially playing back the entire session of a testing situation to understand the cause that variables are some value or another.  This is like being able to run the application within debugger in Visual Studio – all the time – without actually installing VS on testing environments.  This one feature has definitely been talked about a lot – i was impressed how Brian talked about this one.  I really question any one that says that Microsoft does not innovate – because this really seems an amazing feature that will help countless developers.
• Symbols Server – When a build is deployed and developers can step through the code execution after the fact, the source code that was used for that build can be used to step through.  You don’t have a mismatch between the code a developer has on their local machine and the build that was tested on.  Someone asked if there was a feature to automatically compare that set of code ‘with latest’ – and there wasn’t at this time, but maybe in another release.

Visual Studio Fun Fact #1 – there are 65 Million Lines of Code in VS 2010 today.

Layer Diagram – Apparently, we learned today there was some code in 2005 (TFS) on the server-side [that’s the part that runs in IIS as ASP.NET web services] – that was referring to the namespace System.Windows.Forms [yikes!].  The Layer Diagram could have prevented this by having a team establish limitations on the code and validating that the code written would conform.  I did ask Brian if the BCL (base class library) classes could be used in this diagram – the answer is ‘yes’ there.  I think most demos you see constraints on the application code, so it was nice to see you can enforce architecture for specific BCL namespaces and assemblies.

Multiple Process Methodologies – I think one of the things that I really liked was the fact – and given Microsoft is so big – that just as we as consultants don’t say that one process works for everyone, Microsoft released that this was true – even within the same organization.  I think they mostly have two types of features/deliverables.  Some are called ‘Dark-Tunnel-Features’ and others are called ‘Value-Added-Features.’  Dark Tunnel are those features that once you start, you can’t really stop.  They represented things that you can’t have ‘shippable iterations’ of.  An example of this in VS 2010 was the editor – and it makes sense, Microsoft couldn’t release a half-editor – it’d have to be committed to, or it wasn’t.  For those things – they mostly run those waterfall-esque (Microsoft Project, Phases, etc), for other features, Brian said the teams mostly developed with some Scrum Variant [not the same for all the teams, but each team did things probably a little different in each team].

What Was Not Good About 2010

So – I did ask what could be improved today within DevDiv – and Brian definitely had an answer.  Performance, Performance, Performance.  Here’s the big irony [and Brian also has said this in his blog], that this release of 2010 had more performance focus throughout the entire process than any other release of Visual Studio in history.  Yet when the Beta 2 came out, it was universally panned by the community for having many performance problems.  One problem that was attributing to this was the fact that the team, when developing performance tests, they geared towards making them repeatable, and in an attempt to remove the variations that can happen throughout running the application like [network, disk] and things that you can’t really control.  So in effect the team wrote their performance tests at the ‘unit’ level – small repeatable tests such as (i’m over simplifying):

1. Boot up Computer
2. Open Visual Studio 2010
3. Open Text File [5 lines of code].
4. Compile
5. Save

Those tests were reporting throughout the entire development lifecycle that they were on track.  So – I think – this was akin to ‘writing the test to succeed’ – that can happen in functional unit testing if you’re not careful, or what arguably happens when you’re not writing unit tests in an automated fashion [developers just clicking through the happy path when testing].  Microsoft was planning on reviewing how they do performance testing for future versions of Visual Studio and TFS.

You can read a quick post by Scott Guthrie about the Add Reference dialog here (http://weblogs.asp.net/scottgu/archive/2009/10/29/add-reference-dialog-improvements-vs-2010-and-net-4-0-series.aspx).

I haven’t found a lot of blog postings about this topic so I thought it would be useful to have this point out there.  When a web test is run, it makes a request and then asynchronously – it will follow and request images, stylesheets, JavaScript external files and download them.  During a load test, making these requests (especially back to the server you’re monitoring) are indeed important to keep.  What if you have dependent requests (images/scripts) to external servers that you do not care as much about?  And what if those requests are failing (especially on a network for load testing) that doesn’t allow external requests through.

You can do a few things when you want to ignore those requests:

1. 1.) Set “Parse Dependent Requests” property to false in the request properties screen.
2. 2.) Use a Web Test Request Plugin which can ignore selective requests.

Option 1 may work just fine especially if you don’t have a lot requests coming back to your servers and mostly have the external references that you do not care about.  Option 1 may be a very bad thing – especially if your dependent requests are in fact (in large part) going back to the server you’re monitoring (you want those requests to be made still).  But if you want to just ignore one or two of many – Option 1 may not be the best choice to make.

Option 2 – this requires very little code and can save a lot of time and help to ensure your load test statistics are accurate.  The downside is that it’s a little bit extra code that you have to support.  It’s very trivial though, I should add.

How the Web Test Request Plugin Fits In:

Before getting into the code, let’s first have a quick look at where events are fired and how you can use the web test request plugin (at least from the perspective of this goal).  There are two events that you can override in a Web Test Request Plugin:

• • • • - PreRequest
      • - PostRequest

In the above flow, the Parse Dependent Requests happens after a response is returned from the web server.  Thereafter, Web Test will asynchronously make those dependent requests.  Therefore, below overlayed with the process is where you can inject functionality through the web test request plugin:

Therefore, the PostRequest method is the place to perform our override.  This will happen before the dependent requests have been made to the server, but after the first request has been made.

Below is some simple code with this process included.

class DependentRequestsFilter : WebTestRequestPlugin
    {
        private string _excludeStrings;

        public string ExcludeStrings
        {
            get { return _filter; }
            set { _filter = value; }
        }

        public override void PostRequest(object sender, PostRequestEventArgs e)
        {
            List<WebTestRequest> requestsToRemove = new List<WebTestRequest>();
            foreach (WebTestRequest request in e.Request.DependentRequests)
            {
                foreach (string excludeUrl in ExcludeStrings.Split(new char[] { ','} ))
                {
                    if (request.Url.ToLower().Contains(excludeUrl.ToLower()))
                    {
                        requestsToRemove.Add(request);
                    }
                }
            }

            foreach (WebTestRequest requestToRemove in requestsToRemove)
            {
                e.Request.DependentRequests.Remove(requestToRemove);
            }
        }

A few things to point out.  If you make a public property on your plugin, you’ll see that when you add that plug into your web test.  The Visual Studio Property window recognizes it and allows entry.  Secondly, notice that I split the ExcludeStrings with a comma… that’s to allow multiple filters.  Sometimes in web tests, you may have 10s or 100s of requests and several that need to be filtered out.  This is of course for demo purposes only, but it could be a little more robust with a check for null in there.