Managed World

Techno-babble from yet another babbler RSS 2.0
# Thursday, June 12, 2008

In this post I want to discuss with you the importance of realizing how lambdas work (and why you should care). Let's dive right in with some code.

Given the following code snippet, what would you expect the output to be (no cheating :P)?

            var actions = new List<Action>();

            for (int i = 0; i < 10; i++)
            {
                actions.Add(() => Console.WriteLine(i));
            }

            foreach (var action in actions)
            {
                action();
            }

Would you believe me if I told you this is the output you get?

10
10
10
10
10
10
10
10
10
10

At first glance, one might expect this output:

0
1
2
3
4
5
6
7
8
9

But all tens are output instead. Why does this happen? Let's crank open Reflector and find out why...

 

The first thing you'll notice is that the compiler has created a helper class to enable the closure we have. This helper class created by the compiler contains a local variable that we use to iterate over and a method that our delegate is contained within. This helper class is especially interesting because it is where the magic happens.

 

[CompilerGenerated]
private sealed class <>c__DisplayClass2
{
    // Fields
    public int i;

    // Methods
    public void <Main>b__0()
    {
        Console.WriteLine(this.i);
    }
}

Rather than diving into MSIL, let's look at some pseudo-code of what the compiler _actually_ executes with the original code above (based on the generated MSIL):

 

var actions = new List<Action>();

<>c__DisplayClass2 localFrame = new <>c__DisplayClass2();
for (localFrame.i = 0; localFrame.i < 10; localFrame.i++)
{
	actions.Add(localFrame.<Main>b__0);
}

foreach (var action in actions)
{
	action();
}

 

Perhaps now you can see where the problem is. The "problem" in our original code exists because of the scope that our closures are defined. The local index from our for loop is now stored in our helper class <>c__DisplayClass2. And the code that is executed by the action is contained within the compiler generated <Main>b__0() method now. So the Console.WriteLine() method now uses the local variable from <>c__DisplayClass2 when it is executed.

 

So while we are looping through building all our actions, we are also incrementing the property i in localFrame (an instance of <>c__DisplayClass2). Then at the end when we are actually executing the actions, the <Main>b__0() is called and uses the local i property (which by this time, has already been incremented to 10 from our loop). And that's why every action we execute prints "10" instead of the 0 through 9 like we expected.

 

So, why do you need to know how these work? Take the following code that outputs items from an array of strings:

 

            var actions = new List<Action>();
            string[] urls = 
            { 
                "http://www.url.com", 
                "http://www.someurl.com", 
                "http://www.someotherurl.com", 
                "http://www.yetanotherurl.com" 
            };

            for (int i = 0; i < urls.Length; i++)
            {
                actions.Add(() => Console.WriteLine(urls[i]));
            }

            foreach (var action in actions)
            {
                action();
            }

 

This code looks pretty innocuous. Our bounds are protected, and we just index into our array to output a string. But, is that what we're really doing? Remember how closures work from above. The actual thing that happens when I run this code is this:

 

Confusing

 

Interesting. Even though our index variable should only even be less than the length of our url array, an exception is thrown because the index variable is actually equal to the length of our url array (and hence outside of the bounds thanks to 0-based indices).

 

Well, that wasn't what we were probably expecting. But now that we are having this "problem", what is the easiest way to resolve it? Remember that the problem is happening because of the scope of the variable within our closure. So to fix this, we can essentially declare a temporary variable that is unique in scope to this specific iteration through our array:

 

            for (int i = 0; i < urls.Length; i++)
            {
                string localUrl = urls[i];
                actions.Add(() => Console.WriteLine(localUrl));
            }

 

And now the code is fixed.

 

Understanding how lambdas work is especially important when you start developing with a library that leverages lambdas heavily like LINQ does, or Parallel Extensions to the .NET Framework. And don't worry, even those people that know how lambdas work occasionally get bitten by this behavior.

 

Enjoy the coding, folks!

Posted in C# | Programming
 #       Comments [7]
# Monday, June 09, 2008

This year for PDC 2008, we have decided to learn from a successful model used by various customers (like the ALT.NET crowd) in order to help ensure that PDC 2008 is the best conference ever. Our goal with these un-sessions is to really enable attendees with common interests to have some great conversations in an efficient setting.

So what is Open Space? Taken from the description we have on the PDC website (you can also refer to the Open Space Technology entry in Wikipedia):

Open Space is a way to bring together groups of people interested in a common topic to have an interactive discussion. In an Open Space session, there may be an expert who is passionate about a topic presenting to an audience or there may be a small group of people discussing an idea.

Four principles of Open Space:

  1. Whoever comes are the right people to be there
  2. Whatever happens is the only thing that could have happened
  3. Whenever it starts is the right time
  4. When it's over, it's over

All you need to do is suggest topics onsite that you wish to discuss and participate in sessions that sound interesting to you. It’s the unconference with content by attendees, for attendees.

Not only are we doing Open Space at PDC 2008, we want to make sure it is a _real_ Open Space. None of this "Microsoft doesn't understand, they don't grok it, they entirely and utterly butchered the whole Open Space concept".

 

Not only are we passionate about this topic, we are also fans of the Open Space movement. Of course there will be modifications that will have to be made to scale this to the size of a PDC as well as the fact that this is more of a "conference within a conference", but we hope those changes will keep the essence of Open Space unchanged.

 

If you have any concerns about Open Space @ PDC 2008, you can contact me directly at jason.olsonATmicrosoftDOTcom. I would love to hear from any of you regarding this topic. Otherwise, I look forward to seeing you in L.A. participating in Open Space @ PDC 2008! FOR THE DEVELOPERS!!!!

Posted in Conferences
 #       Comments [9]
# Wednesday, June 04, 2008

What a great week to be a developer. First, the latest Parallel Extensions CTP was released to the net, and now the first Managed Extensibility Framework CTP has been released onto Code Gallery.

So what is the Managed Extensibility Framework? From the announcement:

"The Managed Extensibility Framework (MEF) provides developers with a tool to easily add extensibility to their applications and with minimal impact on existing code. The application developer can define extension points according to the functionality required of an extension, while the extension developer uses those points to interact with the application.

MEF enables this extensibility to take place without creating a hard dependency in either direction. Applications can be extended at run time without recompilation, and extensions can be used by multiple applications sharing the same extension requirements. MEF also allows an application to delay the loading of an extension while still examining its metadata, enabling efficient traversal of large catalogs of extensions."

 

To make a medium-size story even shorter, the Managed Extensibility Framework (MEF) is essentially a framework for dependency inversion/inversion of control.

If you are a CTP junkie and like to play around with all the latest bits like I do, you have some downloading to do :P. Enjoy!

 #       Comments [0]
# Tuesday, June 03, 2008

In "Facet Mapping with LINQ, Part 1", we discussed how to add Facet Mapping to your .NET application using an attribute-based solution. This solution just doesn't leave me very excited. As we recounted, there are some problems with using an attribute-based approach:

  • Littering domain models with presentation layer-specific attributes 
  • Doesn't support existing POCOs (that perhaps can't change)
  • Can't support combinations of members (without create a new method property that actually does the combining (read: ugly)).

I personally only want one area of code to change if I alter my facet mapping. Having to change my POCOs when I want to change facet maps "smells" to me (and is itself a violation of the Single Responsibility Principle). Let's look at another way we can implement facet mapping in .NET via some more LINQ-goodness.

In a previous post, I talked about my new-found love for Fluent Languages in .NET (ala Moq or NInject). With attributes out of the way for facet mapping, I wanted to develop a library for facet mapping that was easy to use and could be used with existing code. Here are some of the "requirements" for the new facet mapping code (read: "requirements" means arbitrary features that I think are good and that I would like to see :P):

  • Usable with an existing set of POCOs (without having to change a single line of code in those objects)
  • Easily support combination of properties/methods or any arbitrary code that a person would want to create a facet on
  • Readable and easy to use (read: my own reason to try to a fluent language interface :P)

Let's take a look at the proposed API (taken straight from my unit tests):

            // Given our list of cars
            List<Car> cars = new List<Car>() {
                new Car { Make = "Toyota", Model = "Corola", Price = 20000, Year = 2001 },
                new Car { Make = "Toyota", Model = "Corola", Price = 30000, Year = 2002 }
            };

            // Generate a facet map (list of Facets) that we can use
            var facets = FacetMap<Car>.Create()
                .AddFacetOn(c => c.Make)
                .AddFacetOn(c => c.Model)
                .GenerateFrom(cars)
                .ToList();

We can also define an arbitrary expression that we want to generate a facet on:

            var facets = FacetMap<Car>.Create()
                .AddFacetOn("MakeAndModel", c => c.Make + " " + c.Model)
                .GenerateFrom(cars)
                .InParallel()
                .ToList();

With FacetMap being a generic class, we can use it to generate facets on any existing CLR class we wish to. And with the use of lambda expressions to specify what we are creating facets on, our code can be refactored a bunch and not break our facet mapping.

Implementing FacetMap isn't actually all that difficult. Since all our operations can chain be chained together with each other, all of them actually return the same interface (our IFacetMap that the FacetMap class implements).

The first thing you notice is that we are creating a generic class so that our facet maps can be used with any POCO that exists out there (theoretically). And, we are hiding the constructor so that we can use a factory method (I happen to think it reads better, could just be my personal opinion though).

    public class FacetMap<TResource> : IFacetMap<TResource>
    {
        /// <summary>
        /// Constructor is hidden as we are using a factory method
        /// </summary>
        private FacetMap() 
        {
            facetGenerators = new Dictionary<string, Delegate>();
        }

        /// <summary>
        /// Factory Method to create a new Facet Map
        /// </summary>
        /// <returns>New FacetMap</returns>
        public static IFacetMap<TResource> Create()
        {
            return new FacetMap<TResource>();
        }

        ...
    }

Our class implemented our IFacetMap interface so that external code using our FacetMap is relying on abstractions outside of our Factory Method. And as you can see, all our methods return this interface so that we can chain all the methods together to form the fluent language shown above (the one exception being our Generate() method where we just return our list of Facets that were generated):

    public interface IFacetMap<TResource>
    {
        IFacetMap<TResource> AddFacetOn<TResult>(Expression<Func<TResource, TResult>> facetValue);
        IFacetMap<TResource> AddFacetOn<TResult>(string facetName, Expression<Func<TResource, TResult>> facetValue);
        IEnumerable<Facet<TResource>> GenerateFrom(IEnumerable<TResource> resources);
    }

Why the "Expression<Func<X, Y>>" instead of just "Func<X, Y>". This is more a matter of convenience for the end-user. The end-user can write the code as if it was a normal Func<X, Y>, but we can do some fun things with the lambda expression like automatically generating the name of the facet based on the property or method the user passes in (notice that in the case of a more complex lambda expression, the user must specify the facet name since there is no way to generate a guaranteed user-friendly name for the lambda expression).

Internally, we are using a Dictionary<string, Delegate> to store all the passed in lambda expressions that are going to be used to generate our facets with. In the case where a facet name isn't explicitly stated, we automatically use the name of the Property or Method (depending on which type of expression was passed in):

        private Dictionary<string, Delegate> facetGenerators;

        public IFacetMap<TResource> AddFacetOn<TResult>(Expression<Func<TResource, TResult>> facetValue)
        {
            string facetName;
            if (facetValue.Body.NodeType == ExpressionType.MemberAccess)
            {
                MemberExpression expression = (MemberExpression)facetValue.Body;
                facetName = expression.Member.Name;
            }
            else if (facetValue.Body.NodeType == ExpressionType.Call)
            {
                MethodCallExpression expression = (MethodCallExpression)facetValue.Body;
                facetName = expression.Method.Name;
            }
            else
            {
                throw new InvalidHeadingExpressionException();
            }

            AddFacetOn(facetName, facetValue);
            return this;
        }

        public IFacetMap<TResource> AddFacetOn<TResult>(string facetName, Expression<Func<TResource, TResult>> facetValue)
        {
            if (!facetGenerators.ContainsKey(facetName))
            {
                facetGenerators.Add(facetName, facetValue.Compile());
            }
            else
            {
                throw new NonUniqueFacetAddedException();
            }

            return this;
        }

The last thing we need to do is implement our Generate() method. Here, we are largely going to reuse the LINQ statement from our last post (check it out), the difference being that instead of pulling facets from properties decorated with our attribute we will use out internal dictionary that is built from our AddFacetOn methods.

        public IEnumerable<Facet<TResource>> GenerateFrom(IEnumerable<TResource> resources)
        {
            return from facet in facetGenerators
                   select new Facet<TResource>
                   {
                       Name = facet.Key,
                       Headings = (from resource in resources
                                   let value = facet.Value.DynamicInvoke(resource)
                                   orderby value
                                   group resource by value into g
                                   select new Heading<TResource>
                                   {
                                       Value = g.Key.ToString(),
                                       MatchCount = g.Count(),
                                       Filter = BuildFilter(facet.Value, g.Key.ToString())
                                   }).ToList()
                   };
        }

        private Func<TResource, bool> BuildFilter(Delegate facetValue, string headingValue)
        {
            return (resource) => facetValue.DynamicInvoke(resource).ToString() == headingValue;
        }

And finally, that for every header we generate, we also generate a Filter (shown above). This Filter is of type Func<TResource, bool> so that it can be used in a Where() method to filter our existing list of resources at a later time based on the header that was chosen.

As you can see, there is some cool stuff you can do with LINQ around fluent languages very easily. And not only that, you can add some cool navigational techniques to your application using Facet Mapping that is quite easy to implement using the new features in LINQ.

Posted in LINQ | Programming
 #       Comments [2]
# Monday, June 02, 2008

The first time I saw some talk about this "movement", it was Martin Fowler discussing "Fluent Interfaces".

An example Martin gives is the idea of time intervals. Here's how one might declare an interval today (in Java):

TimePoint fiveOClock, sixOClock;
...
TimeInterval meetingTime = new TimeInterval(fiveOClock, sixOClock);

Here's how one might to do the same with a fluent interface (anybody who writes Ruby code should recognize this type of programming):

TimeInterval meetingTime = fiveOClock.until(sixOClock);

Like Jeff Atwood, I haven't exactly been a fan of fluent languages in the past (pre-LINQ). I honestly never saw the point in them (and I thought all the object syntax got in the way of what was being done). My mind has definitely changed though. As of late, you could say I'm on a bit of a "fluent languages kick."

LINQ, of course, had a play in this, as did the introduction of lambda expressions. But, it was really seeing two specific libraries that started to get me excited about fluent languages: Moq and NInject. When you compare fluent languages to Regular Expressions and SQL, I can imagine how it's hard to understand the benefit. But when you have a library that reads wonderfully out loud and "fits" an existing problem in order to make it easier to understand, I think it's A Great Thing.

Mocking is a topic I've been pretty passionate about in the past. Even with that passion, there was one thing I didn't like about the libraries back then: breaking changes when refactoring. A lot of mocking frameworks in the pre-.NET 2.0 days were string based. If you wanted to mock an aspect about a method, you put the name of the method in a string and pass that somewhere. Well, the second I renamed that method, BAM!, my unit test would stop passing (of course). Yes, the unit test caught it, but it was still highly annoying. Enter Moq.

Moq leverages lambda expressions to be able to determine what should be mocked. I think the solution Daniel Cazzulino has used is awesome. Let's look at some Moq code to mock some code:

var mock = new Mock<ILoveThisFramework>();

// WOW! No record/reply weirdness?! :)
mock.Expect(framework => framework.ShouldDownload(It.IsAny<Version>()))
    .Callback((Version version) => 
        Console.WriteLine("Someone wanted version {0}!!!", version))
    .Returns(true)
    .AtMostOnce();

I love how this reads. I think the trick behind groking fluent languages is reading them aloud. Go ahead, go back and read that out loud (just ignore the laughing coworkers, we'll get back at them later :P). Not only does this read very well, but if I refactor/rename the "ShouldDownload" method in Visual Studio, the use of that method in my mocking code will be updated as well.

The other fluent language that got me excited was the work Nate Kohari is doing on NInject. Assuming you are familiar with Dependency Injection, the best way to understand NInject is to drive write into code:

class Samurai {
  private IWeapon _weapon;

  [Inject]
  public Samurai(IWeapon weapon) {
    _weapon = weapon;
  }

  public void Attack(string target) {
    _weapon.Hit(target);
  }
}

So I have my injection point defined in my Samurai class. My Samurai is going to need an IWeapon. So how do we bind these two together with NInject?

Bind<IWeapon>().To<Sword>();

Once again, read this out loud. It literally reads like "Bind IWeapon to Sword". I just love that (your mileage may vary :P).

One of the things that bothered me about fluent languages in the past was that people would say "they're so great, they read just like our own language". Well, we've been down that route before, and I definitely wouldn't consider myself a fan of COBOL! So why the love for fluent languages?

One of the problems with spoken languages when it comes to programming is that spoken languages are very ambiguous and allow many meanings for a single word or phrase. In order to avoid that problem, I think COBOL became too obtuse. To be exact enough for a programming language while being solely "written language"-esque, syntax becomes annoyingly long.

The reason I like this new trend of fluent languages in .NET is that it introduces the readability of spoken languages while avoiding the obtrusiveness of languages like COBOL. To me, it's like having my cake and eating it too.

Geek Fact of the Day: Do you know the origin of the term "grok"? The term comes from Robert A. Heinlein's book Stranger In A Strange Land.

Posted in Programming
 #       Comments [6]

The latest CTP for Parallel Extensions to the .NET Framework was released out to the intarw3bs today (Go Digg It on Digg and Kick It on DotNetKicks). This is the second major CTP for the Parallel Extensions.

If you're the kind of person who likes to just get the latest install and play around, you can find it here. If, however, you are a person who would like to read some resources about the new CTP first, there's a bunch of links for you as well:

Here are some of the reactions (or re-announcements) from other bloggers:

I think a very important quote from Joe is (emphasis mine):

"I'm really excited to see our entire stack finally shipping as one cohesive unit: the data structures we use throughout the implementation exposed publicly (what we now call CDS), a new scheduler built from the ground up, TPL and PLINQ better together, and lots more."

I find this release very exciting, especially the "cohesive unit" part. As Ed mentions in the new features post, PLINQ is now running directly on top of TPL, rather than using the ThreadPool like it did before. The data structures used internally are now exposed as CDS. TPL with many improvements to the scheduler. This is a very great CTP for the Parallel Extensions Team. So congratulations to the team!

If you want to play around with concurrent programming in .NET, Parallel Extensions to the .NET Framework is the place to be. The team really wants your feedback and, through this feedback, you can help shape the future of concurrent programming in .NET. How exciting is that?!? So go grab the bits, install them, and enjoy!

And if you're a Parallel Extensions fan, be sure to Digg It and Kick It!

Posted in Concurrency | Programming
 #       Comments [0]

Contact

Email Me Send mail to the author(s)

Calendar

<June 2008>
SunMonTueWedThuFriSat
25262728293031
1234567
891011121314
15161718192021
22232425262728
293012345

About this site

Jason Olson's thoughts on Programming, Games, Music and Life in General

Disclaimer
The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

© Copyright 2008
Jason Olson

Sign In
All Content © 2008, Jason Olson
Theme based on 'Business' created by Christoph De Baene (delarou)