If you're a regular reader of Don Box's weblog then you probably know that Microsoft has made available another Community Technical Preview (CTP) of Language Integrated Query (LINQ) aka C# 3.0. I think the notion of integrating data access and query languages into programming languages is the next natural evolution in programming language design. A large number of developers write code that performs queries over rich data structures of some sort whether they are relational databases, XML files or just plain old objects in memory. In all three cases, the code tends to be verbose and more cumbersome than it needs to be. The goal of the LINQ project is to try to simplify and unify data access in programming languages built on the .NET Framework.
When I used to work on the XML team, we also used to salivate about the power that developers would get if they could get rich query over their data stores in a consistent manner. I was the PM for the IXPathNavigable interface and the related XPathNavigator class which we hoped people would implement over their custom stores to enable them to use XPath to query them. Some developers did do exactly that such as Steve Saxon with the ObjectXPathNavigator which allows you to use XPath to query a graph of in-memory objects. The main problem with this approach is that implementing IXPathNavigable for custom data stores is non-trivial especially given the impedence mismatch between XML and other data models. In fact, I've been wanting to do something like this in RSS Bandit for a while but the complexity of implementing my own custom XPathNavigator class over our internal data structures is something I've balked at doing.
According to Matt Warren's blog post Oops, we did it again it looks like the LINQ folks have similar ideas but are making it easier than we did on the XML team. He writes
What's the coolest new feature? IMHO, its IQueryable<T>. DLINQ's query mechanism has been generalized and available for all to use as part of System.Query. It implements the Standard Query Operators for you using expression nodes to represent the query. Your queries can now be truly polymorphic, written over a common abstraction and translated into the target environment only when you need it to. public int CustomersInLondon(IQueryable<Customer> customers) { int count = (from c in customers where c.City == "London" select c).Count(); return count; } Now you can define a function like this and it can operate on either an in memory collection or a remote DLINQ collection (or you own IQueryable for that matter.) The query is then either run entirely locally or remotely depending on the target. If its a DLINQ query a count query is sent to the database. SELECT COUNT(*) AS [value] FROM [Customers] AS [t0] WHERE [t0].[City] = @p0 If its a normal CLR collection, the query is executed locally, using the System.Query.Sequence classes definitions of the standard query operators. All you need to do is turn your IEnumerable<Customer> into IQueryable<Customer>. This is accomplished easily with a built-in ToQueryable() method. List<Customer> customers = ...; CustomersInLondon(customers.ToQueryable()); Wow! That was easy. But, how is this done? How can you possible turn my List<T> into some queryable thingamabob? Good question. Glad you asked. Check out this little gem: Expression<Func<Customer,bool>> predicate = c => c.City == "London"; Func<Customer,bool> d = predicate.Compile(); Now you can compile lambda expressions directly into IL at runtime! ToQueryable() wraps your IEnumerable<T> in IQueryable<T> clothing, uses the Queryable infrastructure to let you build up your own expression tree queries, and then when you enumerate it, the expression is rebound to refer to your IEnumerable<T> directly, the operators rebound to refer to System.Query.Sequence, and the resulting code is compiled using the built-in expression compiler. That code is then invoked producing your results. Amazing, but true.
What's the coolest new feature? IMHO, its IQueryable<T>.
DLINQ's query mechanism has been generalized and available for all to use as part of System.Query. It implements the Standard Query Operators for you using expression nodes to represent the query. Your queries can now be truly polymorphic, written over a common abstraction and translated into the target environment only when you need it to.
public int CustomersInLondon(IQueryable<Customer> customers) {
int count = (from c in customers
where c.City == "London"
select c).Count();
return count;
}
Now you can define a function like this and it can operate on either an in memory collection or a remote DLINQ collection (or you own IQueryable for that matter.) The query is then either run entirely locally or remotely depending on the target.
If its a DLINQ query a count query is sent to the database.
SELECT COUNT(*) AS [value]
FROM [Customers] AS [t0]
WHERE [t0].[City] = @p0
If its a normal CLR collection, the query is executed locally, using the System.Query.Sequence classes definitions of the standard query operators. All you need to do is turn your IEnumerable<Customer> into IQueryable<Customer>. This is accomplished easily with a built-in ToQueryable() method.
List<Customer> customers = ...;
CustomersInLondon(customers.ToQueryable());
Wow! That was easy. But, how is this done? How can you possible turn my List<T> into some queryable thingamabob?
Good question. Glad you asked.
Check out this little gem:
Expression<Func<Customer,bool>> predicate = c => c.City == "London";
Func<Customer,bool> d = predicate.Compile();
Now you can compile lambda expressions directly into IL at runtime!
ToQueryable() wraps your IEnumerable<T> in IQueryable<T> clothing, uses the Queryable infrastructure to let you build up your own expression tree queries, and then when you enumerate it, the expression is rebound to refer to your IEnumerable<T> directly, the operators rebound to refer to System.Query.Sequence, and the resulting code is compiled using the built-in expression compiler. That code is then invoked producing your results.
I think it's pretty amazing that all I have to do as a developer is implement a simple iterator over my data structures (i.e. IEnumerable) and then I get all the power of Linq for free. Of course, if I want the queries to be performant it would make sense to implement IQueryable directly but the fact that the barrier to entry is so low if my perf needs aren't high is goodness.
For more information on LINQ, read the Linq project overview. If you are like me and are primarily interested in XLinq then check out XLinq: XML Programming Refactored (The Return Of The Monoids) which has the fingerprints of my former team all over it. Way to go guys!