MEF is a new library in .NET Framework 4 and Silverlight 4 that addresses the problem of easily extending and componentising applications by providing a nifty framework. Adding an easy ability to extend your application, whether it be for the purposes of componentising your application or allowing others to extend your application has always been available to Delphi users in the guise of frameworks like RemObjects Hydra, the TMS Plugin Framework or building your own homebrew solution but MEF is very nice framework.

MEF is incredibly powerful and I’ll only really be scratching the surface and introducing it here.

Imagine an application which runs a sequential series of processes on an imaginary input file, not exactly rocket science but this is only a basic introduction after all.

First I begin by defining an Interface in a separate assembly which all my plugins will adhere to:

  IFileAction = public interface
    method ProcessFile(path: String);
    property Name: String read;
    property Version: String read;
  end;

We then write our first plugin, which we’ll conveniently call the MyHelloPlugin. First create a new assembly and reference the base plugin assembly which contains our IFileAction interface and the System.ComponentModel.Composition assembly. We also need to ensure that we reference these in our uses clause of the unit containing our Hello Plugin class. The My Hello Plugin class looks like this:

  [Export(typeof(IFileAction))]
  MyHelloPlugin = public class(IFileAction)
  const
     MYNAME = 'The My Hello Plugin';
     MYVERSION = '1.0.1';
  public
    method ProcessFile(path: String);
    property Name: String read MYNAME;
    property Version: String read MYVERSION;
  end;

This should be pretty self explanatory but we have two properties for our name and version and a ProcessFile method which will be called by our host app, providing it with a path for the file that we’re processing. You’ll notice the Export attribute which denotes what we’re exporting. We’re specifying the type of the export but in many cases you may not even need to do this. MEF has a very powerful ExportMetaData system for providing information about what you’re exporting which is also worth reading up on.

Next, of course, we need to actually consume the plugins in our main application. To manage my collection of plugins, I’ll create a small, basic class to hold and manage them:

  PluginList = class
  public
    [ImportMany(typeof(IFileAction))]
    property FileActions: IList<IFileAction> read write;
    constructor;
  end;

Here I’m using the ImportMany attribute to indicate that I’m expecting to import many IFileAction instances and then providing a list with which to populate (which, as always, you must remember to instantiate before you attempt to fill it!).

Next I’ll load up my Hosting catalog and compose the parts of my application.

  var aggregateCat := new AggregateCatalog();
  // var catalog := new AssemblyCatalog(System.Reflection.Assembly.GetExecutingAssembly());
  // var catalog := new AssemblyCatalog('plugins\MyHelloPlugin.dll');
  var dirCatalog := new DirectoryCatalog('plugins');

  // Add our Catalogs here,
  aggregateCat.Catalogs.Add(dirCatalog);

  var container := new CompositionContainer(aggregateCat);
  // Create our plugin hosting object
  var pluginList := new PluginList();
  // Compose the parts.
  container.ComposeParts(pluginList);

You’ll note that I create an AggregateCatalog which isn’t strictly necessary in this case because I’m only adding one ComposablePartCatalog to it but in other uses this would also allow me to separately load single Assemblies or a collection of DirectoryCatalog’s. I then load the catalog into a composition container before creating my aforementioned PluginList class and finally pulling all the parts together. It really is that simple. You can test what has been loaded using our now populated PluginList.FileActions collection, like this:

  for each plugin: IFileAction in pluginList.FileActions do
  begin
    Console.WriteLine('Loaded ' + plugin.Name + ', version ' + plugin.Version);
  end;

You can now create a new and entirely different plugin assembly (or many, many more) and providing it conforms to our IFileAction interface and is either in our target directory or loaded manually, it will now appear in our list as if by magic.

Once you’ve begun using MEF, you’ll find out how powerful it can be and how easy it makes building modular applications.

If anyone is interested in me uploading the complete project set for this sample then please let me know in the comments below and I can upload it to github. If anyone else has any excellent samples of MEF usage that they can publish we could even start a sample repository.

Further Reading

• The MEF Architecture Overview
• The MEF Programming Guide
• The MSDN System.ComponentModel.Composition Guide

I still believe that participating in these communities and making visible “noise” does credit to the delphi community. Some of you may remember that I extolled the virtues of this point just over a year ago in another post, The Delphi Community: From the outside.

This is all particularly relevant in a month when, the largest community gathering, delphi.non-technical has been filled to the brim with little which might give a potential newcomer to the community any kind of welcome feeling what-so-ever. Also this month a set of stats generated from the GitHub API showed that Delphi was the 39th largest language on there, another online property that I advocated our further involvement with. Without saying that we should be much higher, I would say that I would like to see more Open Source Delphi projects on there. We could also do with more of a vocal presence in many other areas too but I guess this requires organic community growth (allbeit slightly prompted).

The recently unveiled previews of RAD Studio XE may not be the cross-platform upgrade that some people hoped it would be, it does represent the next product version of the Delphi we know and love. If what the Embarcadero staff are saying in the newsgroups of the quality this release is true then it will be a worthy addition to the family (if not as groundbreaking as we’d originally hoped). Indeed I liked the look of the new integrated tools for the Delphi IDE revealed today and will await the next previews but even if that isn’t your cup of tea it still leaves plenty to shout about.

What I’m advocating is that we retain a little perspective and focus on what is in the Delphi XE release and not what isn’t and use this as a conversation starting point to start making more noise about Delphi in “public places” with a view to reinvigorating the community. Which reminds me: what other communities/online properties could the Delphi community benefit from making better use of?

In other news: I’ve also got what I consider to be an exciting Delphi community based side-project which you may all potentially get to see very soon…

Visual Studio 2010 Support

I’ll start with the most obvious change: Delphi Prism now supports Visual Studio 2010 which was only released just over a month ago. The Visual Studio 2010 IDE looks great, feels faster than 2008 and most importantly, in my experience, is very stable.

Visual Studio 2010 with Delphi Prism

The shell is based on WPF and has had considerable work put into allowing it to be flexible and extensible which I understand has allowed a whole host of new plugins to flourish. It’s an extremely minor point but I’ve also found it much better in allowing themes for the code editor, springing forth sites like http://studiostyles.info/. My favourite scheme  is Son of Obsidian which works marvellously for C# code and works reasonably well for Delphi Prism code too. It’s great that we can take advantage of community extras like this, although it would also be great if we could tailor some of these themes specifically for Delphi Prism.

As marc mentioned back on episode #39 of the Podcast at Delphi there are a lot of free minor improvements that come with moving to Visual Studio 2010 (not financially free but that were inherent with the migration).

Other cool stuff includes support for Silverlight 4 and the “Paste C# as Oxygene” feature which allows you to translate code snippets on the fly. This release also includes MonoDevelop which includes support for Mac and Windows.

For, If and Case Expressions

For, if and case I hear you say? Haven’t we had those for as long as we can remember? Not like this we haven’t. Delphi Prism now supports them as expressions. This allows you to do operations like this:

  var sourceList: Array of Integer := [1 ,2 ,4 ,8 ,16 ,32 ,64 ,128];
  var sqList := for each num in sourceList yield num * num;

Now this sample could just as easily be done with a LINQ operation or a separate loop but this is another option available at your disposal and introduces a slightly Functional Programming feel. Delphi Prism now also supports If expressions and my favourite expression addition: case.

These expressions, as with so many language features hold the potential for misuse but they can also make things a little more concise:

method NotifierStr(newMails: integer): string;
begin
  result := 'You have ' + newMails.ToString() + ' new ' +
    case newMails of
      0: 'emails';
      1: 'email';
    else
      'emails';
    end;
end;

Please forgive the horrible string concatenation but I wanted to demonstrate it in this way because I thought it might help make it a little clearer.

Dynamic

Delphi Prism 2011 includes support for the (in some cases) contraversial .NET 4 dynamic type. To quote Scott Hanselman on dynamics: A dynamic is a statically typed dynamic type. Dynamics leave a lot more till run-time and allow a much cleaner interop with the Dynamic Scripting languages built upon the CLR.

For example, the ExpandoObject enables you to add and delete members at runtime, something might be useful when serialising and deserialising JSON or XML where you do not know the spec ahead of time:

dynamic tweetresult := new ExpandoObject();
tweetresult.Id = 1243535345;
tweetresult.Username = 'delphifeeds';

The Delphi Prism 2011 implementation has a required dependency upon a separate assembly, RemObjects.Oxygene.Dynamic.dll, which is freely re-distributable.

For a much better idea of what dynamics are and can be used for, I would refer you to Scott Hanselman’s whirlwind tour of dynamics and the MSDN reference on using dynamics.

Tail Calls

Tail Calls are another feature that could be seen as similar to a Functional Programming Language construct. They are essentially an exit statement followed by a call to a method. They have to be explicitly enabled and disabled for a method using the {$TAILCALL ON} flag. This allows you to code recursive functions, particularly useful for maths based methods. This concept will probably take a bit of getting used to and are something I’d like to cover in subsequent posts.

Tuples

A Tuple is an ordered list of Elements. They can contain 1 to 7 different references and types with an 8th reference being used to reference another Tuple if you need to hold more than 7. They are similar to an array and a class/record but unlike an array can contain more than one type and require considerably less declaration than a class/record. A Tuple in Delphi Prism might be used as follows:

  var telephoneList := new Dictionary<Tuple<string, integer>, String>();
  telephoneList.Add(Tuple.Create('Josh Moore', 1283843), '+0014158343131');
  Console.WriteLine(telephoneList[Tuple.Create('Josh Moore', 1283843)]);

They are particularly useful for returning more than one value from a method where you would otherwise need to explicitly declare a class or record to hold the data otherwise. Tuples are a good way to solve a specific problem and will be more familiar to those who have used them in other languages such as the Tuple support in Python.

Parallel Loops (not actually a new feature)

Parallel loops have been a part of the Delphi Prism (and formerly Oxygene) language for a long, long time. However what I hadn’t appreciated was that the Parallel Task Library upon which it relied was officially bundled into the .NET 4.0 Framework libraries. As such I felt it necessary to re-raise it and show just how nifty it is here even though it isn’t really new to Delphi Prism 2011.

Just to recap, with the parallel keyword you can parallelise a loop.

As a fully fledged .NET 4 language we can also use the AsParallel() method in order to return a ParallelEnumerable source capable of being queried with the .NET Parallel LINQ library.

The rough and ready example below demonstrate these two techniques:

class method ConsoleApp.Main(args: array of string);
begin
  var anagramletters := 'mite';
  var filepath := 'mydictionary.txt';

  var dictList := new List<string>();
  dictList.AddRange(System.IO.File.ReadAllLines(filepath));
  Console.WriteLine('Added ' + dictList.Count + ' lines to the file');

  var sortedList := new Dictionary<string, string>();

  for parallel i: Integer := 0 to dictList.Count - 1 do
  begin
    sortedList[dictList.Item[i]] := dictList[i].Sort();
  end;

  var matchingwords := sortedList.AsParallel().Where(w -> ((w.Value.Length = anagramletters.Length) and (w.Value = anagramletters.Sort()))).Select(w -> w.Key);

  Console.WriteLine('Found: ' + matchingwords.Count().ToString() + ' results');
  matchingwords.ToList().ForEach(x -> Console.WriteLine(x));

  Console.ReadLine;

end;

The code above reads a dictionary file in and then builds an index using a parallel loop before finding the words that are anagrams of the provided string using a Parallel LINQ query.

Extension Methods

I consider it a slightly more minor point as I’ve already briefly talked about the new method of defining Extension Methods, but this is now baked into Delphi Prism 2011 and is a pretty useful tool to have. I used the new Extension Methods syntax in the Parallel Example above to add an implementation specific Sort() method to my strings like so:

interface

extension method String.Sort : String;

implementation

extension method String.Sort : String;
begin
  var c := self.ToCharArray();
  Array.Sort(c);
  result := new string(c);
end;

Summary

Overall Delphi Prism 2011 is a pretty significant release and contains lots of new features that represent a lot of interesting new potential both on the language side and the IDE side, congratulations to the RemObjects team.

Further Reading

• What’s new in Delphi Prism 2011 – The Official Embarcadero page.
• New Features – The Delphi Prism Documentation Wiki page listing newly added features.
• Delphi Prism 2011 Release Notes – Is what it says on the tin.
• Using Type dynamic in .NET 4.0 – MSDN reference on the dynamic type.
• What is tail-recusion – StackOverflow answer on Tail Recursion which might help you to better understand Tail Calls in Delphi Prism.

WarbleCamp is a free unconference style event for the UK/Euro Twitter developer community which was held at the offices of the guardian. I was lucky enough to see an announcement notice before the first set of tickets had vanished in a very short space of time. We arrived at the very smart guardian offices where guardian staff were still working throughout the weekend (apparently there is some sort of big news unfolding or something?) and watched @jot and @kalv give the opening introduction to the event.

Session boards at Warblecamp (Photo from @raffi)

The day was broken up into Barcamp style sessions where anyone could propose a session simply by writing it onto a notecard/post-it note and finding a free timeslot and room on the wall board.

Session boards at Warblecamp (Photo from @raffi)

After several introductions and conversations over cups of coffee and a Strawberry and Banana oatmeal breakfast courtesy of moma foods we selected our first session. I was a little disappointment that there seemed to be a quite a few people who had registered for tickets and then simply never turned up.

Aral Balkan (@aral) – An Introduction to the Streaming API.

I felt very sorry for Aral as he’d bravely volunteered to move his session to one of the first of the day which meant that delays in starting was to be inevitable while people found rooms etc. It also amazes me how you can put 40-50 geeks into a room and then find that noone can work a projector.

After changing rooms we finally got onto the subject of the new Twitter Streaming API. The Streaming API is the 3rd Twitter API Subset which allows you to access various slices of public tweets. The Key differentiator is that unlike the other 2 twitter API services it actively pushes tweets to you in (sort of) real-time so that you don’t have to constantly poll Twitter’s servers to check for new information.

The Streaming API is an invaluable addition to the API Services and also hints at how many API services of this type of data model should actually be modelled. This session was cut rather short but one point that stuck with me is that Json is now the preferred response type and they’re considering XML return methods for depreciation.

Paul Mison (@blech) – Annotations vs Machine Tags

This session was a good discussion around the uses and power of Machine Tags and Annotations which focussed around Flickr Machine Tag usage as at this point in time Twitter doesn’t support adding metadata to Tweets (little did we know what was to come later in the day).

The basic concept is to decorate a resource with metadata in some way, this usually follows a simple model. This normally takes the form Namespace:key=value. For Example, on Flickr, One might tag a photo with Car:Manufacturer=Ferrari to indicate that it is a photo of a Ferrari car. This makes data about our photo more specific as a photo simply tagged “F357″ might be be any number of things.. a room, a camera, a car.

@aral started a good discussion around how you might “standardise” the namespaces of these tags and the hierarchy as it’s entirely possible that people may have different words for the same descriptor. There was popular consensus for an organic community driven approach.

@tommorris raised a good point: that even with a hierarchy of key-value pairs, is this really enough to actually associate a relationship perspective with an object? The example he gave within the context of Annotated Tweets is that you can tag a tweet with a book:title=”Treasure Island” but how can you tell whether this refers to a book I’ve read, a book I bought, a book I wrote?

Nick Halstead (@nickhalstead), CEO of Tweetmeme – #NoSQL

Nick gave us a brief overview of Tweetmeme‘s platform growth, they started with a well trodden track of a LAMP stack.

Tweetmeme services a fairly eye-watering amount of work: they store 4TB of data per day, index 500 million URLs and service 10,000 Req/s from about 25 servers. MySQL worked fine, following the usual Master / Slave division pattern until they got above a certain number of rows after which deletions started to become a pain point.

They have now moved to a Custom PHP job queuing system,  Nginx and HA_PROXY and use a tiered system of graceful degradation (similar to Twitter’s own system) which allows them to service requests differently according to the load on individual machines.

They first evaluated Tokyo and concluded that whilst it was very very fast  it was going to prove a difficult as it would mean carefully sharding data themselves.

They then evaluated Redis from which they were very impressed with the clever features to create Lists and Sets of data.

Finally they settled on Cassandra which provides eventual consistency and easy scaling across commodity hardware (a key advantage for a startup like Tweetmeme).

Christian Heilmann (@codepo8) – Geo Platforms

I’ve been a follower of Christian on Twitter for quite a while so I’m used to his enthusiasm for the Yahoo YQL platform but it was great to actually see his enthusiasm in person.

Whilst we were waiting for people to find their way to the room Christian revealed that some people are so into Geo that apparently they decide to walk around London on a particular trail in order to create a phallic route.. ergo gpsc**ks.com (conceivably NSFW) was born.

He first made the point that we’re beyond the point of it being novel to be able to display our GPS location on our mobiles and how we actually needed to do something useful with it. He covered how a few lines of Javascript we can use the HTML5 Geo API to locate a user (with varying degrees of accuracy). He pointed out that working with Geo locations is considerably more complicated than simply obtaining the Lat/Long of the current user. To help with this he suggested several resources:

• Geo Planet Explorer – A Yahoo web service that allows you to submit a WOE (Where on Earth) location and get back data about the neighbourhood and it’s neighbouring areas.
• YQL Geo Library – A JS Library which uses the YQL Library to provide services like geolocation, reverse geocoding, content analysis all in one easy library.

He then gave us a live demo of the YQL Service which which truly very powerful. It provides structured, consistent access to a whole load of different web services and datasets. This reminded me of what Ander Heijlberg said in his Technet 2010 keynote about programming for the what and not the how.

An example of a YQL queries might be: select * from geo.places where text=”san francisco, ca”

As someone who is used to working with web services the hard way, the YQL frees you from some of the effort of Authentication, gives you a consistent output format and does some caching on the server side.

Raffi Krikorian (@raffi) – Twitter Annotations

@raffi is Twitter’s Dev Lead for the API Platform (aside from being a generally nice chap) and had gotten permission just before he flew out to the UK to give us the first public draft of the upcoming annotations feature planned. You can check out his slides on the Twitter API Annotations and even watch the recording of the talk over on Vimeo.

The basic idea is that you will shortly be able to decorate tweets with up to 512 Bytes of data (excluding formatting and data classification structures). This could be used to append data about a book that the tweet refers to, a movie which it is talking about or any number of uses, including pure machine-to-machine communication.

This adds context to a tweet, something which twitter started doing when it added GeoLocated tweets and intends on doing more of in the future.

It is stressed that this is still a very early draft but that the Annotations may look something like the following:

{(type =>  (attribute => value),
               (attribute => value)}

And as an example:

[(“tv episode” => {“episode” => “The Vampires of Venice”,
		“series” => “Dr Who”,
		“air date” => “8 May 2010”}}]

The names of types and attributes will be allowed to grow organically from the community although they will have a broad set in use at the time of launch with support from a few partners. Attribute types and key name usage will be tracked and stats made available on dev.twitter.com to allow developers to browse existing usage implementations.

@codepo8 asked if they looked at RDF and microformats to see if they could adopt an existing standard for metadata and while they wouldn’t have been directly applicable, there is still a chance that they could reuse some of the taxonomy.

Lots of people asked if it would be possible to have metadata only tweets so that you could filter out all those annoying Foursquare checkin tweets and those “I’m listening to” tweets. This is now easily doable with the support of 3rd party client providers.

As with nearly all the previous new features released by Twitter, this may eventually be built into the twitter.com interface but it is initially planned as an API Platform feature.

Annotations will be made available to application developers  first as a preview within the next month sometime.

Others

There were several more sessions that I particularly enjoyed including Remy Sharp‘s (@rem) presentation on a conference dashboard he built he built for Chirp conference that works entirely with Javascript. Aral Balkan (@aral) also chaired what turned out to be an introduction and discussion of Twitter’s xAuth process (an adapted version of OAuth for Desktop Applications) which turned out to be a penny drop moment for me with the workflow process for OAuth Desktop applications.

I’m also keen to follow up on @ketan‘s proposal for a twitter:// protocol handler and to have a play around with the User Streaming API.

Thanks to the organisers, speakers and sponsors of warblecamp, I think it’s safe to say it was a well deserved great success. A special thanks to the primary sponsors, Guardian Open Platform, oneforty, Yahoo Developer Network, Paypal X and O2 litmus and to the micro-sponsors for supporting us with food, drink and entertainment.

]]> http://jamiei.com/blog/2010/05/warblecamp/feed/ 1 http://jamiei.com/blog/2010/03/yajl-for-delphi/ http://jamiei.com/blog/2010/03/yajl-for-delphi/#comments Wed, 31 Mar 2010 20:40:13 +0000 jamiei http://jamiei.com/blog/?p=575 Introduction

I have been toying with the idea of trying to convert the yajl parser bindings to Delphi in order to build a wrapper on top of the original C dll. yajl (Yet Another JSON Library) is a small fast SAX style JSON parser written and open sourced in C over at lloyd’s yajl GitHub page. I wanted to be able to use this library in particular because it’s very fast, small, portable and able parse from a stream. Plus, I had never really done any conversion of this nature on C header files before so I thought it might be a small, easy project to start with… I was a perhaps a little naive on this front!

I’ll leave you with a disclaimer early on: the code that I’ve been working on and have shared is not currently in a working state. This is the first part of my journey and I do intend on posting further as I learn from the major mistakes that I make in the process.

Approach

The 3 header files that I need to translate can be found in the src/api directory:

• yajl_common.h
• yajl_gen.h
• yajl_parse.h

I’m not sure what a *typical* process for conversion might look like but I started out by trying to use Dr Bob’s HeadConv tool as a way of getting me started and doing some of the legwork. However, the callback structure used by the parsing callbacks confused the tool and it never produced anything of use. After this setback I tried to install Rudy Velthuis‘s Conversion Helper Tool but I wasn’t able to load this into the Delphi 2009 IDE.

Having failed to get any of the conversion tools to do any of the work for me, I accepted my task and began to convert the headers from scratch. There were two resources that I found invaluable: the first is the Type Conversion table half way down the page on Dr Bob’s HeadConv page.  The second was the comprehensive Pitfalls of Conversion guide from Rudy.

As I began converting the headers I realised that although I thought I was translating the headers without a problem, the devil is in the detail and as my first major conversion project, I didn’t have the specfic knowledge necessary to correct my specific mistakes. As a result, please correct me if I have misunderstood any bits which I have picked up in the process.

Const parameters

One issue that I did notice is that there were quite a few parameters marked const in the header files, such as this declaration in yajl_parse:

   YAJL_API unsigned char * yajl_get_error(yajl_handle hand, int verbose,
                                            const unsigned char * jsonText,
                                            unsigned int jsonTextLength);

Rudy’s guide states that where possible Delphi programmers should ignore the const declaration as it’s easy to get wrong depending on whether the argument is passed by value or by reference, complicated by the issue that it means something subtly different in C. I translated this heading and simply ignored the const keyword:

    Tyajl_get_error = function(handle: yajl_handle;
                               verbose: integer;
                               jsonText: PChar;
                               jsonTextLegnth: Cardinal): PChar; stdcall;

Multiple Indirection Parameters

There was one declaration that I had to do a bit of searching for: it was in the yajl_gen file, in the declaration of yajl_gen_get_buf function:

YAJL_API yajl_gen_status yajl_gen_get_buf(yajl_gen hand,
                                              const unsigned char ** buf,
                                             unsigned int * len);

I had never seen the double asterisk before and found it surprisingly difficult to reliably Google for it. Luckily, this comprehensive page on C pointers was pretty useful in explaining that it is in fact a technique called multiple indirection which essentially creates a pointer to a pointer, in this case to a char.

    Tyajl_gen_get_buf = function (handle: Tyajl_gen;
                                 out buf: PChar;
                                 out len: Cardinal): yajl_gen_status; stdcall;

I wasn’t sure if this was indeed the correct translation but if you know otherwise then please let me know in the comments below.

Record Alignment and Enum size

When reading up on the perils of record alignment and enum sizes I found this blog post from Vlad loan Topan on Record Alignment and Enum sizes. Both problems are potentially applicable to the yajl header files. In his post, Vlad mentions that in order to line up the enum sizes you will need to use the {Z4} compiler directive to change the size from the Delphi default of 1 to the C compatible 4 as so:

type
  {$Z4}
  yajl_status = (
                  // no error was encountered
                  yajl_status_ok,
                  // a client callback returned zero, stopping the parse
                  yajl_status_client_canceled,
  ... snip ...

Reading through the Record alignment section of Rudy’s article I understood that I needed to search through the yajl project source to look for an Alignment setting. However, I couldn’t find a reference to the pragma directive anywhere in  the  source.

Next Steps

Well, it doesn’t actually work yet but I have put the Delphi-yajl project on GitHub where you can view my efforts so far, fork it and hopefully provide feedback or code contributions to show me where I’m going wrong. I have also setup a Delphi-yajl page here to serve as a placeholder for when it actually works!

Futher Reading

• yajl – The yajl project home page.
• Dr Bob’s Header Conversion Tool – The useful tool from the ever excellent Dr Bob.
• The Pitfalls of Conversion – Rudy Velthuis’s long but excellent article on commonly encountered problems when converting
• Conversion Helper Package – The Conversion package helper from Rudy but I’m uncertain as to how compatible it is any more.

So what is the Rx Framework?

It’s a framework for asynchronous programming from Microsoft Research (including Erik Meijer & Wes Dyer amongst others) which could potentially be para-phrased as LINQ to Events. The basic problem is that Asynchronous programming is hard, the exacerbating issue for this problem is that it is entirely necessary for modern development. It is pretty common for Desktop applications to need to chain a series of events together with the correct order and sometimes correct time but doing so at present can land you in a spaghetti-like sea of event callbacks. Enter the Microsoft Rx Framework.

What do I need?

The Framework requires a minimum .NET Framework version of .NET 3.5 SP1 or .NET 4 but downloads for both versions are available on the Microsoft Research site. I couldn’t work out why but Delphi Prism appears to have a problem with the Libraries use of Extension Methods but this issue appears to be fixed in the latest Beta of Delphi Prism. I now suspect it might have started working as a result of the updated support for Extension methods which marc hoffman recently posted about.

How does it work?

In Brief: The Rx Framework introduces a of a pair of interfaces, the IObserver & IObservable, which allow you to represent push-based observable collections, along with a library of extension methods (the bit that wasn’t previously working in Delphi Prims) that implement the Standard LINQ Query Operators.

And…

I always find that the easiest way to understand it is to view some examples. First you’ll need to make sure reference the System.CoreEx and System.Reactive assemblies.

As with always, we’ll start with an entirely useless example just to demonstrate a basic example which uses an Observable Timespan interval to print out a number which doubles every 2 seconds:

class method ConsoleApp.Main(args: array of string);
begin
  var oneNumberPerSecond := Observable.Interval(TimeSpan.FromSeconds(2));

  var numbersTimesTwo := from n in oneNumberPerSecond
                        select n * 2;

  Console.WriteLine('Numbers * 2:');

  numbersTimesTwo.Subscribe(num -> begin Console.WriteLine(num) end);

  Console.ReadKey();
end;

Clearly, this is a contrived example but I think this hints at a few of the more powerful features and versatility of the Rx Framework. The first thing to note is that this is an Asynchronous process, imagine the same code using the standard .NET Timer class. This is a demonstration of the power of the Observable Interfaces and how it can allow standard interfaces to concepts that would otherwise complicate our code such time windows to be easily expressed. The second lines to notice is the highlighted LINQ To Events code on lines 5 and 6 which show how easily you can filter specific events from any events. This is a truly powerful and important concept because it is fairly common to generate a stream of events, only some of which we want to react to and the Linq extensions allow us to express this in a succinct way.

This is only a basic console application but it’s not hard to imagine how this could make the complex ordering and timing of GUI events in many applications much easier. As another example I have translated the C# ISubject ping pong example which mimics the classic Ping Pong Actor example that the Scala folks get so excited about.

First of all the Ping ISubject and Pong ISubject classes:

namespace PingPongExample;

interface

uses
  System.Collections.Generic,
  System.Linq,
  System.Text;

type
  Pong = public class (ISubject<Ping, Pong>)
  private
  protected
  public
    method OnError(exception: System.Exception);
    method OnCompleted;
    method Dispose;
    method Subscribe(observer: System.IObserver<PingPongExample.Pong>): System.IDisposable;
    method OnNext(value: PingPongExample.Ping);
  end;

implementation

method Pong.Dispose;
begin
  OnCompleted();
end;

method Pong.OnCompleted;
begin
  Console.WriteLine('Pong finished Ponging');
end;

method Pong.OnError(exception: System.Exception);
begin
  Console.WriteLine('Exception');
end;

method Pong.OnNext(value: PingPongExample.Ping);
begin
  Console.WriteLine('Pong received Ping..');
end;

method Pong.Subscribe(observer: System.IObserver<PingPongExample.Pong>): System.IDisposable;
begin
  result := Observable.Interval(TimeSpan.FromSeconds(2.5))
                .Where(n -> n < 10)
                .Select(n -> Self)
                .Subscribe(observer);
end;

end.

namespace PingPongExample;

interface

uses
  System.Collections.Generic,
  System.Linq,
  System.Text;

type
  Ping = public class (ISubject<Pong, Ping>)
  private
  protected
  public
    method OnNext(value: Pong);
    method OnError(error: Exception);
    method OnFinished;
    method Subscribe(observer: System.IObserver<PingPongExample.Ping>): System.IDisposable;
    method OnCompleted;
    method Dispose;
  end;

implementation

method Ping.OnNext(value: Pong);
begin
  Console.WriteLine('Ping received Pong..');
end;

method Ping.OnError(error: Exception);
begin
  Console.WriteLine('Exception');
end;

method Ping.OnFinished;
begin
  Console.WriteLine('Completed Ping Ponging');
end;

method Ping.Dispose;
begin
  OnFinished();
end;

method Ping.OnCompleted;
begin

end;

method Ping.Subscribe(observer: System.IObserver<PingPongExample.Ping>): System.IDisposable;
begin
  result := Observable.Interval(TimeSpan.FromSeconds(2))
                .Where(n -> n < 10)
                .Select(n -> Self)
                .Subscribe(observer);
end;

end.

And then our setup code:

class method ConsoleApp.Main(args: array of string);
begin
  var ping := new Ping();
  var pong := new Pong();

  Console.WriteLine('Any key to stop..');

  var pingSubscription := ping.Subscribe(pong);
  var pongSubscription := pong.Subscribe(ping);

  Console.ReadKey();

  pongSubscription.Dispose();
  pingSubscription.Dispose();

  Console.WriteLine('Ping Pong Completed..');
end;

You can embrace IObservable in your own business objects quite easily and begin pushing events throughout your application asynchronously:

type
  Invoice = class

  private
    paidDate: DateTime;
    paidSubject: Subject<Invoice>;
  public
    constructor;
    method MarkPaid(paidDate: DateTime);
    property Paid: IObservable<Invoice> read paidSubject.Hide();
  end;

  ConsoleApp = class
  public
    class method Main(args: array of string);
  end;

implementation

class method ConsoleApp.Main(args: array of string);
begin
  var myInvoice := new Invoice();
  myInvoice.Paid.Subscribe(n -> Console.WriteLine('Paid'));
  myInvoice.MarkPaid(DateTime.Now);
  Console.ReadLine();
end;

constructor Invoice;
begin
  Self.paidSubject := new Subject<Invoice>;
end;

method Invoice.MarkPaid(paidDate: DateTime);
begin
  Self.paidDate := paidDate;
  Self.paidSubject.OnNext(Self);
end;

It’s true that the examples above can be replicated quite easily without using the Rx Framework but within a short time of experimenting with the Rx Framework, I would hope that you should be able to see that it allows asynchronous event handling patterns to be expressed in a standard and more expressive manner.

I realise that this post is a little short on more practical examples but in my next post I’ll expand on some slightly more practical examples and some examples which demonstrate the Frameworks Power in handling GUI events in WPF Applications. It is also worth noting that the Rx Team recently released the RxJs framework which brings the Reactive Programming environment to Javascript.

Further Reading

For further information I would recommend:

• The Rx Wiki
• Wes Dyer’s introduction to the Rx Framework
• Wes Dyer and Erik Meijer from MS Research explaining the Rx Framework
• 101 Rx Samples (not quite 101 yet!)

Does the framework appeal to you? As always, if you have any thoughts or questions about using the Rx Framework with Delphi Prism then let me know in the comments below.

Last time, I had created an attribute which could only be attached to methods returning strings and used a home-made caching class, the Aspect looked like this:

namespace AOPCacheLibrary;

interface

uses
  RemObjects.Oxygene.Cirrus.*,
  RemObjects.Oxygene.Aspects;

type
   [AttributeUsage(AttributeTargets.Method)]
  MemCacheAttribute = public class(System.Attribute, IMethodImplementationDecorator)
  private
  public

    method HandleImplementation(Services: IServices; aMethod: IMethodDefinition);
  end;

implementation

method MemCacheAttribute.HandleImplementation(Services: IServices; aMethod: IMethodDefinition);
begin
  // Get a Reference to our Result Value
  var newResult := new ResultValue();

  // Get our MemCache Object
  var cacheType := Services.FindType('AOPCacheLibrary.MemCache');
  // Setup our static methods.
  var getDataProc := new ProcValue(new TypeValue(cacheType), 'getData', [new NamedLocalValue('cacheKey')]);

  // Assign the result of our cache request to a local variable.
  var getDataAssignment := new AssignmentStatement(new NamedLocalValue('strData'), getDataProc);

  // CACHE DATA USED: To assign our Cache'd data to the Method Result
  var useCacheData := new AssignmentStatement(newResult, new NamedLocalValue('strData'));
  // CACHE DATA NOT USED: To assign the result of our OriginalBody to the Method Result.
  var useOriginalMethodResult := new AssignmentStatement(new NamedLocalValue('resultOriginal'), newResult);

  aMethod.SetBody(Services,
    method begin
      var strData: String;
      // Get our Cache Key Name
      var cacheKey := MemCache.getKeyFromMethod(Aspects.MethodName, Aspects.GetParameters);

      try
        // See if we have anything in the cache for this key.
        unquote(getDataAssignment);
        // Replace this
        unquote(useCacheData);
      except
        on E: Exception do
        begin
          var resultOriginal: String;
          // No cache data found, execute the original method body.
          Aspects.OriginalBody;

          // Assign the result of the Original Method to a new variable.
          unquote(useOriginalMethodResult);
          // Send the result of the Original Method to the Cache for next time.
          MemCache.setData(cacheKey, resultOriginal);
        end;
      end;
    end);
end;

end.

Modifying my original code into working with the Microsoft Enterprise Caching black wasn’t particularly hard. If you haven’t worked with the Microsoft P&P Enterprise Library Caching Block before I highly recommend the Quickstart guide. The Enterprise Caching Block allows you to configure your cache using configuration file and change the backing store to suit own your needs. I would certainly recommend tweaking the expiration time and frequency of cache cleaning times before you begin.

Whilst rewriting the code of my Aspect I found it absolutely invaluable to have the Cirrus documentation wiki open and a piece of paper next to my keyboard with the approximate psuedo-code that I was intending to build with Cirrus written on it. I did actually find that despite the fact that the code I was building was not complicated, I did require a reminder of the statements and structure of it.

One additional problem that I did encounter very quickly on my modification was that I had a method which would generate a string key from the method name being called and its parameters which looked like this:

class method CacheKeyFactory.GetKeyFromMethod(aName: String; Args: Array of object): String;
begin
  var argConcat: String;

  argConcat := aName;
  for argTemp in Args do
  begin
    argConcat := argConcat + argTemp.ToString();
  end;

  Result := argConcat;
end;

Which I had intended to inject into the target class using the AutoInjectIntoTargetAttribute which as the name suggests creates the method tagged within the class you are targetting with your attribute. The problem I encountered was that I used this Attribute on a method within an IMethodImplementationDecorator attribute which meant that it worked perfectly when I applied my attribute to just one method within a class but applying my attribute to more than one meant that the AutoInjected method got injected more than once – causing a conflict (Is this a bug? I was told not but it seems odd to me).

[Update: 22/11/2009] I have reported this issue as QC: 79705. [/Update]

Therefore as a workaround, I was forced to move this key generating method into a static class elsewhere in the assembly but maybe someone can work out a way around this limitation.

I also encountered an annoying trait of the Cirrus Framework which is that some errors produce a fairly unhelpful message and lead you to a line in the Project’s build file which whilst being logical can be infuriating when you’re presented with this:

A rather confusing exception and causal code identification.

After a tiny bit of tweaking, My next iteration of the Caching Attribute looks like this:

type
  [AttributeUsage(AttributeTargets.Method)]
  MethodCacheAttribute = public class(System.Attribute, IMethodImplementationDecorator)
  private
  public
    method HandleImplementation(Services: IServices; aMethod: IMethodDefinition);
  end;

implementation

method MethodCacheAttribute.HandleImplementation(Services: IServices; aMethod: IMethodDefinition);
begin
  // Get a Reference to our Result Value
  var methodResult := new ResultValue();

  // Get a reference to our Static Key Factory.
  var cacheKeyGen := Services.FindType('CirrusCachingAspect.CacheKeyFactory');
  // Generate our cache key from the method name and it's parameters
  var getCacheKey := new AssignmentStatement(new NamedLocalValue('cacheKey'), new ProcValue(new TypeValue(cacheKeyGen), 'GetKeyFromMethod', [aMethod.Name, aMethod.GetParameterArrayValue()]));

  // Get our CacheManager Type and our CacheFactory Type
  var cacheType := Services.FindType('Microsoft.Practices.EnterpriseLibrary.Caching.CacheManager');
  var cacheFactoryType := Services.FindType('Microsoft.Practices.EnterpriseLibrary.Caching.CacheFactory');

  // Get a Reference to our Cache Instance.
  var getCacheInstance := new AssignmentStatement(new NamedLocalValue('methodCache'), new ProcValue(new TypeValue(cacheFactoryType), 'GetCacheManager'));

  // Get data from our cache.
  var getDataFromCache := new ProcValue(new NamedLocalValue('methodCache'), 'GetData', [new NamedLocalValue('cacheKey')]);

  // Cast our result from the Cache to the Result Type.
  var valueCastToResultType := new UnaryValue(new NamedLocalValue('cacheData'), UnaryOperator.Cast, aMethod.Result);

  // Assign the result of our cache request to a local variable.
  var getDataAssignment := new AssignmentStatement(new NamedLocalValue('cacheData'), getDataFromCache);

  // Assign our Cache'd data to the Method Result
  var useCacheData := new AssignmentStatement(methodResult, valueCastToResultType);

  // Add our data to the Cache for next time
  var addDataToCache := new StandAloneStatement(new ProcValue(new NamedLocalValue('methodCache'), 'Add', [new NamedLocalValue('cacheKey'), methodResult]));

  // Key not found in the cache, insert the original body and add a call to the Cache.Add() method.
  var notFoundInCache := new BeginStatement();
  notFoundInCache.Add(new PlaceHolderStatement());
  notFoundInCache.Add(addDataToCache);

  // If found in the cache.
  var ifCacheResult := new IfStatement(new BinaryValue(new NamedLocalValue('cacheData'), new NilValue(), BinaryOperator.Equal), notFoundInCache, useCacheData);

  aMethod.SetBody(Services,
    method begin
      var cacheData: Object;
      var methodCache: CacheManager;
      var cacheKey: String;

      // Get out cache instance
      unquote(getCacheInstance);

      // Generate a key for this method call.
      unquote(getCacheKey);

      // Try to retrieve the data from the cache.
      unquote(getDataAssignment);

      // If data from cache = nil then
      //    call original method
      //    add to cache
      // else
      //    return the data from the cache
      unquote(ifCacheResult);

    end);
end;

Which may look quite complex at first but when you look at the actual statements and values being passed around, you can quickly get a feel for how the code works. We can then easily apply this attribute and cache the result of any method in a class automatically like this:

type
  Flickr = public class
  private
    _username: string;
  public
    method Flickr(username: String);
    [Aspect:MethodCache]
    method GetPhotos: XmlDocument;
    [Aspect:MethodCache]
    method GetUsername: String;
    [Aspect:MethodCache]
    method GetAuthLevel(user: String): Integer;
  end;

Where GetPhotos, GetUsername and GetAuthLevel will all have their results cached automatically by our Aspect. You can see the resulting AOP’d code disassembled in the reflector below:

The RedGate .NET Reflector showing our Cirrus doctored caching code.

There is still one slight limitation in our method of key generation which is that it will currently only generate a unique key for a method with parameters if the Types of the methods parameters provide a proper implementation of ToString() that identifies them. Before you mention it, I had considered using the GetHashCode() method for this purpose but the default implementation of the GetHashCode method does not guarantee unique return values for different objects which makes it no more useful than relying on the user to implement the ToString() method.

I have made my implementation available on GitHub as the CirrusCachingAttribute where you can feel free to download and play with the source but please note that you will need to download and install the Enterprise Library from Microsoft first. I have also added a stub to the Code section on my site for the Cirrus Caching Attribute.

There is also a Standard Library of Aspects over on code.remobjects.com which provides a good set of Aspects from which you can learn more about how Cirrus works.

So, I decided to do something about it. Some of you might have seen some of the marvellous single purpose websites floating around the internet such as:

• Lets turn this f***ing website yellow
• Is the Apple Store Down?
• Do websites need to look the same in every browser?
• Is Lost a repeat?
• and Yet another useless website.

All are simple, single purpose websites, which generally do exactly what they say on the tin (or in the URL as is the case here) and as we know: The information they give must be true after all: “I read it on the internet, so it must be true”. So, late last night I got thinking..

May I proudly introduce to you, the Delphi community’s new and hopefully favourite single serving sites:

• www.isdelphidead.com
• www.isdelphidying.com

Next time you see someone on a forum, in the newsgroups or on stackoverflow asking the most dull and tedious of all the questions I could possibly hear: Is Delphi Dying? or Is Delphi Dead?,  point them to one of these sites.

Is Delphi Dead? Is Delphi Dying?

I should point out that the people who like to bring up this particular topic are normally quite persistent. In order to dissuade them from rehashing the same tedious and dull discussions time and time again I have cunningly built in an API.

People who are concerned that that answer might change without them being informed can build an application based on the API which spits out your choice of XML or JSON. This way, the aforementioned doom mongerers can simply build an application (in Delphi, of course) that sits in their tray and periodically polls the service and reassures those greatly concerned of the answer.

The API is very simple, you merely need to append

/API

to the isdelphidead.com or isdelphidying.com domain of your choice and then request either format by querying for XML (default):

/API?xml

or JSON:

/API?json

It’s that simple.  So go now and spread the word so that the rest of us can carry on in peace.

[Update 01/11/09]: Russian Delphi programmer Valerian Kadyshev has posted below to inform us that he has in fact made a tray monitoring application (in Delphi of course!) so that concerned people can download his pre-made one if they don’t even want to goto the trouble of making it themselves. You can find version 1 of his “Is Delphi Dying Monitor” over on his site. Marvellous Work Valerian!

Before you get too hasty with building your own class generators it’s also worth noting that Microsoft’s xsd.exe tool can also be used with Delphi Prism (thanks to Jeroen Pluimers and Peter Nowotnick for sharing this tip with us).

Getting Started

In Order to get started you’ll need to reference the RemObjects.Oxygene.CodeModel.dll and add the CodeDom namespace from this plus System.CodeDom to your uses.

It is worth noting that automated code generation is like our work with the Delphi Prism AOP Cirrus framework in that the code required to create a small amount of target output code can become quite hard to read at a glance so organise it well.

We’re going to start off by creating a basic class for Saying Hello (highly innovative you understand! ) and then generating the code and writing it to a new .pas file. Our basic first steps in generating a class involve setting up the unit (with a CodeCompileUnit), the namespace (with a CodeNamespace) and the class itself (with a CodeTypeDeclaration):

namespace CodeDomGeneration;

interface

uses
  System.Linq,
  System.Reflection,
  System.IO,
  System.CodeDom,
  System.CodeDom.Compiler,
  RemObjects.Oxygene.CodeDom;

type
  ConsoleApp = class
  public
    class method Main;
  end;

implementation

class method ConsoleApp.Main;
const
  outputFile = 'uMyClass.pas';
var
  uNewUnit: CodeCompileUnit;
  MyHelloNS: CodeNamespace;
  MyNewClass: CodeTypeDeclaration;
  provider: CodeDomProvider;

begin
  uNewUnit := new CodeCompileUnit;

  MyHelloNS := new CodeNamespace('Hello');
  MyHelloNS.Imports.Add(new CodeNamespaceImport('System'));

  MyNewClass := new CodeTypeDeclaration('TSayHelloClass');
  MyNewClass.IsClass := true;
  MyNewClass.TypeAttributes := TypeAttributes.Public + TypeAttributes.Sealed;
  MyNewClass.Comments.Add(new CodeCommentStatement('This is our CodeDom created class  '));

  MyHelloNS.Types.Add(MyNewClass);
  uNewUnit.Namespaces.Add(MyHelloNS);

  // Get our CodeDomProvider
  provider := CodeDomProvider.CreateProvider('Oxygene');

  // Options.
  var options: CodeGeneratorOptions := new CodeGeneratorOptions();

  // Create our output filestream.
  var sourceWriter: StreamWriter;
  sourceWriter := new StreamWriter(outputFile);

  // Generate our code.
  provider.GenerateCodeFromCompileUnit(uNewUnit, sourceWriter, options);

  // Close our output filestream
  sourceWriter.Close();

  Console.WriteLine('Press Enter to exit..');
  Console.ReadLine;
end;

end.

If you run this code, you’ll find a new uMyClass.pas file has been generated in the output directory which contains little more than a class shell and a couple of comments: One to note that the source file was auto-generated by a tool and another to which is our own comment on the TSayHelloClass. You’ll also note that unlike the previous article, we’re using:

 CodeDomProvider.GenerateCodeFromCompileUnit();   

To Generate code from the Code Document Object Model rather than the Compile method we used last time. You could also compile directly from a CodeDom object as alluded to in my previous post on Dynamically compiling Delphi Prism code using the CodeDomProvider.CompileAssemblyFromDom method.

The Next step is to add a private field (CodeMemberField) and a public property (CodeMemberProperty) to our class with its own getter and setter methods:

  var MyNameProperty: CodeMemberProperty;
  var MyNameField: CodeMemberField;

 // Add our private field.
  MyNameField := new CodeMemberField;
  MyNameField.Name := 'fFullName';
  MyNameField.Type := new CodeTypeReference('System.String');
  MyNameField.Attributes := MemberAttributes.Private;

  // Add the public property with a Getter and Setter
  MyNameProperty := new CodeMemberProperty;
  MyNameProperty.Name := 'FullName';
  MyNameProperty.Type := new CodeTypeReference('System.String');
  MyNameProperty.Attributes := MemberAttributes.Public;
  MyNameProperty.GetStatements.Add(new CodeMethodReturnStatement(new CodeFieldReferenceExpression(new CodeThisReferenceExpression(), 'fFullName')));
  MyNameProperty.SetStatements.Add(new CodeAssignStatement(new CodeFieldReferenceExpression(new CodeThisReferenceExpression(), 'fFullName'), new CodePropertySetValueReferenceExpression()));

  // Add our two new objects to the parent code tree
  MyNewClass.Members.Add(MyNameField);
  MyNewClass.Members.Add(MyNameProperty);

It could very easy to get carried away creating an initialising your new class, methods, fields or properties and forget to .Add() them to the target CodeStatementCollection so if your new objects do not show up in the generated code, check that you have actually added them. We can now add a public SayHello method to our class which will greet the previously named entity:

  var MySayHello: CodeMemberMethod;

  // Our public method to say "Well Hi there <fFullName>"
  MySayHello := new CodeMemberMethod();
  MySayHello.Name := 'SayHello';
  MySayHello.ReturnType := new CodeTypeReference(typeof(System.String));
  MySayHello.Attributes := MemberAttributes.Public;
  MySayHello.Statements.Add(new CodeMethodReturnStatement(new CodeBinaryOperatorExpression(new CodePrimitiveExpression('Well Hi there '), CodeBinaryOperatorType.Add, new CodeFieldReferenceExpression(new CodeThisReferenceExpression(), 'fFullName'))));

  // Add our new method to the parent class.
  MyNewClass.Members.Add(MySayHello);

The Method statements that I have added here have been created on the fly and not been structured well for readability at all. As I mentioned at the beginning, you will need to structure your code carefully to ensure readability. If you run this code again now, you should see our SayHello Method as below:

method TSayHelloClass.SayHello: System.String;
begin
    exit(('Well Hi there ' + self.fFullName));
end;

You can also add statements from “snippets“, which allows you to essentially inject code as a string using the CodeSnippetExpression class:

  var snippetSayHelloMethod:  CodeSnippetExpression  := new CodeSnippetExpression('Result := Self.fFieldName');
  //Convert the snippets into Expression statements.
  var stmtSayHelloResult: CodeExpressionStatement := new CodeExpressionStatement(snippetSayHelloMethod);
  // Add it to our SayHello Method
  MySayHello.Statements.Add(stmtSayHelloResult);

Warning: Adding method statements from snippets in this manner should be used with great care as you have very little protection against a typo causing a mal-formed code output and I would not recommend generating complex code statements from the CodeSnippetExpression.

There is a Code object representation of anything that you could want to generate, you simply need to find the right class. For example, if we wanted to add a class entry point to our class then we could create one with the CodeEntryPointMethod class:

  var MyMain: CodeEntryPointMethod := new CodeEntryPointMethod();
  MyMain.Name := 'Main';
  MyMain.Attributes := MemberAttributes.Public + MemberAttributes.Static;
  var cs1: CodeMethodInvokeExpression  := new CodeMethodInvokeExpression(
        new CodeTypeReferenceExpression('System.Console'), 'WriteLine',
        new CodePrimitiveExpression('Hello World!') );
  MyMain.Statements.Add(cs1);

  MyNewClass.Members.Add(MyMain);

Conclusion

The System.CodeDom space is a very very powerful tool and one that developers used to developing native Win32 Delphi might not be aware of. You will probably be able to dream up of a number of cases where generating some of your Delphi Prism code dynamically could save you time. Use it wisely.

It is my intention that this post should introduce you to the process of dynamically compiling code and that the next part will hopefully be an introduction to the process of dynamically generating and compiling code using the .NET CodeDom.

To begin with, create a New Console Application in Delphi Prism and Add a Reference to the RemObjects.Oxygene.CodeModel assembly. For my installation, this was found in the Bin directory of the Delphi Prism program files directory.

Once you’ve done this you can use the following code:

namespace DynamicCompilation;

interface

uses
  System.Collections,
  System.CodeDom.Compiler,
  RemObjects.Oxygene.CodeDom;

type
  ConsoleApp = class
  public
    class method Main;
  end;

implementation

class method ConsoleApp.Main;
var
  provider: CodeDomProvider;
  strFileOut: string;
  results: CompilerResults;
  cp: CompilerParameters;
  strSource: String;
begin

strSource := "namespace HelloWorld;

interface

type
  ConsoleApp = public class
  public
    class method Main;
  end;

implementation

class method ConsoleApp.Main;
begin
  Console.WriteLine('Hello World.');
end;
end.";

  // Get our CodeDomProvider
  provider := CodeDomProvider.CreateProvider('Oxygene');

  cp := new System.CodeDom.Compiler.CompilerParameters;
  cp.CompilerOptions := '';
  // Defaults to false which creates a dll instead.
  cp.GenerateExecutable := true;
  strFileOut := 'C:\test.exe';
  cp.OutputAssembly := strFileOut;

  // Compile our code.
  results := provider.CompileAssemblyFromSource(cp, [strSource]);

  Console.WriteLine('Number of Errors Encountered: {0}', results.Errors.Count);
  for error: CompilerError in results.Errors do
      begin
        Console.WriteLine('{0}: {1}', [error.ErrorNumber, error.ErrorText]);
      end;
  if (Assigned(results.PathToAssembly)) then
     begin
       Console.WriteLine('Path to Assembly: {0}', results.PathToAssembly);
     end;

  Console.WriteLine('Press Enter to exit..');
  Console.ReadLine;
end;

end.

Our Test Console Output

This code should be fairly straight forward to understand but there are a few parts that are worth drawing attention to. The first is this line:

provider := CodeDomProvider.CreateProvider('Oxygene');

I originally tried to create a new instance of the RemObjects.Oxygene.CodeModel.OxygeneCodeProvider as you can when creating a CSharpCodeProvider:

            var provider_options = new Dictionary
                         {
                             {"CompilerVersion","v3.5"}
                         };
            var provider = new Microsoft.CSharp.CSharpCodeProvider(provider_options);

However in my experience calling the constructor on the OxygeneCodeProvider class merely causes a Null Reference Exception when you actually try to use a Compile method on the resulting class. It is worth noting that by replacing the string ‘Oxygene’ above with ‘VisualBasic’ or ‘CSharp’ you can obtain an instance of other language providers.

The next part worth paying careful attention to is the setup of the CompilerParameters:

  cp := new System.CodeDom.Compiler.CompilerParameters;
  cp.GenerateExecutable := true;
  strFileOut := 'C:\test.exe';
  cp.OutputAssembly := strFileOut;

This part of the code, configures the Provider to generate an executable instead an assembly (which is the default configuration) and tells the Provider where to output the resulting executable. This should be relatively straightforward and for a full reference of CompilerParameters – see the MSDN documentation on System.CodeDom.Compiler.Compilerparameters.

Those sharp-eyed amongst you may have noticed that the code that I’m compiling above does not contain any uses declarations. First we need to Reference the relevant assemblies using the CompilerParameters.ReferencedAssemblies StringCollection:

  cp := new System.CodeDom.Compiler.CompilerParameters;
  // Clear our Assembly List
  cp.ReferencedAssemblies.Clear;
  // Add a few usual suspects
  cp.ReferencedAssemblies.Add('System.dll');
  cp.ReferencedAssemblies.Add('System.Core.dll');

You can then “uses” any namespace in the referenced assemblies (be careful: There is no official .NET assembly called System.Linq.dll for example).

The final part which is worth drawing attention to is the actual compile command:

   results := provider.CompileAssemblyFromSource(cp, [strSource]);

For reference, the relevant parts of the declaration for a CodeDomProvider might look something like the following:

public CodeDomProvider = class abstract(Component)
    public function CompileAssemblyFromDom(options: CompilerParameters; [ParamArray] compilationUnits: CodeCompileUnit[]): CompilerResults; virtual;
    public function CompileAssemblyFromFile(options: CompilerParameters; [ParamArray] fileNames: string[]): CompilerResults; virtual;
    public function CompileAssemblyFromSource(options: CompilerParameters; [ParamArray] sources: string[]): CompilerResults; virtual;

You can feed the CompileAssemblyFrom methods several different files, strings, or Dom structures which might represent different .pas files that you would create when you created a project in Delphi Prism. The example below includes an additional class:

namespace DynamicCompilation;

interface

uses
  System.Collections,
  System.CodeDom.Compiler,
  RemObjects.Oxygene.CodeDom;

type
  ConsoleApp = class
  public
    class method Main;
  end;

implementation

class method ConsoleApp.Main;
var
  provider: CodeDomProvider;
  strFileOut: string;
  results: CompilerResults;
  cp: CompilerParameters;
  strSource: String;
begin

strSource := "namespace HelloWorld;

interface

type
  ConsoleApp = public class
  public
    class method Main;
  end;

implementation

class method ConsoleApp.Main;
var
  PingPong: TPingPong;
begin
  PingPong := new TPingPong;
  Console.WriteLine(PingPong.Ping);
  Console.WriteLine('Hello World.');
end;
end.";

var strSource2 := "namespace HelloWorld;

interface

type
  TPingPong = public class
  public
    method Ping: string;
  end;

implementation

method TPingPong.Ping: string;
begin
  Result := 'PONG!';
end;
end.";

  // Get our CodeDomProvider
  provider := CodeDomProvider.CreateProvider('Oxygene');

  cp := new System.CodeDom.Compiler.CompilerParameters;

  cp.CompilerOptions := '';
  // Defaults to false which creates a dll instead.
  cp.GenerateExecutable := true;
  strFileOut := 'C:\test.exe';
  cp.OutputAssembly := strFileOut;

  // Clear our Assembly List
  cp.ReferencedAssemblies.Clear;
  // Add a few usual suspects
  cp.ReferencedAssemblies.Add('System.dll');
  cp.ReferencedAssemblies.Add('System.Data.dll');

  // Compile our code.
  results := provider.CompileAssemblyFromSource(cp, [strSource2, strSource]);

  //Console.WriteLine('Generated assembly name: ' + results.CompiledAssembly.FullName);
  Console.WriteLine('Number of Errors Encountered: {0}', results.Errors.Count);
  for error: CompilerError in results.Errors do
      begin
        Console.WriteLine('{0}: {1}', [error.ErrorNumber, error.ErrorText]);
      end;
  if (Assigned(results.PathToAssembly)) then
     begin
       Console.WriteLine('Path to Assembly: {0}', results.PathToAssembly);
     end;

  Console.WriteLine('Press Enter to exit..');
  Console.ReadLine;
end;

end.

And voila.. the output from our dynamically compiled executable can be seen as below:

The output from our dynamically generated executable

If the other compiler methods available with our CodeProvider caught your eye then this leads me nicely onto my intended next steps. While generating code from a string containing your code is nice and easy, sometimes it may not be the best way to approach Code Generation.

I realise that the end result of the above could have been achieved by simply writing a file out and then executing the command line compiler but where’s the fun in that? Next time I intend on looking at generating a CodeDom tree and providing this to the Compiler (no more potentially troublesome string manipulation required).