5

It’s not a secret that I’m a big advocate of Continuous Delivery and I’ve also found that I’ve been using a lot of Ruby and Python development workflow tooling recently because I like those languages in this setting and I find the tools to be relatively easy to use (I recently wrote about Fabric for push deployments).

I wanted a build tool to run the builds of my multi-platform targeted Delphi XE2 application because it was getting tedious to publish. As with any Continuous Delivery pipeline, the first step is to map out the steps involved in getting the application to the customer. For this particular application, there were several steps:

  1. Build the application in Release configuration for each platform (Win32 and MacOS32)
  2. Compile a zip file for each platform
  3. Create a manifest XML file with the version for auto-update purposes
  4. Upload the applications binary zip files to Amazon S3
  5. Upload the manifest to Amazon S3

Rake is a fairly mature Ruby based build tool which Martin Fowler has written about in the past. There were a couple of articles about using Rake with Delphi, one from Ed Vander Hoek and another from Shawn Oster but they were both a little out of date for my purposes. So I thought I’d cover enough Rake to get you started with a Delphi Rakefile. The msbuild driven versions of Delphi make building your Delphi project considerably simpler than trying to wrangle the appropriate dcc32 flags.

If you’re already familiar with Ruby or Python then you will find building Rakefiles much easier. If you’re not familiar with ruby, I’d suggest RubyMonk.

To start with, we define a function which will run msbuild after running rsvars.bat to setup the environment:

  def release_project(project_file, platform) 
    buildcmd = "msbuild #{project_file} /t:Rebuild /p:Config=Release /p:Platform=#{platform}";
    result = `"#{RAD_STUDIO_BIN_PATH}\rsvars.bat"&#{buildcmd_win}`
    if result.include? 'Build succeeded.'
      puts "Successfully built Windows 32 project #{project_file}"
    else
      puts result
      puts "Our build failed, there were build errors!"
      raise "Errors during Windows build"
    end
  end

We can then call put this into a simple rake task to build the Win32 version like this:

  desc "Builds MyProject.dproj for Win32 in release configuration"
  task :build_release do
    puts 'Building Project MyProject.dproj..'
    release_project('MyProject.dproj', 'Win32')
  end

We can then call this via:

> rake build_release

We can also list the tasks that rake has via:

> rake -T 
rake build_release       # Builds MyProject.dproj for Win32 in release configuration

We can adapt this method to enable us to run any DUnit tests and code coverage tests.

In order to find the version of our executable, we need to call GetFileVersionInfo on the Windows API. Luckily, there was a StackOverflow question on calling GetFileVersionInfo with Ruby. This gives us something like this:

  def get_version(artefact_path)
    require "Win32API"
    s=""
    vsize=Win32API.new('version.dll', 'GetFileVersionInfoSize', 
                       ['P', 'P'], 'L').call(artefact_path, s)
    if (vsize > 0)
      result = ' '*vsize
      Win32API.new('version.dll', 'GetFileVersionInfo', 
                   ['P', 'L', 'L', 'P'], 'L').call(artefact_path, 0, vsize, result)
      rstring = result.unpack('v*').map{|s| s.chr if s<256}*''
      r = /FileVersion..(.*?)\000/.match(rstring)
      "#{r ? r[1] : '??' }"
    else
      raise "GetFileVersionInfoSize returned 0 for #{artefact_path}"
    end
  end

We can create our zip file in two different ways, we can either use a zip gem for Ruby native creation or we can shell out to a command line version. This function uses the zip gem and takes an array of files to zip and a name for the zip file.

  def make_zip_distro(files_to_zip, zip_name)
    require 'zip/zip'

    Zip::ZipFile.open(zip_name, Zip::ZipFile::CREATE) { |zipfile|
      files_to_zip.each { |file_to_zip|
        if File.exists?(file_to_zip) 
          zipfile.get_output_stream(File.basename(file_to_zip)) { |f| 
            File.open(file_to_zip) do |in_file|
              while blk = in_file.read(1024**2)
                f << blk
              end
            end
          }
        else
          raise "Could not find #{file_to_zip}"
        end
      }
    }
  end

For the manifest file, we can either generate it using Ruby string interpolation or a template system like erb, depending on how complex your requirements are.

The final step is the upload all artefacts (the manifest, the artefacts):

  # You need to define AWS_ENDPOINT, AWS_BUCKET, AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID
  def upload_to_s3(file, s3_destination)
    require "aws/s3"
    if File.exists?(file)
      AWS::S3::DEFAULT_HOST.replace AWS_ENDPOINT
      AWS::S3::Base.establish_connection!(
        :access_key_id     => AWS_ACCESS_KEY_ID,
        :secret_access_key => AWS_SECRET_ACCESS_KEY
      )
      AWS::S3::S3Object.store(s3_destination, 
                          open(file), 
                          AWS_BUCKET,
                          :access => :public_read)
      puts "Uploaded #{file} to #{s3_destination}"
    else
      puts "Could not file #{file} to upload to S3"
    end
  end

You will need to define the access keys, secret keys, endpoints and bucket.

The next step is to put it into composite parts and draw up a set of tasks. Rake tasks can be standalone or can have prerequisite dependencies:

  task :my_task => [:dependent_task_1, :dependent_task_2] do
    # task implementation
  end

Rake tasks can also have parallel prerequisite dependencies like this:

 multitask :my_task => [:build_task1, :build_task2] do
    puts "Built all the things!"
 end

Rake tasks can also have parameters, this might be useful if you wanted to pass in a version number manually at build or publish time.

Back to our project pipeline from earlier, here are a selection of tasks that wrap everything together:

  desc "Builds the Win32 release of MyProject.dproj"
  task :build_win_release do
    puts 'Building Project MyProject.dproj for Windows'
    release_project('MyProject.dproj', 'Win32')
  end

  desc "Builds the Mac OS X release of MyProject.dproj"
  task :build_mac_release do
    puts 'Building Project MyProject.dproj for Mac'
    release_project('MyProject.dproj', 'MacOS32')
  end

  desc "Writes the manifest out to manifest.xml in the current working dir"
  task :write_manifest => [:build_win_release] do
    puts 'Writing the update manifest'
    v = get_version(get_executable_path(:Win32))
    write_update_manifest_to(File.join(get_cwd(), 'manifest.xml'), v)
  end

  desc "Uploads the manifest to Amazon S3"
  task :upload_manifest => [:write_manifest] do
    puts 'Uploading the manifest..'
    upload_to_s3(File.join(get_cwd(), 'manifest.xml'), 'myproject/manifests/manifest.xml')
  end

  desc "Builds the Windows zip distributable"
  task :make_win_zip => [:build_win_release] do
    exe = get_executable_path(:Win32)
    v = get_version(exe)
    zip_name = 'MyProject-win-' + v + '.zip'
    make_zip_distro([exe], zip_name) 
    puts "Create zip: " + zip_name
  end

  desc "Builds the Mac zip distributable"
  task :make_mac_zip => [:build_mac_release] do
    exe = get_executable_path(:MacOS32)
    # Call get_version on the Win32 executable. If the Win and Mac OS X version numbers 
    # differ, you will need to extract the version from the generated Info.plist instead.
    v = get_version(get_executable_path(:Win32))
    zip_name = 'MyProject-mac-' + v + '.zip'
    make_zip_distro([exe], zip_name)
    puts "Create zip: " + zip_name
  end

  desc "Uploads all of the zip files"
  task :upload_zips => [:make_win_zip, :make_mac_zip] do
    v = get_version(get_executable_path(:Win32))
    win_name = 'MyProject-win-' + v + '.zip'
    mac_name = 'MyProject-mac-' + v + '.zip'
    puts 'Uploading the win zip version .. ' + v
    upload_to_s3(win_name, 'myproject/downloads/' + win_name) 
    puts 'Uploading the mac zip version .. ' + v
    upload_to_s3(mac_name, 'myproject/downloads/' + mac_name)
  end

  desc "Builds, zips and uploads the artefacts and manifests"
  task :release_all => [:upload_zips, :upload_manifest]

You can build your releases, create the manifest and upload all with:

> rake release_all 

Simple, repeatable and easy to extend when you have more steps to your pipeline. Further steps for your project might be adding in your acceptance tests or automatically generating and publishing some release notes from your git logs.

Other Links

5 Comments

  1. ObjectMethodology.com on the 25th February 2013 remarked #

    Hey cool, a continous delivery article.

  2. Joseph on the 26th February 2013 remarked #

    Quick question: once you’ve programmed with Python, how in the world do you possibly go back to programming with Delphi? I’ve just begun learning Python and already I’m doing things like going to Nick Hodges’ blog and rewriting all his code samples into one line of Python code. 🙂

    It’s not just the language – although the dynamic typing vs. super-strict typing is certainly part of it. The high quality of the tools, the plethora of books, magazines, videos, etc., including free, the conferences, the 28,000 libraries like SQLAlchemy, Pandas, Matplotlib, IPython, etc., being able to interface with everything, the kinder, gentler open source community that values your input, and the price of free doesn’t hurt either.

    I’ve been programming in Pascal since Turbo Pascal as a kid but now that I’ve found Python… I just can’t imagine going back to declaring all those laborious types, all the single-pass compiler holdover behavior, etc. How can you do it?

  3. Jamie I on the 26th February 2013 remarked #

    ObjectMethodology: 😉

    Joseph: I always enjoy Python and Ruby, as you say, the languages themselves are enjoyable to use but the communities and the support for so many libraries and interfaces in such an accessible way is very liberating.

    I also love the way there is a strict convention guide in PEP 8 (http://www.python.org/dev/peps/pep-0008/) that you can validate against.

    Having said that, in our world, it’s always a case of identifying the right tool for the job…

  4. Phil on the 3rd July 2013 remarked #

    @Joseph
    About Python & Delphi I am not really right with you.
    First I really like super-strict typing for a lot of reasons.
    For all the other parts of the language I prefere Python to Delphi.
    So why always using Delphi ?
    It’s because it’s not only a language but also an IDE which can generate quickly, clean user interfaces.

  5. Phil on the 3rd July 2013 remarked #

    Good post ! For version number management, since XE2, I use an external .rc file (with {$R ‘fileVersion.res”FileVersion.rc’} in my .dpr) so it’s more easy to centralise the version number.

Leave a Comment