Dienstag, 14. Oktober 2014

Modular build scripts with NAnt

We are using NAnt for our build process. Since the product is large, this amounts to things like this:
  1. clean up the source tree => clean slate
  2. build all projects (~50 mins) => compiled libraries and executables
  3. run all tests (~2 hours)
  4. run benchmarks
  5. create installer (msi/bootstrapper)
  6. publish to a file share
This all amounts to several thousand lines of nant code. And because we know naught about modularizing things and decoupling things, this all becomes basically one big nant file. That is not maintainable.

We tried to modularize things a bit by putting common code into nant files, like this:
<include buildfile=".\installer.prop" />
which are just included by other nant files. So, we kind of reduced code duplication a bit.

Problems remain:
  • you don't know where your property has been set for the first time
  • if it has been reset to something else
Additionally, we have to deal with the problem that by using nant we have to use a declarative language to do loops, maintain file lists, have few insight in how properties are kept in memory.
It's even difficult to see the workflow of this whole monster.

I currently think, that for such large projects nant is the wrong tool. I'd like to try rake.
However, maybe we're just using nant the wrong way.

There is a way we should use more to modularize all those build scripts and make them independent:
  • separate nant files, each of which does one thing, and one thing well. Kinda like the Single Responsibility Principle. It's code, after all.
  • have integrating nant files which call other nant files. Those integrating nant files should not have logic.
  • when calling other nant files, do not inherit the properties of your current state (to inherit is actually the default).
In practice, this would look like this:
<project name="integrator" basedir=".">
    <property name="baseDir" value="${project::get-base-directory()}" />
    <target name="build">
        <!-- we integrate stupid targets here.
                they do not contain logic, they just call someone else -->
        <call target="cleanBuildfiles" />
        <call target="buildSoftware" />
        <call target="buildInstaller" />
        <call target="publishSoftware" />
    </target>
    <target name="buildSoftware">
        <!-- inheritall="false" prevents the called nant scripts from
                inheriting all what we already might have screwed up.
                It's a fresh start. An error in this script doesn't
                affect those other scripts. Modular :) -->
        <nant 
            buildfile="${baseDir}\myFramework\default.build" 
            target="buildAll"
            inheritall="false">
            <properties>
                <!-- if that nant script depends on a certain global
                        variable, inject it -->
                <property name="CCNetLabel" value="${CCNetLabel}" />
            </properties>
            </nant>
    </target>
    <target name="buildInstaller">
        <nant 
            buildfile="${baseDir}\installer\default.build"
            target="buildInstaller"
            inheritall="false" >
            <properties>
                <!-- or directly give an apt name 
                        to such input variables -->
                <property name="BuildNumber" value="${CCNetLabel}" />
            </properties>
            </nant>
    </target>
</project>
I can't yet say that this will make nant easier to use, but I already feel that I have a better overview of what is done where. Maybe it will help me keep order and find errors in the future.

Mittwoch, 19. Juni 2013

Authoring a patch bundle

After the main program is deployed the maintenance will have to deploy fixpacks and service packs. I wondered how I should handle this with burn and decided to use a separate patch bundle.

Versioning used in this sample is MajorNumber.ServicePack.FixPack. Each patch bundle also has a separate UpgradeCode, because when removing bundle v1.0.0.1 while bundle v1.0.0.2 was installed  might leave the system with the v1.0.0.1 patch (issues about superseded patches.. see here).

Now I had several bundles, which all were authored to be a patch for the main bundle. That looks like this:
<Bundle Name="PatchingBundle 1.0.0.1" Version="1.0.0.1"
    Manufacturer="Jakob Devs Inc." UpgradeCode="[patch_bundle_code_v1.0.0.1]"
    ParentName="BaselineBootstrapper">
    <RelatedBundle Id="[parent_bundle_code]" Action="Patch" />
</Bundle>

There are several things of note here:
  • The UpgradeCode of this patch bundle is new, this leads to a separate entry in ARP's updates
  • ParentName: this puts the bundle in ARP's update section under the name BaselineBootstrapper. This should be the same for all patches.
  • RelatedBundle element: here I specify that this bundle patches the <parent_bundle>. This leads to this bundle being uninstalled when the <parent_bundle> is uninstalled. Cool :)
    To remove a Sp1Fp1 patch when the service pack is uninstalled, the parent_bundle_code will be the UpgradeCode of the ServicePack patch bundle.
Once a service pack is deployed, there'll be patches which target the service pack. These msp patches won't install on systems which don't have the service pack installed. I talked about that topic here.
To get  a state on the system where the user can remove installed patches, the burn engine offered me basically one solution:
Install every patch under a different UpgradeCode, so that all patch bundles show up in ARP.

The patch itself could theoretically be installed as superseding or not, I didn't find any difference so far. However, burn does not remove superseded patches, that's why I opted for non-superseding patches. They still all contain the diff to the baseline, though. Baselines will be the RTM or SP releases.
[Edit]: Apparently, an upgrade of version 1.0.0.x to 1.0.1 (yep, a service pack) always makes the patch superseding. That also means, when I'd uninstall bundle 1.0.0.1 while 1.0.1 is installed, the superseded patch stays on the system.

Just for the sake of completeness: what would the complete sequence of main installer, service pack, fix pack for SP look like?
<Bundle Name="Main RTM Bundle 1.0.0.0" Version="1.0.0.0"
    Manufacturer="Jakob Devs Inc." UpgradeCode="[main_bundle_code]" >
    <!-- no related bundle here ... -->
</Bundle>

The service pack is basically again a patch, so we'll relate it to the main RTM bundle above.
<Bundle Name="PatchingBundle 1.0.1.0 - SP1" Version="1.1.0"
    Manufacturer="Jakob Devs Inc." UpgradeCode="[patch_bundle_code_v1.0.1.0]"
    ParentName="BaselineBootstrapper">
    <RelatedBundle Id="[main_bundle_code]" Action="Patch" />
</Bundle>

Then I would author the fix pack for this service pack as this, but I relate it to the service pack patch. Of course this fix pack should have the diff from the SP1 baseline to the patch. It doesn't have to use the RTM as the baseline anymore. It'll get uninstalled when the SP is uninstalled:
<project name="integrator" basedir=".">
 <property name="baseDir" value="${project::get-base-directory()}" />
 <target name="build">
  <call target="cleanBuildfiles" />
  <call target="buildSoftware" />
  <call target="buildInstaller" />
  <call target="publishSoftware" />
 </target>
 <target name="buildSoftware">
  <nant 
   buildfile="${baseDir}\myFramework\default.build" 
   target="buildAll"
   inheritall="false"/>
 </target>
 <target name="buildInstaller">
  <nant 
   buildfile="${baseDir}\installer\default.build"
   target="buildInstaller"
   inheritall="false" />
 </target>
</project>
Here it also makes sense to use the <bal:Condition> element so that the Sp1Fp1 only installs if the service pack for the main RTM bundle is installed.

Dienstag, 18. Juni 2013

Conditioning a wix bundle when it should or should not run

I started to have this question when I prepared a patch bundle which should only install when the main product is already installed. Additional complexity is given when a service pack was applied and patches require the service pack to be installed. In an additional post I explain how to author patch bundles and how to remove fix pack bundles on top of SPs along with the service pack bundle.

Wix bundles have two points where you can configure whether they should execute or not.

The first which you'll find is the bundle/@Condition attribute:
<Bundle
    Name="TestBundle" Version="1.0.0.0"
    Manufacturer="TestDeploy Inc." UpgradeCode=""
    Condition="((VersionNT = v6.0) AND (ServicePackLevel &gt;= 2)) OR (VersionNT &gt;= v6.1)"
>
<!-- bundle stuff -->
</Bundle>
Here the condition checks that at least NT in version 6.0 (Windows Server 2008) is installed with at least ServicePack 2, or that the windows versions bigger than 6.1 (Windows 7). Microsoft provides a list of the windows versions.

This can only be used with prebuilt wix burn variables. A list of those can be found here: http://wix.sourceforge.net/manual-wix3/bundle_built_in_variables.htm.

The second method requires the WixBalExtension's Condition element:
<bal:Condition
   Message="The Bootstrapper has to be installed in version $(var.BaselineVersion)">  
      WixBundleInstalled OR      
      ((SampleMsiInstalledState = 5) AND (SampleMsiInstalledVersion &gt;= v$(var.BaselineVersion)))
</bal:Condition>
<util:ProductSearch Guid="[msi_prerequisite_package_product_code]"
    Result="version" Variable="SampleMsiInstalledVersion" />
<util:ProductSearch Guid="[msi_prerequisite_package_product_code]"
    Result="state" Variable="SampleMsiInstalledState" />
Here we use a ProductSearch from the WixUtilExtension to find the state and versions of related msi packages. The version is then compared to the minimum version of a bundle that's required for the bundle (BasellineVersion).
I haven't found a way to search for installed bundles, but it is apparently possible to do a registry search for the bundle UpgradeCode. However, that's wix implementation details and should be used with care.

What i found important is: Use some kind of fallback condition, like the WixBundleInstalled (this variable is 1 if installed, 0 else). Without that, the SampleMsiInstalledState could be false because someone forced the bundle to uninstall, then the conditions would evaluate to false and the bundle won't run anymore. No user likes bundle corpses on the system.

Donnerstag, 11. April 2013

Setting up RaspberryPi with ArchLinux to automatically connect to WLAN

Got a RaspberryPi.
After simply dd'ing the ArchLinux onto an SD card, booting was possible, and composite output was working. Neat.
Plug in a wlan usb stick. Try to configure it like it was described here, but things just didn't work out fine. Problem was, the wlan0 device wasn't yet existing, when the boot process reached the starting of the wlan connection. Fail followed.

What kind of saved me, after I understood what to do, was this thread.
However, what I did was slightly different:
I called
systemctl --full
and search for the line providing something like this:
sys-devices-pci0000:00-0000:00:1c.1-0000:02:00.0-bcma0:0-net-wlan0.device
Then i put that into the file to After and BindTo
[edit]
Had to change my approach. Before, I changed the file /etc/systemd/system/network.service, but that kind of didn't work nicely. Probably, because network.service is not netcfg. The change in netcfg file below just adds two lines which tells Arch to wait with this netcfg service until the wlan is up and running:
After=sys-subsystem-net-devices-wlan0.device
BindsTo=sys-subsystem-net-devices-wlan0.device
so that we get the resulting file:
cat /etc/systemd/system/multi-user.target.wants/netcfg\@Stadtkater.service 
[Unit]
Description=Netcfg networking service for profile %i
Before=network.target
Wants=network.target
After=sys-subsystem-net-devices-wlan0.device
BindsTo=sys-subsystem-net-devices-wlan0.device

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/netcfg check-iface %i
ExecStop=-/usr/bin/netcfg down %i
KillMode=none

[Install]
WantedBy=multi-user.target
Not sure what I did there exactly and what's the difference between After and BindTo, but it worked and the raspberry pi connects now to the wlan after a fresh start. So now i can ssh and don't have to use a crappy TV signal anymore :)

Mittwoch, 20. März 2013

BURN #04: passing preprocessor variables into the build

A nice solution is explained here on stackoverflow:

it basically says that you should:

1st in the wixproj file add the following:
<Target Name="BeforeBuild">
    <CreateProperty Condition="$(BuildNumber) != ''"
Value="BuildNumber=$(BuildNumber);$(DefineConstants)">
      <Output TaskParameter="Value" PropertyName="DefineConstants" />
    </CreateProperty>
    <CreateProperty Condition="$(RevisionNumber) != ''"
Value="RevisionNumber =$(RevisionNumber);$(DefineConstants)">
      <Output TaskParameter="Value" PropertyName="DefineConstants" />
    </CreateProperty>
  </Target>
2nd in your msbuild task do this:
<MSBuild Projects="YourWixProject.wixproj" 
   Properties="BuildNumber=$(VerBuildNumber);RevisionNumber=$(RevisionNumber)" 
/>

Donnerstag, 14. März 2013

Writing custom NANT and msbuild tasks

Just a friendly reminder to reuse work of others.

See here for msbuild:
Nant help can be found here:
Or, since i only wanted to learn how to write a task which deletes directories which are older than XX:
Look at slashdot to delete all directories which are more than 7 days old