Sunday, May 2, 2010

Python distutils and manifest files (Visual Studio 2010)

Visual Studio 2005 introduced a new deployment model for Windows client applications based on isolated applications and side-by-side assemblies. The redistributable libraries (such as MFC, ATL, CRT) have been rebuilt as shared side-by-side assemblies and installed in the native assembly cache, WinSxS folder in Windows. This has provided a common location for particular versions of the dll-s and the applications had to declare their dependencies by using a manifest file. The manifest file is an XML file which contain references to the dependent dlls located in WinSxS, these files had to be deployed along with every dll or executable which depend on the redistributables. Without having the manifest (either by a separate file or embedded in the executable as a resource) the application generates an error.
Though the purpose of this change was to simplify deployment, the result was probably the opposite. As a result Microsoft changed deployment requirements in Visual C++ 2010. As of Visual Studio 2010 Microsoft has backed out the manifest generation and no manifests are generated by default with the VC2010 builds. There's no requirement to embed manifests in the executables anymore and CRT-s need not be installed in WinSxS. All you need to do is copy the VC++ dependent DLLs to the application folder and run.

This change may however cause issues for those build systems (like python distutils) which don't expect that the manifest may eventually be missing and stop the compilation in this condition. I don't see any change in this regard in the python SVN trunk either. Fortunately to prevent from this issue we could easily alter the corresponding file (/Lib/distutils/msvc9compiler.py) to something like:

   # embed the manifest
   # XXX - this is somewhat fragile - if mt.exe fails, distutils
   # will still consider the DLL up-to-date, but it will not have a
   # manifest. Maybe we should link to a temp file? OTOH, that
   # implies a build environment error that shouldn't go undetected.
   if os.path.isfile(temp_manifest):
        mfid = 1 if target_desc == CCompiler.EXECUTABLE else 2
        out_arg = '-outputresource:%s;%s' % (output_filename, mfid)
        try:
             self.spawn(['mt.exe', '-nologo', '-manifest',
                  temp_manifest, out_arg])
        except DistutilsExecError as msg:
              raise LinkError(msg)
        else:
              log.debug("skipping %s (up-to-date)", output_filename)

Which may work even if the deployment model will frequently be altered by Microsoft from versions to versions ;-)

Visual Studio 2010 (Win32/Win64) packages are now available

After some investigation of a couple of issues, I've finished to set up daily build packages compiled with Visual Studio 2010. Just as for the previous packages I've compiled all the dependecies with Visual Studio 2010 to prevent from potential memory corruption issues as described in my earlier post: Redistribution of the shared C runtime component in Visual C++

The packages can be downoaded from the usual location: http://vbkto.dyndns.org/sdk

In addition to the binary packages the build SDKs for VS2010 are also available (containing the libs and headers)

Although the builds seem to be in good condition (the auto tests are running correctly) don't hesitate to contact me if any further issues encountered.

Note on the MrSID driver: Currently there are no available VS2010 versions of the LizardTech GeoExpress Raster DSDK, therefore the MrSID plugin is not included in the Visual Studio 2010 packages. However you might want to copy (bin\lti_dsdk_dll.dll and bin\gdal\plugins\gdal_MrSID.dll) from the corresponding VS2008 packages in order to have MrSID support switched on.