things learnt while coding on pyshapelib // Bramz' warehouse

things learnt while coding on pyshapelib

March 22nd, 2007

The last couple of weeks I’ve mostly been coding on pyshapelib, that’s a small Python wrapper around shapelib, a C++ library to read ESRI shapefiles. It is originally written by Bernhard Herzog and is now maintained as part of Thuban, an open source interactive geographic data viewer. My contribution to pyshapelib is rewriting it to use hand-crafted Python bindings instead of SWIG generated ones, to support shapefiles with height (Z) or measurement (M) information, and to have better support for Unicode strings.

Until now it’s been quite interesting. I’ve learnt some new things about writing Python extension modules, for example that METH_VARARGS functions do not take keyword arguments (something we assumed otherwise in the Lass Python API =) (update: it seems we’ve gotten that right after all). But the best surprise was this one:

mixing msvcr71.dll and msvcr80.dll can be really inauspicious

I’m not yet sure what exactly is going on, but when I tried to import my freshly built wxproj (part of Thuban) in Python, LoadLibraryEx in dynload_win.c failed resulting in a nice error message in the style of …

    from wxproj import point_in_polygon_shape, shape_centroid
ImportError: DLL load failed: A dynamic link library (DLL) 
initialization routine failed.

… and a nice message box:

R6034 An application has made an attempt to load the C runtime library incorrectly. Please contact the application's support team for more information

It’s always nice to get the advice to contact yourself for more information, it’s so helpful =)

Opening wxproj.pyd in Dependency Walker revealed that wxproj.pyd was dependent on both msvcr71.dll (through wxmsw26uh_vc.dll of wxPython 2.6) and msvcr80.dll (through proj.dll of PROJ.4). wxmsw26uh_vc.dll came straight out of an installer, but proj.dll I have compiled myself.

From there, it was easy guess work. I replaced proj.dll with a version compiled with VC7.1, and the problems mysteriously vanished.

Thing learnt: when building Thuban yourself, strictly use DLLs built with VC7.1 =)

It still makes me wonder though. I’ve compiled extension modules with VC8 before, and I’ve never encountered this problem before. And yet, they all must have been indirectly dependent on msvcr71.dll through python24.dll (Python 2.4 is compiled with VC7.1). So why is it complaining in wxproj‘s case only?

Perhaps to be continued …

update: (04/04/2007) The error was caused by a missing manifest in proj.dll. The issue is finally resolved by changing the makefile.vc to embed the manifest, as following:

proj_i.lib:     $(LIBOBJ)
    link /debug /dll /def:proj.def /out:$(PROJ_DLL) \\
        /implib:proj_i.lib $(LIBOBJ)
    if exist $(PROJ_DLL).manifest mt.exe -manifest \\
        $(PROJ_DLL).manifest -outputresource:$(PROJ_DLL);2

A ticket has been submitted to PROJ.4’s BugZilla.

This entry was posted on Thursday, March 22nd, 2007 at 4:33 pm and is filed under coding. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.

4 Responses to “things learnt while coding on pyshapelib”

Tom Says:
March 27th, 2007 at 12:58 pm
Hi Bramz, welcome to SxS, Isolation and Manifest HELL :) The error you see is that you tried to load a dll without using a proper manifest. Anyway it is always a bad idea to mix runtime versions with each other but sometimes this is unavoidable.

Tom
Bramz Says:
April 3rd, 2007 at 2:17 pm
Hi Tom,

It took me quite some time to respond =) I already suspected it has to do with a missing manifest. I had a related problem with Bass last week, and my diggings on MSDN put me on trail of it. Of course, a search for R6034 on MSDN also helps =).

Anyway, I’ve found this MSDN page on how to embed a manifest using makefiles, which I tried for the proj.dll library. I believe I was successful in embedding it, but it didn’t help. I still get the same error.

What I really would like to have is a way to check if a DLL has its manifest embedded, because now I kind of staggering in the dark about what DLL is actually causing the troubles.
Bramz Says:
April 3rd, 2007 at 2:54 pm
OK, the fact that the DLL-with-the-supposedly-embedded-manifest had exactly the same size as the one without should have rung a bell =)

Apparently, you can use VS2005 to peek inside an EXE or DLL to see its resources. You simply open the binary like a regular file, and you get a list of the resources. If there’s one named RT_MANIFEST, then you’re lucky =)

So, I finally managed to build proj.dll with an embedded manifest, and now the error has disappeared.

PS: Python’s distutils seems to build the extension modules withouot embedded manifest … to be continued.
Homer Says:
August 13th, 2010 at 4:06 pm
Dude, thanks so much for keeping this thread up! I found your website while researching exactly the same problem. I have been chasing it for a day and a half. I tried a whole range of hair-brained suggestions without any success before I tried yours =)

I’m writing an application using wxWidgets and wxMathPlot and couldn’t get it to run on any computer other than my own. Turning on the “Embed Manifest” option in my VC 2005 project’s references (Configuration Properties->Manifest Tool) did the trick.

Bramz' warehouse