Staging
v0.5.1
v0.5.1
https://github.com/python/cpython
Tip revision: c26ef13d23bab6328b4f9164b4eb3619c754c45f authored by cvs2svn on 30 September 2003, 07:10:26 UTC
This commit was manufactured by cvs2svn to create tag 'r232c1'.
This commit was manufactured by cvs2svn to create tag 'r232c1'.
Tip revision: c26ef13
mpwextensions.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN">
<HTML>
<HEAD>
<TITLE>HOWTO: Compiling Python Modules with MPW</TITLE>
</HEAD>
<BODY>
<H1>HOWTO: Compiling Python Modules with MPW</H1>
<blockquote>
This HOWTO is a slightly reformatted version of an original by
<A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, whose
<A HREF="http://www.nevada.edu/~cwebster/Python/">Python page</A>
may contain a more up-to-date version.
</blockquote>
<HR>
<P>
The <A HREF="http://www.cwi.nl/~jack/macpython.html">Macintosh version</A>
of the <A HREF="http://www.python.org/">Python programming language</A> is
usually compiled with <A HREF="http://www.metrowerks.com/">Metrowerks
CodeWarrior</A>. As a result, C extension modules are also usually
compiled with CodeWarrior, and the documentation and sample code reflects
this. CodeWarrior is a commercial product, and may be beyond the budgets
of hobbyist hackers, making them dependent on others to compile C extension
modules. At the present time, many standard C extension modules compile
"out of the box" on the Macintosh, but in only a few cases is the plugin
for the Macintosh included in the distribution.
</P>
<P>
The <A HREF="http://developer.apple.com/tools/mpw-tools/">Macintosh
Programmer's Workshop</A> (MPW) is Apple's development environment, and is
freely available for <A
HREF="ftp://ftp.apple.com/developer/Tool_Chest/Core_Mac_OS_Tools/MPW_etc./">download</A>
from Apple, as well as on their Developer CDs. Since Python was originally
developed using MPW, before CodeWarrior became the dominant MacOS
development environment, most of the idiosyncrasies of MPW are already
supported, and compilation of C extension modules in MPW is possible.
</P>
<P>
This HOWTO only deals with compiling for PowerPC Macintoshes. The process
should be similar for 68k Macintoshes using the code fragment manager, but
I have not attempted this - my old Mac is running NetBSD.
</P>
<P>
This way of compiling modules is still experimental. Please read the
caveats section below.
</P>
<H2><A NAME="setup">Setting Up MPW for Compiling Python Modules</A></H2>
<P>
This assumes that you have successfully installed both MPW and Python with
the Developer's Kit on your Macintosh.
</P>
<P>
The first step is to let MPW know where you keep Python. This step is not
strictly necessary, but will make development easier and improve
portability. Create a new file in the <CODE>Startup Items</CODE> folder of
MPW called <A HREF="Python"><CODE>Python</CODE></A>. Type the lines:
</P>
<PRE>
set Python "Macintosh HD:Applications:Python 1.5.2c1:"
set PythonIncludes "{Python}Include"
set PythonMacIncludes "{Python}Mac:Include"
set PythonCore "{Python}PythonCore"
export Python PythonIncludes PythonMacIncludes PythonCore
</PRE>
<P>
where <CODE>Macintosh HD:Applications:Python 1.5.2c1:</CODE> is replaced by
the path to the directory where you keep your copy of Python, and the other
variables reflect where you keep your header files and Python core files.
The locations here are the standard for Python 1.5.2c1, but they are
different for Python 1.52b2 and earlier (most notably, the PythonCore is
kept in the Extensions folder).
</P>
<P>
Next, you need to update the <A HREF="config.h"><CODE>config.h</CODE></A>
file for the <CODE>MrC</CODE> compiler included with MPW. This header file
is located in the <CODE>:Mac:Include</CODE> folder in the standard
distribution. You can update it by hand, by adding the lines:
</P>
<PRE>
#ifdef __MRC__
#define BAD_STATIC_FORWARD
#endif
</PRE>
<P>
at the after the similar defines for <CODE>__MWERKS__</CODE> and
<CODE>__SC__</CODE> in the file. This step is critical: many modules,
including ones in the standard distribution, will not compile properly
without this modification (see common problems below).
</P>
<P>
Copies of both the <A HREF="Python"><CODE>Python</CODE></A> startup item
for MPW and the <A HREF="config.h"><CODE>config.h</CODE></A> are included
here for your convenience.
</P>
<P>
If you are porting Unix modules to the mac, you may find it useful to
install <A
HREF="http://www.iis.ee.ethz.ch/~neeri/macintosh/gusi-qa.html">GUSI</A> for
your copy of MPW. GUSI provides some amount of POSIX compatibility, and is
used by Python itself for this purpose - at the very least having it's
header files available may be useful. Also of note for people porting Unix
modules, the most recent alpha version (4.1a8) of <CODE>MrC</CODE> and
<CODE>MrCpp</CODE> at this writing permits using unix-style pathnames for
includes via the <CODE>-includes unix</CODE> command line option. I have
not experimented heavily with this, but will be doing so in the future and
report my findings.
</P>
<P>
You now have MPW and Python set up to allow compilation of modules.
</P>
<H2><A NAME="compiling">Compiling a Module</A></H2>
<P>
This assumes that you have a C extension module ready to compile. For
instructions on how to write a module, see the Python documentation.
</P>
<P>
There are three approaches you can take to compiling in MPW: using the
command line interface, using the MPW <CODE>CreateMake</CODE> command
(available as the "Create build commands..." menu item, and writing a
Makefile by hand.
</P>
<P>
Before you start any of these, you'll need to know:
</P>
<UL>
<LI>The names and locations of the C source files. In the examples, this
is the file <A HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>, and is in
MPW's current working directory.
<LI>The name that Python expects to import your module under. In the
examples, this is <CODE>xx</CODE>, so the shared library file will be
called <CODE>xx.ppc.slb</CODE>.
<LI>The location of any additional header files use by the C source. The
example does not use any additional header files.
<LI>The location of any additional shared libraries which the module needs
to link to. The example does not link to any other shared libraries.
<LI>The name of the entry point to your module. This is usually the last
function in the main C source file, and the name usually starts with
<CODE>init</CODE>. In the examples, this is <CODE>initxx</CODE>.
</UL>
<H3>Using the Command Line</H3>
<P>
For simple modules consisting of one or two C files, it's often convenient
to simply use commands in a MPW Worksheet. Usually you will want to set
MPW's working directory to the directory containing the C source code. The
following commands compile and link the standard Python test module <A
HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>:
</P>
<PRE>
MrC "xxmodule.c" -o "xx.c.x" -w off -d HAVE_CONFIG_H ∂
-i "{PythonMacIncludes}" ∂
-i "{PythonIncludes}"
PPCLink ∂
-o "xx.ppc.slb" ∂
"xx.c.x" ∂
-t 'shlb' ∂
-c 'Pyth' ∂
-xm s ∂
-d ∂
"{PythonCore}" ∂
"{SharedLibraries}InterfaceLib" ∂
"{SharedLibraries}MathLib" ∂
"{SharedLibraries}StdCLib" ∂
"{PPCLibraries}StdCRuntime.o" ∂
"{PPCLibraries}PPCCRuntime.o" ∂
"{PPCLibraries}PPCToolLibs.o" ∂
-export initxx
</PRE>
<P>
(Note: The last character on each line should appear as "partial
derivative" symbol, which you type as <KBD>option-d</KBD> and which is
MPW's line continuation symbol.)
</P>
<P>
Any additional header files should be specified by adding their directories
as extra <CODE>-i</CODE> options to the <CODE>MrC</CODE> command. Any
additional shared libraries should be added before the PythonCore library
in the <CODE>PPCLink</CODE> command.
</P>
<P>
If there is more than one source file, you will need to duplicate the
compile command for each source file, and you will need to include all the
object files in the place where <CODE>"xx.c.x"</CODE> appears in the
<CODE>PPCLink</CODE> command.
</P>
<H3>Using CreateMake</H3>
<P>
For more complex modules, or modules that you are writing yourself, you
will probably want to use a makefile. Unfortunately MPW's makefiles are
incompatible with the standard Unix makefiles, so you will not be able to
use any makefiles which come with a C module.
</P>
<P>
Usually, you will want the makefile to reside in the same directory as the
C source code, so you should set MPW's working directory to that directory
before proceeding.
</P>
<P>
To create a makefile for the standard Python test module <A
HREF="xxmodule.c"><CODE>xxmodule.c</CODE></A>:
</P>
<UL>
<LI>Select "Create build commands..." from the "Build" Menu.
<LI>Type <KBD>xx.ppc.slb</KBD> for the Program Name.
<LI>Select "Shared Library" for the Program Type.
<LI>Select "PowerPC Only" for the Target.
<LI>Click on the "Source Files..." button, and add your module's C source
files to the list.
<LI>Click on the "Other Options..." button and change the creator type to
"Pyth". If you are using additional header files, you can also add their
directories at this stage. Click on "Continue" once you have done this.
<LI>Click on the "Exported Symbols..." button and type <KBD>initxx</KBD>
into the entry field. Click on "Continue" once you have done this.
<LI>At this stage, your CreateMake window should look like this: <IMG
SRC="html.icons/createmake.png" ALT="[picture of commando window for CreateMake]">
<LI>Click on the "CreateMake" button.
</UL>
<P>
You will now need to edit the makefile that was just created. Open the
file "xx.ppc.slb.make" in the current directory and make the following
changes:
</P>
<UL>
<LI>Change the line
<PRE>
Includes =
</PRE>
<P>
to read
</P>
<PRE>
Includes = -i "{PythonIncludes}" -i "{PythonMacIncludes}"
</PRE>
<P>
If you have any additional headers than need to be included, you can add
them here as well.
<LI>Change the line
<PRE>
PPCCOptions = {Includes} {Sym•PPC}
</PRE>
<P>
to read
</P>
<PRE>
PPCCOptions = -w off -d HAVE_CONFIG_H {Includes} {Sym•PPC}
</PRE>
<P>
<LI>After the line
<PRE>
-xm s ∂
</PRE>
<P>
add
</P>
<PRE>
-d ∂
"{PythonCore}" ∂
</PRE>
<P>
If you have any other shared libraries you need to link to, add each on a
line before PythonCore, terminating each line with a <CODE>∂</CODE>.
</P>
</UL>
<P>Save the file. You are now ready to build.
</P>
<P>
Go to the "Build" or "Full Build" menu items, type in
<KBD>xx.ppc.slb</KBD>, and MPW should take things from there. Any time you
need to rebuild the shared library, you can simply do another "Build" or
"Full Build".
</P>
<H3>Writing a Makefile by Hand</H3>
<P>
For modules which have complex interdependencies between files, you will
likely need a more sophisticated makefile than the one created by
<CODE>CreateMake</CODE>. You will need to be familiar with the MPW
makefile format, but you can get a start by either using
<CODE>CreateMake</CODE> to get a simple starting point, or taking another
MPW makefile as a starting point.
</P>
<P>
It is beyond the scope of this HOWTO to go into the generalities of MPW
makefiles. Documentation on MPW's <CODE>Make</CODE> command can be found
with the MPW distribution, in particular the documents <A
HREF="http://developer.apple.com/tools/mpw-tools/books.html#Building">Building
and Maintaining Programs with MPW (2nd Edition)</A> and the <A
HREF="http://developer.apple.com/tools/mpw-tools/books.html#CommandRef">MPW
Command Reference</A>.
</P>
<P>
There are a couple of important points to keep in mind when writing a
makefile by hand:</P>
<UL>
<LI>When there are multiple symbols with the same name in object files or
shared libraries, <CODE>PPCLink</CODE> used the symbol from the file which
appears first in arguments of the <CODE>PPCLink</CODE> command. For this
reason, you will usually want the PythonCore and any other shared libraries
which are not part of the standard MPW runtime environment to appear before
the standard runtime libraries. This is particularly the case with
StdCLib. The "-d" option turns off the (often copious) warnings about
multiply defined symbols.
<LI>You will want to make sure that the <CODE>HAVE_CONFIG_H</CODE>
preprocessor symbol is defined for most C source files using the <CODE>-d
HAVE_CONFIG_H</CODE> option to <CODE>MrC</CODE>.
</UL>
<P>
The file <A HREF="xx.ppc.slb.make.sit.hqx"><CODE>xx.ppc.slb.make</CODE></A>
is included here for you to use as a starting point.
</P>
<H2><A NAME="using">Using the Extension Module</A></H2>
<P>
Once you have compiled your extension module, you will need to let Python
know where it is. You can either move it into a place on Python's search
path - such as the <CODE>:Mac:Plugins</CODE> folder - or modify the path to
include the location of your new module using the
<CODE>EditPythonPrefs</CODE> applet.
</P>
<P>
Your work may not be completely done, as many extension modules have a
Python wrapper around them. If the Python was not written with portability
in mind, you may need to do some more work to get that up and running.
Indeed, if the Python part uses OS-specific features, like pipes, you may
have to completely rewrite it if you can make it work at all.
</P>
<H2><A NAME="problems">Common Problems</A></H2>
<P>
There are a couple of common problems which occur when porting a module
from another platform. Fortunately, they are often easy to fix.
</P>
<H3>Static Forward Definitions</H3>
<P>
If you get a compiler error which looks something like:
</P>
<PRE>
File "xxmodule.c"; line 135 #Error: 'Xxo_Type' is already defined
</PRE>
<P>
then most likely either you have not set up <CODE>config.h</CODE> correctly
to handle static forward definitions, or the module author has not adhered
to the standard python conventions. If the second is the case, find where
the variable is first defined, and replace the <CODE>static</CODE> with
<CODE>staticforward</CODE>. Then find the second place it is defined
(usually the line where the compiler complained) and replace
<CODE>static</CODE> with <CODE>statichere</CODE>.
</P>
<P>
If you have set up things correctly, you should now be able to compile.
</P>
<H3>Automatic Type Conversion</H3>
<P>
<CODE>MrC</CODE> seems to be a little pickier about automatically
converting from one type to another than some other C compilers. These can
often be fixed by simply adding an explicit cast to the desired type.
</P>
<P>
XXX There may be a compiler option which relaxes this. That would be a
better solution.
</P>
<H2><A NAME="caveats">Caveats</A></H2>
<P>
As Jack Jansen pointed out on the Mac Python mailing list, there could
potentially be conflicts between the MetroWerks C runtime which the Python
core and standard modules was compiled with, and the MPW C runtime which
your extension module is compiled with. While things seem to work fine in
everyday use, it is possible that there are bugs which have not been
discovered yet. Most likely these world take the form of standard C
functions (most likely I/O functions due to conflicts between the SIOUX
libraries and the SIOW libraries) not working as they are supposed to, or
memory leaks caused by improper malloc/free.
</P>
<P>
Some such problems have been demonstrated by compiling modules with
PythonCore linked after StdCLib - printf does not work properly in this
setup, and I suspect that there will also be malloc/free problems in
situations where the module allocates memory which is later disposed of by
Python, or vice-versa. Compiling with PythonCore taking precedence over
StdCLib seems to give the correct behaviour.
</P>
<P>
This method of compiling should be considered experimental for the time
being. <STRONG>Use it at your own risk.</STRONG>
</P>
<P>
If you notice any quirks in modules compiled this way, or have insight into
what may go wrong or right with this situation, <A
HREF="mailto:cwebster@nevada.edu">please contact me</A> so that I can add
it to the HOWTO.
</P>
<P>
The ideal solution to this problem would be to get Python to compile using
MPW (and a Python MPW Tool would be very neat indeed). However, that does
seem to be a major project.
</P>
<DIV class=footer>
<HR>
<BR>
©<A HREF="mailto:cwebster@nevada.edu">Corran Webster</A>, 1999. <BR>
<!-- #LASTMODIFIED TEXT="Last modified" FORM="SHORT,TIME" -->
Last modified 14/12/99 12:17 PM
<!-- /#LASTMODIFIED -->
</DIV>
</BODY>
</HTML>
