1.1 一个简单的例子¶

我们来创建一个名为spam的扩展模块（spam，monty python粉丝们最喜欢的食物），我们想给C库函数system()创建一个Python的接口。[1] 这个函数以一个空字符结尾的字符串为参数，返回一个整数。我们希望在Python中这样调用这个函数：

>>> import spam
>>> status = spam.system("ls -l")

首先创建名为spammodule.c的文件。（按历史惯例，如果模块叫做spam，那么包含实现的C文件就叫做spammodule.c；如果模块的名字很长，比如spammify，那么模块的名字可以是spammify.c。）

文件的第一行为：

#include <Python.h>

这会引入Python API的声明（如果乐意，你可以加上关于模块的说明和版权信息的注释）。

除了在标准头文件中定义的符号外，Python.h中定义的用户可见的符号以Py或者PY打头。

接下来我们加入C实现函数，当Python表达式spam.system(string)被演算时，该函数会被调用（不久我们会看到它是如何被调用的）：

static PyObject *
spam_system(PyObject *self, PyObject *args)
{
    const char *command;
    int sts;

    if (!PyArg_ParseTuple(args, "s", &command))
        return NULL;
    sts = system(command);
    return Py_BuildValue("i", sts);
}

可以很清楚地看出Python的参数列表（如"ls -l"）是如何转化成C函数的参数的。

对于模块级函数，self参数指向模块对象；如果是方法，self指向对象实例。

args参数是一个指向包含参数的Python元组对象的指针。元组中的每一个元素对应调用参数列表中的一个参数。这些参数是Python对象——为了能在C函数中处理这些Python对象，我们需要将其转成C可操作的值。Python API中的PyArg_ParseTuple()函数检查参数类型，并将其转成对应的C值。它使用模板字符串来决定所需要的参数类型，同时也决定用来存转换后的值的C变量的类型。后面有更多的说明。

如果所有的参数有正确的类型，且值被存入变量（通过传地址）PyArg_ParseTuple()返回真（非0值）。如果传递了不正确的参数，它返回假（0值）。对于后一种情况，它还会抛出异常，所以函数立即返回NULL（如同例子所示）。

1.2. 插曲：错误和异常¶

贯穿Python解释器的一个重要惯例如下：如果一个函数失败，它应该设置一个异常条件并返回一个错误值（一般来说是个NULL指针）。异常存储于解释器中的一个静态全局变量；如果这个变量为NULL表示没有异常发生。第二个全局变量保存异常“所关联的值”（raise的第二个参数）。如果Python代码中发生了错误，第三个变量包含了函数调用栈。这三个变量是Python变量sys.exc_type，sys.exc_value和sys.exc_traceback在C中的对等物（参见Python库参考手册中模块sys对应的章节）。了解这三个变量有助于理解错误是如何传递的。

Python API定义了一组函数来设置各种异常。

最常用的一个函数是PyErr_SetString()。它的参数是一个异常对象和一个C字符串。异常对象一般是个预定义的对象，比如PyExc_ZeroDivisionError。C字符串表明了错误的原因，它被转成Python字符串对象，并被当成异常“所关联的值”而保存。

另一个有用的函数是PyErr_SetFromErrno()它只有一个异常参数，它通过检查全局变量errno来构造异常值。最通用的函数是PyErr_SetObject()它有两个对象参数，异常和它的值。对于传递给这些函数的对象，你不需要调用Py_INCREF()（增加引用计数）。

你可以通过调用PyErr_Occurred()来非破坏性地检查是否有无异常。它返回当前的异常对象，或者如果没有异常发生，它返回NULL。一般来说你不需要调用PyErr_Occurred()来检查在函数调用中是否有错误发生，因为可以从函数的返回值中得知。

当函数f调用另一个函数g发现g失败了，f本身需要返回一个错误值（一般是NULL或者-1）。它不应该调用PyErr_*()函数——因为g已经调用过了。f的调用者对它的调用者也应该返回一个错误值，同样不应该调用PyErr_*()，依此类推——错误详细的原因由第一个发现错误的函数来报告。一旦错误到达了Python解释器的主循环，这会中止当前Python代码的执行，然后尝试寻找由Python程序员指定的异常处理器。

（模块可以通过调用PyErr_*()函数来给出更详细的错误消息，这在某些情况下是可行的。尽管如此，作为一个通用的规则，这是不必要的，它有可能导致错误原因信息的丢失：大多数操作失败的原因千奇百怪。）

要忽略函数调用失败所设置的异常，可以通过调用PyErr_Clear()来清除异常。唯一需要调用PyErr_Clear()的情况是不希望向解释器传递错误，而是想完全由C代码自己来处理异常（可能是换种尝试或者假装什么事都没有发生）。

每一个失败的malloc()调用必须转成一个异常——malloc()（或者realloc()）的直接调用者必须调用PyErr_NoMemory()并且返回一个错误值。所有的对象创建函数（例如PyInt_FromLong()）已经做过这件事了，所以这个只对直接调用malloc()的场合有效。

请注意PyArg_ParseTuple()系列函数的例外，这些函数返回整数值来表示状态，一般正整数或者零表示成功，-1表示失败，和Unix系统调用类似。

最后当返回错误值的时候要小心垃圾回收（对已经创建的对象调用Py_XDECREF()或者Py_DECREF()）！

抛出何种异常的选择权在于你。你可以直接使用预先声明的C对象，它们对应于内置的Python异常，如PyExc_ZeroDivisionError。当然，你应该明智地选择异常——不要使用PyExc_TypeError来表示文件不能打开（这种情况应该使用PyExc_IOError）。PyArg_ParseTuple()函数一般抛出PyExc_TypeError，如果它的参数列表有问题的话。如果你的参数必须在一个特定的范围或者必须满足特定的条件，可以使用PyExc_ValueError。

你也可以定义特定于你模块的新异常。你需要在你文件的开头声明一个静态对象变量：

static PyObject *SpamError;

用一个异常对象在你模块的初始化函数（initspam()）中初始化该静态对象变量（目前先不考虑错误检查）：

PyMODINIT_FUNC
initspam(void)
{
    PyObject *m;

    m = Py_InitModule("spam", SpamMethods);
    if (m == NULL)
        return;

    SpamError = PyErr_NewException("spam.error", NULL, NULL);
    Py_INCREF(SpamError);
    PyModule_AddObject(m, "error", SpamError);
}

注意异常对象的Python名字是spam.error。PyErr_NewException()创建一个类，它的基类是Exception（除非传递了另一个类而不是NULL）（Exception在Built-in Exceptions中有描述）。

请注意SpamError变量保持有对新建异常类的一个引用；这是有意的！因为可以通过外部代码从模块中移除该异常，拥有对该类的引用是必要的，可以保证它不会被丢弃，从而SpamError不会是个空指针。如果它成了空指针，抛出该异常的C代码将会导致内存转储或者其它意外的副作用。

我们将在这个例子的后面讨论函数返回类型PyMODINIT_FUNC。

可以在扩展模块中通过调用PyErr_SetString()抛出spam.error异常，如下所示：

static PyObject *
spam_system(PyObject *self, PyObject *args)
{
    const char *command;
    int sts;

    if (!PyArg_ParseTuple(args, "s", &command))
        return NULL;
    sts = system(command);
    if (sts < 0) {
        PyErr_SetString(SpamError, "System command failed");
        return NULL;
    }
    return PyLong_FromLong(sts);
}

1.3. 回到例子¶

回到例子函数，现在你应该能够理解这个语句了：

if (!PyArg_ParseTuple(args, "s", &command))
    return NULL;

如果参数列表有错误，依赖于PyArg_ParseTuple()设置的异常，它返回NULL（函数返回对象指针时的错误值）。否则，参数字符串的值被拷贝到局部变量command。这是指针赋值，你不应该修改它所指向的字符串（所以在标准C中，变量command应该被声明为const char *command）。

接下来的语句调用Unix的函数system()，从PyArg_ParseTuple()中得到的字符串作为参数。

sts = system(command);

spam.system()函数必须以Python对象的方式返回值sts。这是通过调用Py_BuildValue()来完成的，Py_BuildValue()是PyArg_ParseTuple()的反函数：它以格式化字符串和C值为参数，返回一个Python对象。后面有更多关于Py_BuildValue()的信息。

return Py_BuildValue("i", sts);

这里，它返回一个整数对象。（在Python中，整数也是在堆上的对象！）

如果C函数不需要返回值（函数返回void），对应的Python函数必须返回None。你可以这样做（也可以调用Py_RETURN_NONE宏）：

Py_INCREF(Py_None);
return Py_None;

Py_None是Python对象None在C中的名字。它是一个真正的Python对象，而不是NULL指针（空指针在大多数上下文表示“错误”）。

1.4. 模块的方法表和初始化函数¶

我答应过演示如何在Python程序中调用spam_system()。首先，我们需要在“方法表”中加入它的名字和地址：

static PyMethodDef SpamMethods[] = {
    ...
    {"system",  spam_system, METH_VARARGS,
     "Execute a shell command."},
    ...
    {NULL, NULL, 0, NULL}        /* Sentinel */
};

注意第三个条目（METH_VARARGS）。这是一个标志位，它告诉解释器应该如何调用该C函数（调用惯例）。它一般是METH_VARARGS或者METH_VARARGS | METH_KEYWORDS；如果值为0表示使用了PyArg_ParseTuple()的变体（被废弃了）。

如果只有METH_VARARGS，在Python中参数应该以元组方式传递，该元组可由PyArg_ParseTuple()来解析；下面有该函数的详细信息。

如果函数接受关键字参数，那么第三个字段可以置METH_KEYWORDS位。这时，C函数应有第三个参数PyObject *，它是一个关键字字典。可以用PyArg_ParseTupleAndKeywords()来解析这些参数。

方法表在模块的初始化函数中被传递给解释器。初始化函数的名字必须是initname()，name是模块名；初始化函数是模块文件中唯一的non-static（非静态的）。

PyMODINIT_FUNC
initspam(void)
{
    (void) Py_InitModule("spam", SpamMethods);
}

注意PyMODINIT_FUNC声明函数的返回类型为void，同时还声明平台所需要的其他链接声明，对于C++，它声明函数为extern "C"。

当Python程序首次导入spam模块的时候，initspam()被调用。当Python程序首次导入 spam 将调用initspam() 。（参见下面关于嵌入Python的注释。）initspam()调用Py_InitModule()，它创建一个“模块对象”（该模块对象被插入字典sys.modules，对应的键为"spam"），根据第二个参数——方法表（PyMethodDef结构的数组），将函数对象插入该模块。Py_InitModule()返回它所创建的模块对象的指针（这里没有使用返回值）。某些错误可以使该函数中止，如果模块不能被初始化，它返回NULL。

当嵌入Python时，initspam()不会被自动调用，除非它在_PyImport_Inittab表中有登记。最简单的处理方法就是在调用Py_Initialize()后，直接调用initspam()，静态地初始化你的静态链接模块。

int
main(int argc, char *argv[])
{
    /* Pass argv[0] to the Python interpreter */
    Py_SetProgramName(argv[0]);

    /* Initialize the Python interpreter.  Required. */
    Py_Initialize();

    /* Add a static module */
    initspam();

    ...

可以在Python源码包中Demo/embed/demo.c里找到一个例子。

在Python源码包中Modules/xxmodule.c里有个更实际的例子。该文件可以当成模板，或者只是当成例子来读。

1.5. 编译和链接¶

在你能够使用新扩展之前，你得先将其编译和链接到Python系统。如果你使用动态装载，细节取决于你系统使用的动态装载机制；参见构建扩展模块相关的章节（Building C and C++ Extensions with distutils），Windows特定的构建信息参见（Building C and C++ Extensions on Windows）。

如果你无法使用动态装载，或者你希望你的模块永久的成为解释器的一部分，你需要修改配置设置并重新构建解释器。幸运的是，这在Unix上非常简单：在解压的源码包中的Modules/目录下放置你的文件（例子中的spammodule.c），在Modules/Setup.local 中添加一行来描述你的文件：

spam spammodule.o

然后在顶层目录下运行make来重新构建解释器。你也可以在Modules/子目录下运行make，但是你得先运行“make Makefile”以重新构建Makefile。（每次修改了Setup文件都要这样做。）

如果模块需要链接额外的库，它们也可以出现在配置文件中，例如：

spam spammodule.o -lX11

1.6. Calling Python Functions from C¶

到目前为止，我们已经重点讨论了如何从Python调用C的函数。反之，从C调用Python的函数也非常有用。这个功能特别是在创建支持“回调函数”的库时候会很便利。 当一个C接口使用回调，提供同样功能的Python代码经常需要为Python程序员提供一个回调机制； 这个执行将从一个C的回调请求调用Python的回调函数。当然，也有很多其它的用途。

幸运的是，Python解释器可以简单的调用递归，并且有一个标准的调用接口去调用Python函数。 （我不打算详细说明怎样用特定的字符串作为输入去呼叫Python的语法分析器 – 如果你有兴趣，可以从Python的源代码查看-c命令行的Modules/main.c选项）

调用Python函数很容易。 Python程序一定会以某种方式传递给你Python的函数对象。 为此，你应该为其提供一个函数（或其它某种接口）。当这个函数被调用，一个指向Python函数对象（小心对其使用Py_INCREF()！）的指针被保存在一个全局变量 – 或其它任何你认为合适的地方。 比如：以下的函数可能成为模块定义的一部分

static PyObject *my_callback = NULL;

static PyObject *
my_set_callback(PyObject *dummy, PyObject *args)
{
    PyObject *result = NULL;
    PyObject *temp;

    if (PyArg_ParseTuple(args, "O:set_callback", &temp)) {
        if (!PyCallable_Check(temp)) {
            PyErr_SetString(PyExc_TypeError, "parameter must be callable");
            return NULL;
        }
        Py_XINCREF(temp);         /* Add a reference to new callback */
        Py_XDECREF(my_callback);  /* Dispose of previous callback */
        my_callback = temp;       /* Remember new callback */
        /* Boilerplate to return "None" */
        Py_INCREF(Py_None);
        result = Py_None;
    }
    return result;
}

This function must be registered with the interpreter using the METH_VARARGS flag; this is described in section The Module’s Method Table and Initialization Function. The PyArg_ParseTuple() function and its arguments are documented in section Extracting Parameters in Extension Functions.

The macros Py_XINCREF() and Py_XDECREF() increment/decrement the reference count of an object and are safe in the presence of NULL pointers (but note that temp will not be NULL in this context). More info on them in section Reference Counts.

Later, when it is time to call the function, you call the C function PyObject_CallObject(). This function has two arguments, both pointers to arbitrary Python objects: the Python function, and the argument list. The argument list must always be a tuple object, whose length is the number of arguments. To call the Python function with no arguments, pass in NULL, or an empty tuple; to call it with one argument, pass a singleton tuple. Py_BuildValue() returns a tuple when its format string consists of zero or more format codes between parentheses. For example:

int arg;
PyObject *arglist;
PyObject *result;
...
arg = 123;
...
/* Time to call the callback */
arglist = Py_BuildValue("(i)", arg);
result = PyObject_CallObject(my_callback, arglist);
Py_DECREF(arglist);

PyObject_CallObject() returns a Python object pointer: this is the return value of the Python function. PyObject_CallObject() is “reference-count-neutral” with respect to its arguments. In the example a new tuple was created to serve as the argument list, which is Py_DECREF()-ed immediately after the PyObject_CallObject() call.

The return value of PyObject_CallObject() is “new”: either it is a brand new object, or it is an existing object whose reference count has been incremented. So, unless you want to save it in a global variable, you should somehow Py_DECREF() the result, even (especially!) if you are not interested in its value.

Before you do this, however, it is important to check that the return value isn’t NULL. If it is, the Python function terminated by raising an exception. If the C code that called PyObject_CallObject() is called from Python, it should now return an error indication to its Python caller, so the interpreter can print a stack trace, or the calling Python code can handle the exception. If this is not possible or desirable, the exception should be cleared by calling PyErr_Clear(). For example:

if (result == NULL)
    return NULL; /* Pass error back */
...use result...
Py_DECREF(result);

Depending on the desired interface to the Python callback function, you may also have to provide an argument list to PyObject_CallObject(). In some cases the argument list is also provided by the Python program, through the same interface that specified the callback function. It can then be saved and used in the same manner as the function object. In other cases, you may have to construct a new tuple to pass as the argument list. The simplest way to do this is to call Py_BuildValue(). For example, if you want to pass an integral event code, you might use the following code:

PyObject *arglist;
...
arglist = Py_BuildValue("(l)", eventcode);
result = PyObject_CallObject(my_callback, arglist);
Py_DECREF(arglist);
if (result == NULL)
    return NULL; /* Pass error back */
/* Here maybe use the result */
Py_DECREF(result);

Note the placement of Py_DECREF(arglist) immediately after the call, before the error check! Also note that strictly speaking this code is not complete: Py_BuildValue() may run out of memory, and this should be checked.

You may also call a function with keyword arguments by using PyObject_Call(), which supports arguments and keyword arguments. As in the above example, we use Py_BuildValue() to construct the dictionary.

PyObject *dict;
...
dict = Py_BuildValue("{s:i}", "name", val);
result = PyObject_Call(my_callback, NULL, dict);
Py_DECREF(dict);
if (result == NULL)
    return NULL; /* Pass error back */
/* Here maybe use the result */
Py_DECREF(result);

1.7. Extracting Parameters in Extension Functions¶

函数 PyArg_ParseTuple() 定义如下：

int PyArg_ParseTuple(PyObject *arg, char *format, ...);

The arg argument must be a tuple object containing an argument list passed from Python to a C function. The format argument must be a format string, whose syntax is explained in Parsing arguments and building values in the Python/C API Reference Manual. The remaining arguments must be addresses of variables whose type is determined by the format string.

Note that while PyArg_ParseTuple() checks that the Python arguments have the required types, it cannot check the validity of the addresses of C variables passed to the call: if you make mistakes there, your code will probably crash or at least overwrite random bits in memory. So be careful!

Note that any Python object references which are provided to the caller are borrowed references; do not decrement their reference count!

举例如下:

int ok;
int i, j;
long k, l;
const char *s;
int size;

ok = PyArg_ParseTuple(args, ""); /* No arguments */
    /* Python call: f() */

ok = PyArg_ParseTuple(args, "s", &s); /* A string */
    /* Possible Python call: f('whoops!') */

ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */
    /* Possible Python call: f(1, 2, 'three') */

ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size);
    /* A pair of ints and a string, whose size is also returned */
    /* Possible Python call: f((1, 2), 'three') */

{
    const char *file;
    const char *mode = "r";
    int bufsize = 0;
    ok = PyArg_ParseTuple(args, "s|si", &file, &mode, &bufsize);
    /* A string, and optionally another string and an integer */
    /* Possible Python calls:
       f('spam')
       f('spam', 'w')
       f('spam', 'wb', 100000) */
}

{
    int left, top, right, bottom, h, v;
    ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)",
             &left, &top, &right, &bottom, &h, &v);
    /* A rectangle and a point */
    /* Possible Python call:
       f(((0, 0), (400, 300)), (10, 10)) */
}

{
    Py_complex c;
    ok = PyArg_ParseTuple(args, "D:myfunction", &c);
    /* a complex, also providing a function name for errors */
    /* Possible Python call: myfunction(1+2j) */
}

1.8. 扩展函数的关键字参数¶

The PyArg_ParseTupleAndKeywords() function is declared as follows:

int PyArg_ParseTupleAndKeywords(PyObject *arg, PyObject *kwdict,
                                char *format, char *kwlist[], ...);

The arg and format parameters are identical to those of the PyArg_ParseTuple() function. The kwdict parameter is the dictionary of keywords received as the third parameter from the Python runtime. The kwlist parameter is a NULL-terminated list of strings which identify the parameters; the names are matched with the type information from format from left to right. On success, PyArg_ParseTupleAndKeywords() returns true, otherwise it returns false and raises an appropriate exception.

Here is an example module which uses keywords, based on an example by Geoff Philbrick (philbrick@hks.com):

#include "Python.h"

static PyObject *
keywdarg_parrot(PyObject *self, PyObject *args, PyObject *keywds)
{
    int voltage;
    char *state = "a stiff";
    char *action = "voom";
    char *type = "Norwegian Blue";

    static char *kwlist[] = {"voltage", "state", "action", "type", NULL};

    if (!PyArg_ParseTupleAndKeywords(args, keywds, "i|sss", kwlist,
                                     &voltage, &state, &action, &type))
        return NULL;

    printf("-- This parrot wouldn't %s if you put %i Volts through it.\n",
           action, voltage);
    printf("-- Lovely plumage, the %s -- It's %s!\n", type, state);

    Py_INCREF(Py_None);

    return Py_None;
}

static PyMethodDef keywdarg_methods[] = {
    /* The cast of the function is necessary since PyCFunction values
     * only take two PyObject* parameters, and keywdarg_parrot() takes
     * three.
     */
    {"parrot", (PyCFunction)keywdarg_parrot, METH_VARARGS | METH_KEYWORDS,
     "Print a lovely skit to standard output."},
    {NULL, NULL, 0, NULL}   /* sentinel */
};

void
initkeywdarg(void)
{
  /* Create the module and add the functions */
  Py_InitModule("keywdarg", keywdarg_methods);
}

1.9. Building Arbitrary Values¶

This function is the counterpart to PyArg_ParseTuple(). It is declared as follows:

PyObject *Py_BuildValue(char *format, ...);

It recognizes a set of format units similar to the ones recognized by PyArg_ParseTuple(), but the arguments (which are input to the function, not output) must not be pointers, just values. It returns a new Python object, suitable for returning from a C function called from Python.

One difference with PyArg_ParseTuple(): while the latter requires its first argument to be a tuple (since Python argument lists are always represented as tuples internally), Py_BuildValue() does not always build a tuple. It builds a tuple only if its format string contains two or more format units. If the format string is empty, it returns None; if it contains exactly one format unit, it returns whatever object is described by that format unit. To force it to return a tuple of size 0 or one, parenthesize the format string.

Examples (to the left the call, to the right the resulting Python value):

Py_BuildValue("")                        None
Py_BuildValue("i", 123)                  123
Py_BuildValue("iii", 123, 456, 789)      (123, 456, 789)
Py_BuildValue("s", "hello")              'hello'
Py_BuildValue("ss", "hello", "world")    ('hello', 'world')
Py_BuildValue("s#", "hello", 4)          'hell'
Py_BuildValue("()")                      ()
Py_BuildValue("(i)", 123)                (123,)
Py_BuildValue("(ii)", 123, 456)          (123, 456)
Py_BuildValue("(i,i)", 123, 456)         (123, 456)
Py_BuildValue("[i,i]", 123, 456)         [123, 456]
Py_BuildValue("{s:i,s:i}",
              "abc", 123, "def", 456)    {'abc': 123, 'def': 456}
Py_BuildValue("((ii)(ii)) (ii)",
              1, 2, 3, 4, 5, 6)          (((1, 2), (3, 4)), (5, 6))

1.10. Reference Counts¶

In languages like C or C++, the programmer is responsible for dynamic allocation and deallocation of memory on the heap. In C, this is done using the functions malloc() and free(). In C++, the operators new and delete are used with essentially the same meaning and we’ll restrict the following discussion to the C case.

Every block of memory allocated with malloc() should eventually be returned to the pool of available memory by exactly one call to free(). It is important to call free() at the right time. If a block’s address is forgotten but free() is not called for it, the memory it occupies cannot be reused until the program terminates. This is called a memory leak. On the other hand, if a program calls free() for a block and then continues to use the block, it creates a conflict with re-use of the block through another malloc() call. This is called using freed memory. It has the same bad consequences as referencing uninitialized data — core dumps, wrong results, mysterious crashes.

Common causes of memory leaks are unusual paths through the code. For instance, a function may allocate a block of memory, do some calculation, and then free the block again. Now a change in the requirements for the function may add a test to the calculation that detects an error condition and can return prematurely from the function. It’s easy to forget to free the allocated memory block when taking this premature exit, especially when it is added later to the code. Such leaks, once introduced, often go undetected for a long time: the error exit is taken only in a small fraction of all calls, and most modern machines have plenty of virtual memory, so the leak only becomes apparent in a long-running process that uses the leaking function frequently. Therefore, it’s important to prevent leaks from happening by having a coding convention or strategy that minimizes this kind of errors.

Since Python makes heavy use of malloc() and free(), it needs a strategy to avoid memory leaks as well as the use of freed memory. The chosen method is called reference counting. The principle is simple: every object contains a counter, which is incremented when a reference to the object is stored somewhere, and which is decremented when a reference to it is deleted. When the counter reaches zero, the last reference to the object has been deleted and the object is freed.

An alternative strategy is called automatic garbage collection. (Sometimes, reference counting is also referred to as a garbage collection strategy, hence my use of “automatic” to distinguish the two.) The big advantage of automatic garbage collection is that the user doesn’t need to call free() explicitly. (Another claimed advantage is an improvement in speed or memory usage — this is no hard fact however.) The disadvantage is that for C, there is no truly portable automatic garbage collector, while reference counting can be implemented portably (as long as the functions malloc() and free() are available — which the C Standard guarantees). Maybe some day a sufficiently portable automatic garbage collector will be available for C. Until then, we’ll have to live with reference counts.

While Python uses the traditional reference counting implementation, it also offers a cycle detector that works to detect reference cycles. This allows applications to not worry about creating direct or indirect circular references; these are the weakness of garbage collection implemented using only reference counting. Reference cycles consist of objects which contain (possibly indirect) references to themselves, so that each object in the cycle has a reference count which is non-zero. Typical reference counting implementations are not able to reclaim the memory belonging to any objects in a reference cycle, or referenced from the objects in the cycle, even though there are no further references to the cycle itself.

The cycle detector is able to detect garbage cycles and can reclaim them so long as there are no finalizers implemented in Python (__del__() methods). When there are such finalizers, the detector exposes the cycles through the gc module (specifically, the garbage variable in that module). The gc module also exposes a way to run the detector (the collect() function), as well as configuration interfaces and the ability to disable the detector at runtime. The cycle detector is considered an optional component; though it is included by default, it can be disabled at build time using the –without-cycle-gc option to the configure script on Unix platforms (including Mac OS X) or by removing the definition of WITH_CYCLE_GC in the pyconfig.h header on other platforms. If the cycle detector is disabled in this way, the gc module will not be available.

void
bug(PyObject *list)
{
    PyObject *item = PyList_GetItem(list, 0);

    PyList_SetItem(list, 1, PyInt_FromLong(0L));
    PyObject_Print(item, stdout, 0); /* BUG! */
}

void
no_bug(PyObject *list)
{
    PyObject *item = PyList_GetItem(list, 0);

    Py_INCREF(item);
    PyList_SetItem(list, 1, PyInt_FromLong(0L));
    PyObject_Print(item, stdout, 0);
    Py_DECREF(item);
}

void
bug(PyObject *list)
{
    PyObject *item = PyList_GetItem(list, 0);
    Py_BEGIN_ALLOW_THREADS
    ...some blocking I/O call...
    Py_END_ALLOW_THREADS
    PyObject_Print(item, stdout, 0); /* BUG! */
}

1.11. Writing Extensions in C++¶

It is possible to write extension modules in C++.Some restrictions apply.If the main program (the Python interpreter) is compiled and linked by the C compiler, global or static objects with constructors cannot be used.This is not a problem if the main program is linked by the C++ compiler.Functions that will be called by the Python interpreter (in particular, module initialization functions) have to be declared using extern "C".It is unnecessary to enclose the Python header files in extern "C" {...} — they use this form already if the symbol __cplusplus is defined (all recent C++ compilers define this symbol).

1.12. Providing a C API for an Extension Module¶

Many extension modules just provide new functions and types to be used from Python, but sometimes the code in an extension module can be useful for other extension modules. For example, an extension module could implement a type “collection” which works like lists without order. Just like the standard Python list type has a C API which permits extension modules to create and manipulate lists, this new collection type should have a set of C functions for direct manipulation from other extension modules.

At first sight this seems easy: just write the functions (without declaring them static, of course), provide an appropriate header file, and document the C API. And in fact this would work if all extension modules were always linked statically with the Python interpreter. When modules are used as shared libraries, however, the symbols defined in one module may not be visible to another module. The details of visibility depend on the operating system; some systems use one global namespace for the Python interpreter and all extension modules (Windows, for example), whereas others require an explicit list of imported symbols at module link time (AIX is one example), or offer a choice of different strategies (most Unices). And even if symbols are globally visible, the module whose functions one wishes to call might not have been loaded yet!

Portability therefore requires not to make any assumptions about symbol visibility. This means that all symbols in extension modules should be declared static, except for the module’s initialization function, in order to avoid name clashes with other extension modules (as discussed in section The Module’s Method Table and Initialization Function). And it means that symbols that should be accessible from other extension modules must be exported in a different way.

Python provides a special mechanism to pass C-level information (pointers) from one extension module to another one: Capsules. A Capsule is a Python data type which stores a pointer (void *). Capsules can only be created and accessed via their C API, but they can be passed around like any other Python object. In particular, they can be assigned to a name in an extension module’s namespace. Other extension modules can then import this module, retrieve the value of this name, and then retrieve the pointer from the Capsule.

There are many ways in which Capsules can be used to export the C API of an extension module. Each function could get its own Capsule, or all C API pointers could be stored in an array whose address is published in a Capsule. And the various tasks of storing and retrieving the pointers can be distributed in different ways between the module providing the code and the client modules.

Whichever method you choose, it’s important to name your Capsules properly. The function PyCapsule_New() takes a name parameter (const char *); you’re permitted to pass in a NULL name, but we strongly encourage you to specify a name. Properly named Capsules provide a degree of runtime type-safety; there is no feasible way to tell one unnamed Capsule from another.

In particular, Capsules used to expose C APIs should be given a name following this convention:

modulename.attributename

The convenience function PyCapsule_Import() makes it easy to load a C API provided via a Capsule, but only if the Capsule’s name matches this convention. This behavior gives C API users a high degree of certainty that the Capsule they load contains the correct C API.

The following example demonstrates an approach that puts most of the burden on the writer of the exporting module, which is appropriate for commonly used library modules. It stores all C API pointers (just one in the example!) in an array of void pointers which becomes the value of a Capsule. The header file corresponding to the module provides a macro that takes care of importing the module and retrieving its C API pointers; client modules only have to call this macro before accessing the C API.

The exporting module is a modification of the spam module from section A Simple Example. The function spam.system() does not call the C library function system() directly, but a function PySpam_System(), which would of course do something more complicated in reality (such as adding “spam” to every command). This function PySpam_System() is also exported to other extension modules.

The function PySpam_System() is a plain C function, declared static like everything else:

static int
PySpam_System(const char *command)
{
    return system(command);
}

The function spam_system() is modified in a trivial way:

static PyObject *
spam_system(PyObject *self, PyObject *args)
{
    const char *command;
    int sts;

    if (!PyArg_ParseTuple(args, "s", &command))
        return NULL;
    sts = PySpam_System(command);
    return Py_BuildValue("i", sts);
}

In the beginning of the module, right after the line

#include "Python.h"

two more lines must be added:

#define SPAM_MODULE
#include "spammodule.h"

The #define is used to tell the header file that it is being included in the exporting module, not a client module. Finally, the module’s initialization function must take care of initializing the C API pointer array:

PyMODINIT_FUNC
initspam(void)
{
    PyObject *m;
    static void *PySpam_API[PySpam_API_pointers];
    PyObject *c_api_object;

    m = Py_InitModule("spam", SpamMethods);
    if (m == NULL)
        return;

    /* Initialize the C API pointer array */
    PySpam_API[PySpam_System_NUM] = (void *)PySpam_System;

    /* Create a Capsule containing the API pointer array's address */
    c_api_object = PyCapsule_New((void *)PySpam_API, "spam._C_API", NULL);

    if (c_api_object != NULL)
        PyModule_AddObject(m, "_C_API", c_api_object);
}

Note that PySpam_API is declared static; otherwise the pointer array would disappear when initspam() terminates!

The bulk of the work is in the header file spammodule.h, which looks like this:

#ifndef Py_SPAMMODULE_H
#define Py_SPAMMODULE_H
#ifdef __cplusplus
extern "C" {
#endif

/* Header file for spammodule */

/* C API functions */
#define PySpam_System_NUM 0
#define PySpam_System_RETURN int
#define PySpam_System_PROTO (const char *command)

/* Total number of C API pointers */
#define PySpam_API_pointers 1


#ifdef SPAM_MODULE
/* This section is used when compiling spammodule.c */

static PySpam_System_RETURN PySpam_System PySpam_System_PROTO;

#else
/* This section is used in modules that use spammodule's API */

static void **PySpam_API;

#define PySpam_System \
 (*(PySpam_System_RETURN (*)PySpam_System_PROTO) PySpam_API[PySpam_System_NUM])

/* Return -1 on error, 0 on success.
 * PyCapsule_Import will set an exception if there's an error.
 */
static int
import_spam(void)
{
    PySpam_API = (void **)PyCapsule_Import("spam._C_API", 0);
    return (PySpam_API != NULL) ? 0 : -1;
}

#endif

#ifdef __cplusplus
}
#endif

#endif /* !defined(Py_SPAMMODULE_H) */

All that a client module must do in order to have access to the function PySpam_System() is to call the function (or rather macro) import_spam() in its initialization function:

PyMODINIT_FUNC
initclient(void)
{
    PyObject *m;

    m = Py_InitModule("client", ClientMethods);
    if (m == NULL)
        return;
    if (import_spam() < 0)
        return;
    /* additional initialization can happen here */
}

The main disadvantage of this approach is that the file spammodule.h is rather complicated. However, the basic structure is the same for each function that is exported, so it has to be learned only once.

Finally it should be mentioned that Capsules offer additional functionality, which is especially useful for memory allocation and deallocation of the pointer stored in a Capsule. The details are described in the Python/C API Reference Manual in the section Capsules and in the implementation of Capsules (files Include/pycapsule.h and Objects/pycapsule.c in the Python source code distribution).

脚注