-
Notifications
You must be signed in to change notification settings - Fork 17
Adding builtin types
TODO structure, finish and generally tidy up
If you wish to add your own type to the possible types, there are many things you need to do
First you need to define the macro, it should be of the form pack_<Name> where <Name> is what you want to refer to it as in the macro's. The macro should resolve to the classpath to the class you want to add. Here is an example for AbstractDict:
#define pack_pyAbstractDict "org/python/core/AbstractDict"
The inner class (KeysIter) can be accessed like this:
#define pack_KeysIter "org/python/core/AbstractDict$KeysIter"
If you don't want to be able to convert the java object to a PyObject in C, you may be done here, but I'm not sure. The rest of this guide may also be required to allow garbage collection to work properly. If you want it to be possible to convert it to a PyObject, then you definitely need to follow the rest of the guide.
An index must be assigned in JyNI.h, around line 880 there is a list of indexes, these should take the form TME_INDEX_<Name>. the <Name> doesn't need to be the same as in previous parts, but it is helpful to make it clear what the index refers to. Here are example macros for dictionaries and dictionary keys iter:
#define TME_INDEX_Dict 30
#define TME_INDEX_KeysIter 31
When you add macro's, they need to have unique indexes and TME_INDEX_BaseObject must be left as the last index in the list. You must also increment the builtinTypeCount macro which can be found at the top of the list.
TODO at this point my knowledge is severely lacking, the rest of this guide is dubious at best.
Now you must add to the builtinTypes array, this can be found in JyNI.c in the initBuiltinTypes() function around line 700. There is a comment there which explains a little about what this list is, but basically each type that can be converted between a Jython PyObject and the C API PyObject has an entry, each entry is of type TypeMapEntry.
typedef struct {
PyTypeObject* py_type;
jclass jy_class;
jclass jy_subclass;
SyncFunctions* sync;
size_t truncate_trailing;
unsigned short flags;
char* type_name;
} TypeMapEntry;
This is the PyTypeObject for the C API PyObject
This is the JNI class ID for the Jython PyObject
To let a builtin type support being extended by native custom types, a specialized PyCPeer (see PyCPeer.java) class is needed. That class would not subclass PyCPeer but instead the Jython class corresponding to the natively subclassed builtin type. Thus it can be used in place of the subclassed builtin type on Jython side. To indicate its nature as a PyCPeer it must implement CPeerNativeDelegateSubtype. If a builtin type supports native-level subclassing in JyNI the corresponding special PyCPeer-like class must be registered in this field (jy_subclass). Currently only Tuple and Dictionary support this. For more detail see the Java classes PyDictionaryCPeer, PyTupleCPeer, especially javadoc of PyDictionaryCPeer. Note that PyCPeerType does not fit into this pattern as it vastly predates the introduction of this concept. It may be refined to fit into this scheme at some point.
This is a struct for the sync functions:
typedef struct {
jy2pySync jy2py;
py2jySync py2jy;
jyInitSync jyInit;
pyInitSync pyInit;
pyChecksum pyCheck;
jyChecksum jyCheck;
} SyncFunctions;
This should only be needed if using the mirrored or partially mirrored pattern(TODO wiki page on 3 patterns. For now, this is a good start: https://arxiv.org/pdf/1404.6390.pdf section 4-4.4), if C/Java is simply wrapping the other then this won't be needed.
If mirroring:
- jy2py and py2jy will be called when synchronizing an object's state from one already existing object to the other. The resulting object must already exist. Only its internal state is adjusted to match the state of the input object.
- jyInit and pyInit will be called when converting an existing object to a not yet existing counterpart. The resulting object is newly created here, thus 'initialized'.
- pyChecksum and jyChecksum are currently not used. They are intended to check based on a checksum if an object and its counterpart have diverged in internal state, i.e. if a call to jy2py or py2jy would be required.
This is the amount of memory that should be allocated to the PyObject on the C side when partially mirroring, for example: truncate_trailing=4*sizeOf(int) would allow the first 4 integers(excluding the header) in the PyObject to be stored. All the memory before the field of interest must be allocated so that the memory will line up with where macro's expect it to be. This cam allow macros that access the memory inside C API PyObjects to work. But in this case, the memory accessed does need to be kept in sync, which may require a sync function.
This tells Jython about the builtin type. TODO I'm not sure what all the flags do, add a list and explain them.
This is the name of the type
You will add a code block that looks something like this:
builtinTypes[TME_INDEX_KeysIter].py_type = &PyDictIterKey_Type;
builtinTypes[TME_INDEX_KeysIter].jy_class = KeysIterClass;
builtinTypes[TME_INDEX_KeysIter].flags = JY_TRUNCATE_FLAG_MASK;
PyDictIterKey_Type.tp_flags |= Jy_TPFLAGS_DYN_OBJECTS;
The details will be different depending on what it is that you are adding. For instance, some classes need to sync some values, in which case you will have something like this:
builtinTypes[TME_INDEX_Set].py_type = &PySet_Type;
builtinTypes[TME_INDEX_Set].jy_class = pySetClass;
builtinTypes[TME_INDEX_Set].flags = JY_TRUNCATE_FLAG_MASK | JySYNC_ON_INIT_FLAGS;
builtinTypes[TME_INDEX_Set].truncate_trailing = sizeof(Py_ssize_t);
builtinTypes[TME_INDEX_Set].sync = malloc(sizeof(SyncFunctions));
builtinTypes[TME_INDEX_Set].sync->jyInit = NULL;//(jyInitSync) JySync_Init_JySet_From_PySet;
builtinTypes[TME_INDEX_Set].sync->pyInit = (pyInitSync) JySync_Init_PySet_From_JySet;
PySet_Type.tp_flags |= Jy_TPFLAGS_DYN_OBJECTS;
some classes sync all of their contents, in which case you will have something like this:
builtinTypes[TME_INDEX_Tuple].py_type = &PyTuple_Type;
builtinTypes[TME_INDEX_Tuple].jy_class = pyTupleClass;
builtinTypes[TME_INDEX_Tuple].jy_subclass = pyTupleCPeerClass;
builtinTypes[TME_INDEX_Tuple].flags = JySYNC_ON_INIT_FLAGS;
builtinTypes[TME_INDEX_Tuple].sync = malloc(sizeof(SyncFunctions));
builtinTypes[TME_INDEX_Tuple].sync->jyInit = (jyInitSync) JySync_Init_JyTuple_From_PyTuple;
builtinTypes[TME_INDEX_Tuple].sync->pyInit = (pyInitSync) JySync_Init_PyTuple_From_JyTuple;
PyTuple_Type.tp_flags |= Jy_TPFLAGS_DYN_OBJECTS;
until a full wiki on the patters exists:
partially mirroring is basically storing some data in the C API PyObject and mirroring that data across to the jython pyobject while the rest of the C API PyObject is treated as being a wrapper for java. in https://arxiv.org/pdf/1404.6390.pdf wrapper for java is section: 4.1 mirroring is section: 4.2