Compiling an Empty Addon on Windows: KanoAs an example, I want to compile an addon without any native code. I noticed that all installed addons, even the one without any native code, e.g. CGI, do have an init file and a dll. Maybe it is not strictly necessary.
AddonLoader, a helper to the importer, checks for more than one C file - but I will need it later anyway. How does such an empty init file look like? For example, in the distribution IoCGIInit.c is#include "IoState.h"
#include "IoObject.h"
__declspec(dllexport)
void IoCGIInit(IoObject *context)
{
}Easy enough to create such a file Kano/sources/IoKanoInit.c. In the source distribution which I will mention later, there is generate.io in the root of the addons, which does exactly that. The script accepts a path to the addons dir and the name of the addon:> cd "%IO_HOME%\lib\io\addons" > io generate.io . Kano > cd Kano\source > gcc -fPIC -D _NO_OLDNAMES -c IoKanoInit.c IoKanoInit.c:1:21: fatal error: IoState.h: No such file or directoryOf course I need headers. Where is the source distribution?
Io Source Distribution
The latest binary I downloaded from the Io site is iobin-win32-current.zip. Little version information there. Io reports
io --version a version of 5.9.2011 but the executables and libraries were in fact created 5.11.2013. The nearest source release is 4.12.2013 which seems to be compatible. (The older 5.9.2011 source release is definitely incompatible with the addons included in iobin-win32-current.zip.) The source distribution is also interesting because it contains all sources of the standard library in libs\iovm\io\*.io.The required headers are in
libs\iovm\source\*.h and libs\basekit\source\*.h. After setting proper include paths (C_INCLUDE_PATH), compilation is possible. The switch -fPIC seems to be redundant on Windows, but it is set in one of Io's makefiles, so I keep it. I have no idea what I am doing ;-) The switch -D _NO_OLDNAMES defines _NO_OLDNAMES which solves the error: conflicting types for 'SSIZE_T' which occurs when old names are allowed. Sigh. Again, no idea what I am doing.> cd Kano\source
> gcc -fPIC -D _NO_OLDNAMES -c IoKanoInit.c
> gcc -liovmall -shared -Wl,--out-implib,libIoKano.dll.a ^
IoKanoInit.o -o libIoKano.dll
> mkdir ..\_build\dll
> move lib*.* ..\_build\dll
> cd ..\..
> io -e "Kano supportedFiles println"
list(make.io, Kanofile, kanofile, Kanofile.io, kanofile.io)Compiling Native Code: RainbowNext I tackle addons with native code but without any third party dependencies. Most addons of this type are included in the distribution, because their compilation does only depend on Io itself, e.g.
Blowfish, Box, LZO, MD5, Random, Range, etc. An addon I found which is missing is Rainbow, which "allows you to colourize command line output." Rainbow does not have a protos file and it has no package.json neither, but there is an eerie.json which does not say anything about prototypes. The addon contains a single Io file Rainbow.io and a single C header IoRainbow.h. I guess this is the prototype then - see line (1) in the following shell commands:> git clone https://github.com/IoLanguage/Rainbow Cloning into 'Rainbow'... > touch Rainbow\depends > echo Rainbow > Rainbow\protos && rem (1) > io -e "Rainbow println" Exception: Failed to load Addon Rainbow - it appears that the addon exists but needs compiling."This is a useful error message. Io's
AddonLoader checks native sources when loading an addon. Compile it. First> io generate.io . Rainbowcreates a different
IoRainbowInit.c file this time,#include "IoState.h"
#include "IoObject.h"
IoObject *IoRainbow_proto(void *state); // (2)
__declspec(dllexport)
void IoRainbowInit(IoObject *context)
{
IoState *self = IoObject_state((IoObject *)context);
IoObject_setSlot_to_(context, SIOSYMBOL("Rainbow"), IoRainbow_proto(self)); // (3)
}I have marked relevant references to the addon a.k.a. the "primary" prototype in bold face. Line (2) is the forward reference to code in Rainbow.c, which will create a new prototype. Line (3) sets this prototype under the symbol of its name into a slot of the context. (Io stores all variables in slots, many functions in the standard library work with slots, e.g. slotNames() or getSlot(name).) There is some more information about writing C addons for Io in the Io Wikibook. For example it explains the addon structure - which I already knew but at least my "research" is validated. ;-)Back to the compilation of Rainbow. Continue after line (1) in the previous script:
> cd Rainbow\source
> gcc -fPIC -D _NO_OLDNAMES -c IoRainbowInit.c IoRainbow.c
> gcc -liovmall -shared -Wl,--out-implib,libIoRainbow.dll.a ^
IoRainbowInit.o IoRainbow.o -o libIoRainbow.dll
Creating library file: libIoRainbow.dll.a
IoRainbow.o:IoRainbow.c:(.text+0x5c): undefined reference to `_imp__IoTag_newWithName_'
... more errors omitted
Of Course, Linker ErrorsThese errors are strange and I spent a long time investigating. The
libiovmall.dll.a exports __imp__IoTag_newWithName_, with two underscores in the beginning but the linker only looks for one. I verified this with nm libiovmall.dll.a | grep __imp__. This is due to some compiler inconsistency between Linux and Windows and/or GCC and/or MingW regarding function name decoration. GCC Options "-fleading-underscore and its counterpart, -fno-leading-underscore, change the way C symbols are represented in the object file." I tried both of them but then other symbols like fprintf are not found any more. There might be a linker option -Wl,--add-stdcall-underscore but it is not recognised by my MingW GCC. Sigh.The solution is to not link against the link libraries (
*dll.a), but against the dlls directly. (Either delete the *.dll.a from the io/lib folder or change your LIBRARY_PATH to use io/bin instead.) Summary: When your linker looks for a single underscore in dll exported symbols, and the library exports correctly, link against the dll instead of the lib..The change to the library path fixes the linking and
libIoRainbow.dll is created. The dll must be moved to the build result folder ..\_build\dll as before. In addition all headers should be copied to _build\headers, as other addons might depend on them.> mkdir ..\_build\dll > move lib*.* ..\_build\dll > mkdir ..\_build\headers > copy *.h ..\_build\headers > cd ..\.. > io -e "Rainbow println" Rainbow_0x29ba488All addons contained in the Windows distribution have the
_build\headers folder, but there are no headers. I will need these headers to compile Regex in part three. Let's iterate all addons and copy the headers:set ADDONS=%IO_HOME%\lib\io\addons
dir /A:D /b "%ADDONS%" > addons.txt
for /F %a in (addons.txt) do ^
if exist "%ADDONS%\%a\source\*.h" ^
copy /Y "%ADDONS%\%a\source\*.h" "%ADDONS%\%a\_build\headers"
del addons.txtAddon Initialization Recursion IssueOne recurring issues I faced was that
AddonLoader was recursing endlessly trying to load something. For example, when I created IoRainbowInit.c by hand, I made a mistake with the symbol in line (3). On loading the addon, AddonLoader entered an endless loop,> io -e "Rainbow println" IOVM: Received signal. Setting interrupt flag. current coroutine --------- Object Rainbow Rainbow.io 14 FileImporter importPath Z_Importer.io 49 <- FileImporter import Z_Importer.io 125 | List detect Z_Importer.io 125 | true and Z_Importer.io 125 | Object Rainbow Rainbow.io 14 | Importer import Z_Importer.io 138 | FileImporter importPath Z_Importer.io 49 -- ...This was because the proto was not registered under its correct name, see
SIOSYMBOL in line (3) above, and the importer tried to import it again, found the addon (again), and repeated. Tip: When addon importing enters a loop, double check the symbol name in the C init.A similar error happens when the a prototype of an addon uses another prototype of that addon during code loading. For example Eerie comes with two prototypes:
Eerie and SystemCommand. The first line of Eerie.io imports SystemCommand. Now when I evaluate Eerie println, the AddonLoader finds the Eerie addon (by name) and loads the file Eerie.io to load that prototype. During import it sees SystemCommand as a dependency and looks for it. It finds it in the proto file of Eerie and tries to load it. When loading the addon, it loads the prototype of the same name, i.e. Eerie.io again.> io -e "Eerie println" IOVM: Received signal. Setting interrupt flag. current coroutine --------- Object SystemCommand Eerie.io 4 Addon load AddonLoader.io 124 <- AddonLoader loadAddonNamed Z_Importer.io 97 | FolderImporter import Z_Importer.io 125 | List detect Z_Importer.io 125 | true and Z_Importer.io 125 | Object SystemCommand Eerie.io 4 | Importer import Z_Importer.io 138 | Addon load AddonLoader.io 124 -- ...Cyclic imports are a bad idea anyway, it is fair that this does not work. Still I have not seen any warning about it in the Io documentation. Tip: Avoid top level "imports" of prototypes from the same addon.
Compiling Native Code: Thread
Another addon without dependencies is Thread. It has a
proto, depends and package.json file. Using generate.io I follow all the steps from Rainbow above, just with different C files and compilation and linking succeeds. Unfortunately Thread createThread("1+1"), as shown in the documentation, terminates the Io VM when evaluating the expression string (1+1). There are no further hints or messages. I have no idea why and no means to debug the Io virtual machine.Follow along in Part three (coming soon) about compiling addons with third party dependencies.
