Skip to content

Latest commit

 

History

History
254 lines (193 loc) · 10.4 KB

HACKING.adoc

File metadata and controls

254 lines (193 loc) · 10.4 KB
Raw

How to hack the cutting-edge Gauche

Note
 This document is for those who wants to hack the bleeding-edge Gauche source from the git repository. If you just want to compile from the distribution, you don’t need to read any further.

TOOLS REQUIRED FOR BUILDING

On Unix platforms

In order to build Gauche from the git sources, instead of from the distribution, you need to have the latest release of Gauche installed on your machine, for it is required to generate some C files. If you don’t have one, grab the release tarball and install it.

You also need a C compiler supporting C11, e.g. gcc (>= 4.9) or clang (>= 3.3), autoconf (>= 2.71), automake (>= 1.14.1), libtool (>= 2.4.2), and pkg-config (>= 0.26). (Note on macOS users: Xcode now only includes subset of autotools. You have to install full set either manually or thru MacPorts or Homebrew. Xcode’s libtool does not work).

The git source tree doesn’t include configure scripts. You can use the DIST script like the following to generate them.

$ ./DIST gen

Afterwards, you just do the usual ./configure + make. For the configure options, see the INSTALL.adoc.

In case if the latest release of Gauche can’t be built on your target machine but the fix is already in the HEAD, you need to take more steps — you need the source with that fix, but in order to build it, you need the latest release of Gauche installed, which you can’t. If you stuck in this Catch-22 situation, you have to generate tarball on other machines. Find a Linux box (or BSD, or most Unix box would do) and take the following steps:

  1. Build and install the latest release of Gauche on that box

  2. Clone the source on that box, and build the HEAD as explained above.

  3. Run

    $ ./DIST tgz

    This creates a tarball Gauche-$VERSION.tgz in the parent directory of the source tree.

  4. Copy the created tarball to your target machine, untar, configure and build. The created tarball includes generated files, so you don’t need Gauche in this step.

Note for seasoned libtool/automake users: the git source tree includes files that are usually generated automatically by autotools, such as aclocal.m4, ltmain.sh, src/gauche/config.h.in, etc. Avoid clobbering these files, unless you’re sure you want to do so. If your autotools version doesn’t match mine, it tends to break.

On Windows

The recommended way to build Gauche on Windows is mingw-w64 and MSYS2. If you want to build from git source, you need Gauche of the latest release to be installed. The easiest way is to grab the windows installer from https://practical-scheme.net/gauche. For the details of build procedure, see doc/HOWTO-mingw.adoc.

You can build Gauche with Cygwin, but it is no longer actively supported.

Some effort was made to build Gauche with MSVC. You can find project files under winnt. Unfortunately though, it is so much pain to make it work and keep it updated with newer Gauche source tree, and now they are quite obsolete.

BUILD STEPS

After confiugre, just running make build everything. But you want to know how build process proceeds if you got into trouble hacking it. Here’s the internal steps after you type make.

  1. The Boehm-Demers-Weiser garbage collector (bdwgc) is built under gc/. We included the bdwgc source instead of relying on the shared library, since we need a custom configure options to be optimal for Gauche memory usage pattern.

  2. The 'core' engine is built under src/. This includes libgauche-X.so and libgauche-static-X.a, and the gosh command. Some sources are written in Gauche, so they need to be processed with working Gauche; thus you require the latest release of Gauche installed on your system. Note that, at this point, no precompiled extensions under ext/ are available. If you simply 'use' one of those modules during the building, it can’t load the module. To avoid this, the host gosh loads a file src/preload.scm, which lists the modules the host gosh need to load from the host environment, not from the source tree.

  3. The extensions are built under ext/. If it’s not cross compiling, this process uses the freshly built gosh in src/. Some modules under this directory have C code, while others are pure Scheme. The pure Scheme source may be placed under libsrc/, so that those modules are available during the build process; that is, during the build process the modules are loaded in the source form from libsrc/, and once the build is completed, they are dynamically loaded as precompiled modules.

  4. Documents under doc/ is built. Info documents are built only when the system has newer makeinfo.

Knowing which module is used during build, the one in the host environment or the one in the source tree, is important. Modules listed in preload.scm are from the host environment, which means you can’t use the feature added after the latest release of Gauche to build the current source.

You can make sure that the source tree is buildable both by the latest relase and the current built, by running ./DIST self-host-test. It also checks if out-of-tree build works. It performs the following opertaions:

  • It cleans the source tree (by make maintainer-clean and regenerate configure scripts.

  • It creates a temporary directory and run configure in it. This sets up out-of-tree build.

  • It builds Gauche using the host environment (latest release) and runs the tests.

  • It installs the built Gauche inside the temporary directory.

  • It builds Gauche again, using the freshly built Gauche, and runs the tests again.

It leaves a temporary directory ../Gauche-tmp-self-host-test afterwards, so that it’s easier to investigate if something goes wrong. You can remove that directory if build succeeds.

*.sci files

After building the source, you’ll see files with sci suffix. It stands for 'SCheme Interface'. It is generated from *.scm file with precomp, and contains define-module form with using modules and exporting symbols, and a form to dynamically load precompiled file.

If you have foo.scm and foo.sci in the same directory and you try to load foo, sci take precedence. Usually, if you change the source *.scm file and run make, the *.sci file is regenerated. But if you forgot to run make, you may wonder why the changes you did in *.scm file aren’t reflected.

If you’re not sure which file is actually loaded, run gosh with -fload-verbose option; it reports the exact pathname of each loaded file.

The *.sci files are renamed to *.scm when installed, so you won’t see them in the installation directory.

The reason that we use *.sci files is for the precompiler not to overwrite the source file. See doc/HOWTO-precompile.adoc for the details.

DEPENDENCIES

Quite a few files are generated by Gauche itself if you build Gauche from scratch. Consequently, there are quite a few nasty dependency issues. If you modify files listed below, be careful not to introduce undesirable dependencies.

  • The compiler (compile*.scm) shouldn’t depend on anything that are not compiled into the libgauche core. An exception is util.match; it’s ok since it is just a macro and all match forms are expanded fully in precompilation.

  • precomp, genstub, geninsn shouldn’t depend on the extension modules except the ‘pre-loaded’ ones in BUILD_GOSH.

  • An extension compiled by precomp shouldn’t depend on other extensions compiled by precomp. This is because precomp is run by BUILD_GOSH and it may not be able to load the other extensions compiled for the target gosh.

  • ext/xlink shouldn’t depend on anything that requires loading extension modules, since it is called before all the extension modules are generated.

CROSS COMPILATION

In a normal compilation, extension modules (ext/*) are build using the new gosh just built in src/. However, we can’t run src/gosh when we’re cross compiling. So you need to install this version of Gauche compiled on your platform beforehand. Then, configure with the ordinary cross-compiling options.

CHANGELOGS

We have ChangeLog file to record changes of new features, APIs, or internal changes large enough so that other developers need to be aware of. Small changes like ‘quick fix of the bug introduced in the last commit’, or documentation additions, don’t need to be recorded.

In the age of distributed development, ChangeLog seems becoming obsolete, for it causes conflicts easily. Yet I still see it worth, for it is one-stop place where somebody who tries to grasp what was changed since some point of time. SCM’s commit log doesn’t replace that role — commit logs comes with every commit, however little it is, and those small trivial logs are just noise to those want to know changes that matter. Also, commit logs are inherently per-commit, so it’s not a good place to describe overall intention or a plan of series of changes.

To keep ChangeLog in that role and to be friendly to distributed development, I suggest that you don’t update ChangeLog unless you feel you really want to. If I see your change is better mentioned in ChangeLog, I’ll update it separately.

CODING STYLE

We aren’t picky about coding style, but adopt some Gauche-specific style, especially for Gauche macros. For Emacs users, such style is written in .dir-locals.el in the top source directory. The Emacs package lisp-local (available from MELPA) can automatically apply the settings.

TROUBLESHOOTING

If your modification break something fundamental and you can’t even get to REPL, there are a couple of tricks to isolate the problem.

  • Run gosh with -q option, e.g. ./gosh -ftest -q. It suppresses loading startup files, e.g. it won’t read any external Scheme files, nor loading DSO files, before getting to a REPL prompt. If you get gosh> prompt, then the issue is the loading of runtime files. — Run gosh with ./gosh -ftest -fload-verbose option and see which startup file causes the trouble. — With the bare REPL from ./gosh -ftest -q, try to load or evaluate offending files/expressions to figure out the issue. Note that the 'bare' REPL only have minimal features---no toplevel commands, no line editing, etc.

  • If you can’t even get to REPL with -q, it is likely that something is broken during initialization. Run gosh with an environment variable GAUCHE_DEBUG_INITIALIZATION set. It reports which component is being initialized.