Developer Documentation

Implementation Overview

The basic idea behind the implementation of Gargoyle is to do as much of the computation on the client side as possible. The router typically has a 200mhz processor, 32MB or less of RAM and 8MB or less of disk (flash drive) space. The clients connecting will typically have multi-GHz processors and over 512 MB of RAM. Even without considering delay introduced by the network connection, which do you think is going to be faster? Therefore, the majority of Gargoyle is implemented as Javascript. This results in a much faster, more flexible “feel” to Gargoyle. Source files for the javascript can be found in the /www/js directory.

The server-side scripting that is necessary is done using haserl, a very lightweight utility that enables the embedding of shell script into html files. The html/haserl scripts are the *.sh files in the /www directory. This is the same mechanism that X-wrt uses to run server-side scripts, although X-wrt tends to rely much more heavily on these scripts and much less on javascript.

The /www/utility directory contains shell scripts that are never displayed directly by the browser. In particular note the /www/utility/run_commands.sh script. This script is passed a single variable (computed by javascript) which contains a list of shell script commands to execute, which it then does. Yes, this IS a giant, gaping security hole – or would be if access to the router web interface didn't already mean that you could gain complete access to the system. Why bother creating a correctly formatted http request to run your commands when you could just go to the system/access page, change the root password, and then give yourself remote ssh access? If you have access to the gargoyle interface, by definition you already have complete control of the router.

The /www/utility/ directory also contains several scripts for restarting things. It's generally better to use these scripts to restart things because when you restart one module several others that depend on it may need to be restarted as well. In particular, the firewall, QoS and Bandwidth modules depend heavily on each other. Restarting everything could be accomplished by rebooting the router, but merely using these scripts is a more elegant solution.

The header/footer code is generated from the /etc/config/gargoyle config file by a utility written in C. This is necessary, as shell scripts execute slow enough to make loading the page take forever. You can use the /etc/config/gargoyle config file to disable/enable display of pages in the menu. This is how the qos_distributions page is enabled whenever qos is active and disabbled whenever qos is no longer active. The script section associates a shell script file name with an id for each section. The id must be of the form [SECTION-ID]_[PAGE-ID]. Similarly the display section associates a name to display with each page, as well as each section. There is also a section in the config for each menu section. The section is associated with a number as is each sub-section, defined as the options within the config section. These numbers indicate the order that the subsections will appear in the menu. To prevent a page from being displayed just remove it from the menu subsection it is in. A whole section can be removed by deleting the entire config section. It is not necessary to delete the page information in the scripts and/or display section.

To genenerate the menu from the config file the gargoyle_header_footer utility is then run with options that specify the section and page:

 -m Generate Minimal Header (for popup boxes used for editing table rows)  
 -h Generate Standard Header 
 -f Generate Footer 
 -s [SECTION-ID] Section/Main Menu Id 
 -p [PAGE-ID] Page Id 
 -c [CSS FILES] List of additional css files necessary for page 
 -j [JS FILES] List of additional javascript files necessary for page 
 -t [TITLE] Title of page 
 -i Include output of javascript variables that specify network interface ips and MAC addresses  

The utility is run once at the beginning of the page with the -h option to generate a header and once at the end with the -f option to generate the footer. The source for the gargoyle_header_footer utility is included in the /src subdirectory of the gargoyle package.

For more information about writing/compiling C/C++ programs for OpenWrt you may find this tutorial helpful.

This documentation is incomplete and there are probably many subtle points that should be addressed here and are not. However, this should be enough information to get started working with Gargoyle. If you have questions post to the forum or contact the author directly. This page will be updated as issues and points of interest are raised in the forum and via email.

Building from source

The best way to obtain the latest Gargoyle source code is to clone the public Git repository.

You can browse the repository online here, or clone from one of the URLs below. There is both a local copy of the repository and a Github mirror:

apt-get install git
git clone git://gargoyle-router.com/gargoyle.git

git clone http://gargoyle-router.com/git/gargoyle.git 
git clone git://gargoyle-router.com/gargoyle.git 
git clone http://github.com/ericpaulbishop/gargoyle.git 
git clone git://github.com/ericpaulbishop/gargoyle.git

If you want to build an older branch or tag, use the “git checkout” to switch to it after cloning the repository. e.g. to switch to the 1.2 branch:

git checkout 1.2

Anyone who still prefers working with subversion can make use of the subversion support provided by the github mirror:

svn checkout http://svn.github.com/ericpaulbishop/gargoyle.git
cd gargoyle
make

You'll notice there are two scripts in the build directory: full-build.sh and rebuild.sh

If you run make without the FULL_BUILD=true flag, and you've previously done a build, it will only rebuild the packages and not the underlying version of OpenWrt. This speeds up testing small changes in the packages dramatically. However, to do a complete build from scratch you really need to run the make command with “FULL_BUILD=true” set:

make FULL_BUILD=true

Or to build (for example) just the ar71xx architecture:

make FULL_BUILD=true ar71xx

Allow building of only one profile for a given architecture by running e.g. “make ar71xx.usb_large” to build usb_large profile instead of all ar71xx profiles

make ar71xx.usb_large

ar71xx profiles

  • default
  • routerstation
  • usb_large
  • usb_no_tor

Building

Building Gargoyle for the first time for even one supported platform can take several hours of churning on your computer. In addition you can expect each platform to require about 1.6GB of disk space so keep this in mind when deciding how many platforms you will build on your first attempt. Finally, be aware that only the components unique to Gargoyle are stored in the Gargoyle svn. Therefore, in order to build gargoyle, the build process must first download many additional components. If you do not have an internet connection, the build will fail. If it is a slow connection, the build will take a very,very long time.

Before you can build Gargoyle, you will require the appropriate tools to build OpenWRT.

To install these (list edited to add the additional things missing on Ubuntu Server 10.10), use the following on a 32 bit system:

sudo apt-get install build-essential asciidoc binutils bzip2 gawk gettext git libncurses5-dev libz-dev patch unzip zlib1g-dev subversion flex uglifyjs 

Or for a 64 bit system:

sudo apt-get install build-essential asciidoc binutils bzip2 gawk gettext git libncurses5-dev libz-dev patch unzip zlib1g-dev lib32gcc1 libc6-dev-i386 subversion flex uglifyjs git-core gcc-multilib p7zip p7zip-full

For more details on building OpenWRT, you can see their wiki page here.

Simply entering:

make

will result in all targets being built. If you prefer to build a single target (in this example ar71xx, though you could substitute brcm47xx or another valid target), enter:

make ar71xx

The makefile calls the full-build-backfire.sh script found in the same directory. This script controls the overall build process. In addition to the makefile and the build script you will find these subdirectories in the Gargoyle root build directory:

dev-utils/ This directory contains a few scripts that can be useful to developers. They are not used during the normal build process.

fon-flash/ This directory contains code for the fon-flash utility, for flashing redboot based routers. Technically, this is not part of Gargoyle, but it is closely associated with the project, so it's here. This does not automatically get built – to build fon-flash you need to cd into this directory and explicitly build it.

netfilter-match-modules/ This directory contains Gargoyle specific extensions to netfilter. These are used in Gargoyle to implement bandwidth monitoring,quotas,time range matching and web url matching. This code is separate from the package directory because they need to be applied as kernel patches before the kernel is built. Because Gargoyle/OpenWrt supports multiple kernels, the necessary patch files are computed from source files at build time. This makes it a lot easier to make changes to the netfilter modules since a new patch doesn't have to be pieced together by hand every time a change is made.

package/ This directory contains the majority of Gargoyle sources. Each subdirectory is an OpenWrt package. For the most part these are specific to Gargoyle. These packages are installed in the target directories after the OpenWrt kernel for that target is built.

patches-generic/ This directory contains Gargoyle specific kernel patches. These are applied to OpenWrt sources in each target directory before the kernel is built. Unlike the netfilter-match-module situation these patches are manually created so changes to them are more tedious.

sched-modules/ This directory, for now, is just a placeholder. It contains a script that, in the future, may be used to incorporate custom packet scheduling modules (for QoS) into the kernel, much as the netfilter-match-modules folder contains custom netfilter modules.

targets/ This directory contains the configuration files needed to properly build Gargoyle on each of the platforms it supports. There is a profiles subdirectory for each target, and there may also be a patches subdirectory for target-specific patches.

Within the profiles subdirectory there will be one or more directories, each containing a config file (the openwrt .config file for this profile), and a profile_images file which specifies which target images this configuration profile applies to. Only images matching the patterns listed in this file will be copied to the images/[arch]/ directory. The .config file for each target drives the entire build process. You can examine these .config files to see which options are enabled for each target. There are some differences due to the different RAM and FLASH availability in various models.

A target may have a patches subdirectory. If it does then this directory contains patches specific to the target. These patches are applied only during the build of the indicated target and only after the generic patches are applied.

The above directories are the ones in the Gargoyle Git. After running make additional directories are created on your box as follows:

downloaded/ The process of building Gargoyle and OpenWRT requires downloading packages from the internet. These packages can be rather large so rather than download them each time you build, Gargoyle creates this local “cache” of packages it downloads. Each time you build the script will compare the cache with the latest available and only downloads them when required. This speeds subsequent builds especially on slow internet connections. This directory also contains the OpenWrt sources downloaded from svn.openwrt.org/openwrt/branches/backfire.

<target>-src/ When a target is built the backfire-src directory (which is never modified) is copied to this directory, where an openwrt build environment for the desired target is configured. The contents of this directory can be treated exactly as you would any other OpenWrt build environment – this is just specifically setup/configured to build Gargoyle.

built/<target> This directroy contains all of the OpenWrt package files (*.ipk) built for Gargoyle. If everything goes well all the packages which are a standard part of Gargoyle end up here. A complete set of packages is built for each target.

images/<target> The image output directory. Once all the output packages are built we can make images. These are the .bin and .trx files often used to flash routers. There will be a complete set of images for each target you build.

It is also possible to build Gargoyle using a custom build configuration. The process of building OpenWRT involves a configuration step in which packages are either included or excluded from the build image. Due to a lack of memory space on most routers it is not possible to included every package. The architecture of OpenWRT allows most packages not included in the original build image to be installed later if they are needed and space is available. So the general philosophy is to build with the least code practical in order to support routers with the least available memory. If you wish to create your own image with a different set of packages in the base image you should run make custom.

make custom

You will then be given access to the OpenWrt menuconfig screen and can select the packages you want to include in your base image.

developer_documentation.txt · Last modified: 2014/08/25 06:31 by ispyisail
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported