Snippets that should end up in the documentation: RECIPE STRUCTURE Generally there are these sections in a recipe used for building an application: 1. global settings, include recipes with (project, user) settings 2. automatic configuration 3. specify variants 4. generic build rules 5. explicit dependencies For larger projects sections can be moved to other recipes. 1. global settings, include recipes with (project, user) settings When the recipe is part of a project, it's often useful to move settings (and rules) that apply to the whole project to one file. User preferences (e.g. configuration choices) should be in a separate file that the user edits (using a template) 2. automatic configuration Find out properties of the system, and handle user preferences. This may result in building the application in a different way. 3. specify variants Usually debug and release, but can include many more choices (type of GUI, small or big builds, etc.) 4. generic dependencies and build commands Rules that define dependencies and build commands that apply to several files. 5. explicit dependencies and build commands Dependencies and build commands that apply to specific files. COMMAND LINE ARGUMENTS aap [options] [target] Note that all the options must come before any target. If "target" is omitted, one of the targets in the recipe is executed, see RECIPE EXECUTING. OPTIONS: -a --nocache For remote files, don't use cached files, download it once for this invocation. -c command --command command After reading the recipe, execute "command". May appear several times, all the commands are executed. See COMMAND LINE COMMANDS -d flags switch on debugging for 'flags' -f FILE --recipe FILE specify recipe to execute -h --help print help message and exit -I DIR --include DIR directory to search for included recipes -j N --jobs N maximum number of parallel jobs -k --continue continue after encountering an error -n --nobuild only find out what will be done, don't execute build rules -R --refresh-recipe refresh the recipe and child recipes. This is impled by using a "refresh" or "update" target. -S --stop stop building after encountering an error (default) -s --silent print less information, see MESSAGES. -u --up --search-up search directory tree upwards for main.aap recipe -V --version print version information and exit -v --verbose print more information, see MESSAGES. RECIPE EXECUTING: This is done in these steps: 1. Read the startup recipes, these define default rules and variables. These recipes are used: - "default.aap" from the distribution - all recipes matching "/usr/local/share/aap/startup/*.aap" - all recipes matching "~/.aap/startup/*.aap" 2. Read the recipe and check for obvious errors. 3. Execute the toplevel items in the recipe. Dependencies and rules are stored. 4. Apply the clever stuff to add missing dependencies and rules. 5. Update the targets. The first of the following that exists is used: - targets specified on the command line - the items in $TARGET - the "all" target - the first target in the recipe 6. If the "finally" target is specified, execute its build commands. When no dependency is specified, there is one item in $TARGET and $SOURCE is not empty, a dependency is automatically generated. This will have the form: $TARGET : $SOURCE_OBJ :sys $CC $CFLAGS $LDFLAGS -o $target $source Where $SOURCE_OBJ is $SOURCE with all items changed to use $OBJSUF, if there is a suffix. Example: TARGET = foo SOURCE = main.c extra.p Results in: foo : main$OBJSUF extra$OBJSUF :sys $CC $CFLAGS $LDFLAGS -o $target $source This then uses the default rules to make "main$OBJSUF" from main.c and extra$OBJSUF from extra.p. MESSAGES The kind of messages given can be changed with the MESSAGE variable. It is a list of message types for which the message is actually displayed: all everything error error messages warning warnings depend dependencies, the reasoning about what to build info general info (file copy/delete, up/downloads) extra extra info (why something was done) system system (shell) commands changedir changing directories The command line arguments "-v" and "-s" can be used to make the most often used selections: argument MESSAGE (nothing) error,info,system,changedir -v --verbose all -s --silent error Other values can be assigned at the command line. For example, to only see error and dependency messages: aap MESSAGE=error,depend (other arguments) Don't forget that excluding "error" means that no error messages are given! No matter what messages are displayed, all messages are written in the log file. This can be used afterwards to see what actually happened. The name of the log file is "aap/log". This is located relative to the main recipe. Older log files are also remembered. The previous log is "aap/log1". The oldest log is "aap/log9". SIGNATURES The default check for a file that was changed is an md5 checksum. Each time a recipe is executed the checksums for the relevant items are computed and stored in the file "aap/sign". The next time the recipe is executed the current and the old checksums are compared. When they are different, the build commands are executed. This means that when you put back an old version of a file, rebuilding will take place even though the timestamp of the source might be older than the target. Another check can be specified with {check = name}. Example: foo.txt : foo.db {check = time} :sys db_extract $source >$target Other types of signatures supported: time Build the target when the timestamp of the source differs from the last time the target was build. newer Build the target if its timestamp is older than the timestamp of the source. This is what the good old "make" program uses. md5 Build the target if the md5 checksum of the source differs from the last time the target was build. This is the default. c_md5 Like "md5", but ignore changes in comments and amount of white space. Appropriate for C programs. (not implemented yet). none Don't check time or contents, only existence. Used for directories. When mixing "newer" with other methods, the build rules are executed if the target is older than the source with the "newer" check, or when one of the signatures for the other items differs. The "aap/sign" file is normally stored in the directory of the target. This means it will be found even when using several recipes that produce the same target. But for targets that get installed in system directories (use an absolute path), virtual targets and remote targets this is avoided. For these targets the "aap/sign" file is stored in the directory of the recipe that specifies how to build the target. To overrule the directory where "aap/sign" is written, use the attribute {signdirectory = name} for the target. COMMAND LINE COMMANDS Commands to be executed can be specified on the command line. This is useful, for example, to refresh individual files: aap -c ":refresh main.c" Since the recipe is first read (and all child recipes), the attributes that are given to "main.c" will be used for refreshing. When no target is specified nothing is build, only the specified commands are executed. But the toplevel commands in the recipe are always executed. Keep in mind that the shell must not change the argument, use single quotes to avoid $VAR to be expanded by the shell: aap -c ':print $SOURCE' BUILD COMMAND SIGNATURE A special kind of signature is used to check if the build commands have changed. An example: foo.o : {buildcheck = $CFLAGS} foo.c :sys $CC $CFLAGS -c $source -o $target This defines a check for the value of $CFLAGS. When the value changes, the build commands will be executed. The default buildcheck is made from the command themselves. But this is without expanding any variables, thus a change in $CFLAGS would not be found this way. To combine the check for the build commands themselves with specific variable values the $commands variable can be used: VERSION = 1.4 foo.txt : {buildcheck = $commands $VERSION} :del -f $target :print >$target this is $target :print >>$target version number: $VERSION If you now change the value of $VERSION, change one of the ":print" commands or add one, "foo.txt" will be rebuild. To simplify this, $xcommands can be used to check the build commands after expanding variables, thus you don't need to specify $VERSION: foo.txt : {buildcheck = $xcommands} However, this only works when all $VAR in the commands can be expanded and variables used in python commands are not expanded. To avoid checking the build commands, use an empty buildcheck. This is useful when you only want the target to exist: objects : {buildcheck = } :mkdir objects RECIPE FORMAT: A backslash at the end of the line is used for line continuation. - Can't have white space after it! - Also for comments. - Also for Python commands; A leading @ char and white space before it is removed. # comment \ continued comment @ python command \ @ continued python command \ still continued VARIABLES: When concatenating variables, the attributes of the last variable overrule the identical attributes of a previous one. v1 = foo {check = 1} v2 = bar {check = 2} vv = $foo$bar -> vv = foobar {check = 2} When using rc-style expansion, quotes will not be kept as they are, but removed and re-inserted where used or necessary. Example: foo: "file 1.c" foo.c :print "dir/$source" Results in: "dir/file 1.c" "dir/foo.c" Be careful with using "$\" and quotes, you may not always get what you wanted. To get one item out of a variable that is a list of items, use an index number in square brackets. Parenthesis or curly braces must be used around the variable name and the index. The first item is indexed with zero. Example: BAR = beer coffee cola :print $(BAR[0]) beer BAR_ONE = $(BAR[2]) :print $BAR_ONE cola Using an index for which no item exists gives an empty result. When $MESSAGE includes "warning" a message is printed about this. Normally using $VAR gets what you want. A-A-P will use the kind of quoting expected and add attributes when needed. However, when you want something else, this can be specified: $var depends on where it's used $-var without attributes $+var with attributes $=var no quotes or backslashes $'var aap quoted (using ' and/or " where required, no backslashes) $"var quoted with " (doubled for a literal ") $\var special characters escaped with a backslash $!var depends on the shell, either like $'var or $"var In most places $var is expanded as $+'var (with attributes, using ' and " for quoting). The exceptions are: :print, :error $-'var no attributes, normal quoting :sys $-!var no attributes, shell quoting $n in $(v[$n]) $-=var no attributes, no quoting :del, :mkdir, :touch $-'var no attributes, normal quoting The quoted variables don't handle the backslash as a special character. This is useful for MS-Windows file names. Example: prog : "dir\file 1.c" :print $'source Results in: "dir\file 1.c" ASSIGNMENT: overview: var = value assign var += value append (assign if not set yet) var ?= value only assign when not set yet var $= value evaluate when used var $+= value append, evaluate when used var $?= value only when not set, evaluate when used Assignment with "+=" or "$+=" appends the argument as a separate item. This is actually done by inserting a space. But when the variable wasn't set yet it works like a normal assignment VAR += something is equal to: @if globals().has_key("VAR"): @ VAR = VAR + " " + "something" @else: @ VAR = "something" Assignment with "?=" only does the assignment when the variable wasn't set yet. A variable that was set to an empty string also counts as being set. VAR ?= something is equal to: @if not globals().has_key("VAR"): VAR = something When using "$=", "$+=" or "$?=" variables in the argument are not evaluated at the time of assignment, but this is done when the variable is used. VAR = 1 TT $= $VAR VAR = 2 :print $TT prints "2". When first setting a variable with "$=" and later appending with "+=" the evaluation is done before the new value is appended: VAR = 1 TT $= $VAR TT += 2 VAR = 3 :print $TT prints "1 2" Note that evaluating a python expressions in `` is not postponed. ATTRIBUTES Items can be given attributes. The form is: {name = value} "value" is expanded like other items, with the addition that "}" cannot appear outside of quotes. This form is also possible and uses the default value of 1: {name} Examples: bar : thatfile {check = $MYCHECK} foo {virtual} : somefile The "virtual" attribute is used for targets that don't exist (as file or directory) but are used for selecting the dependency to be build. These targets have the "virtual" attribute set by default: TARGET COMMONLY USED FOR all build the usual things clean remove intermediate files from building distclean remove all generated files finally always executed last tryout build and install for trying out install build and install for use refresh obtain the latest version of each file update refresh and build the default target commit commit changes to version control system checkout checkout (and lock) from version control system checkin checkin (and unlock) to version control system unlock unlock files from a version control system add add new files to version control system remove remove deleted files from version control system publish distribute the current version These specific targets may have multiple build commands. They are all executed to update the virtual target. Normally there is up to one target in each (child) recipe. Note that virtual targets are not related to a specific directory. Make sure no other item in this recipe or any child recipe has the same name as the virtual target to avoid confusion. Specifically using a directory "test" while there also is a virtual target "test". Name the directory "testdir" to avoid confusion. The "comment" attribute can be used for targets that are to be specified at the command line. "aap comment" will show them. % aap comment target "all": build everything target "foo": link the program The ":attr" command can be used to add attributes to an item without doing anything else. The ":attribute" command is an alias. :attr {attrname} ... itemname ... :attribute {attrname} ... itemname ... Add the list of attributes "{attrname} ..." to each item in the list of items "itemname ...". :attr itemname {attrname} ... :attribute itemname {attrname} ... Add attributes "{attrname} ..." to item "itemname". The above two forms can be mixed. Example: :attr {refresh = cvs://} foo.c patch12 {constant} This adds the "refresh" attribute to both foo.c and patch12, and the "constant" attribute only to patch12. This does the same in two commands: :attr {refresh = cvs://} foo.c patch12 :attr {constant} patch12 When attributes are used in a rule or dependency, most of them only apply to that dependency. But some attributes are "sticky": Once used for an item they are used everywhere for that item. Sticky attributes are: virtual virtual target, not a file directory item is a directory filetype type of file constant file contents never changes refresh list of locations where to refresh from commit list of locations for VCS publish list of locations to publish to STANDARD VARIABLES: name default info $$ a single $ $# a single # $CC cc command to execute the compiler $CFLAGS arguments always used for $CC $LDFLAGS arguments for $CC when linking $CPPFLAGS arguments for $CC when compiling sources $GMTIME GMT time in seconds since 1970 Jan 1 (the time is set once, it remains equal while executing) $DATESTR Date as a string in the form "2002 Month 11" $TIMESTR Time as a string in the form "23:11:09" (GMT) $OSTYPE Type of operating system used: posix Unix-like (Linux, BSD) mswin MS-Windows (98, XP) msdos MS-DOS (not MS-Windows) os2 OS/2 mac Macintosh java Java OS riscos Risc OS ce MS-Windows CE $TARGET list of target files, usually the name of the resulting program $SOURCE list of source files $CACHE List of directories to search for cached downloaded files. Default for Unix: /var/aap/cache ~/.aap/cache aap/cache For MS-Windows, OS/2: $HOME/aap/cache aap/cache Directories that don't exist are skipped. When using a relative name, it is relative to the current recipe. Thus the recipe specified with ":child dir/main.aap" uses a different cache directory. See CACHE below. When this variable is set, currently cached files are flushed. Otherwise this only happens when exiting. Thus this command can be used to flush the cache: CACHE = $CACHE $cache_update Timeout after which cached files may be downloaded again. See CACHING below. $aapversion Version number of A-A-P. E.g., 31 (version 0.031) or 1003 (version 1.003) CACHING: Remote files are downloaded when used. This can take quite a bit of time. Therefore downloaded files are cached and only downloaded again when outdated. The cache can be spread over several directories. The list is specified with the $CACHE variable. NOTE: Using a global, writable directory makes it possible to share the cache with other users, but only do this when you trust everybody who can login to the system! A cached file becomes outdated as specified with the "cache_update" attribute or $cache_update. The value is a number and a name. Possible values for the name: day number specifies days hour number specifies hours min number specifies minutes sec number specifies seconds The default is "12 hour". When a file becomes outdated, its timestamp is obtained. When it differs from when the file was last downloaded, the file is downloaded again. When the file changes but doesn't get a new timestamp this will not be noticed. When refreshing files the cached files are not used. VARIANTS: Here is an example how build variants can be specified: :variant OPT some CFLAGS = -O2 much compiler == "gcc" CFLAGS = -O6 * CFLAGS = -O "OPT" is the name of a variable. It is used to select one of the variants. Each possible value is listed in the following line and further lines with the same indent. In the example these are "some" and "much". "*" is used to accept any value, it must be the last one. The first value mentioned is the default when the variable isn't set. The $BDIR variable will be adjusted for the variant used. CAREFUL: this means that using $BDIR before ":variant" commands will use a different value, that is probably not what you want. When a target that is being build starts with $BDIR and $BDIR doesn't exist, it is created. $BDIR is relative to the recipe. When using ":child dir/main.aap" the child recipe will use a different build directory "dir/$BDIR". Note that when building the same source file twice from recipes that are in different directories, you will get two results. PYTHON EXPRESSION: Python expressions can be embedded in many places. They are specified in backticks. The result should be a string or a list of strings. A list is automatically converted to a white-separated string of all list items. A Python expression cannot be used for the variable name in an assignment. This doesn't work: `varname` = value Use this instead: @eval(varname + ' = "value"') When using a function from a module, it must be imported first. Example: @from glob import glob SOURCE = `glob('*.c')` However, for your convenience these things are imported for you already: from glob import glob A backtick in the Python expression has to be doubled to avoid it being recognized as the end of the expression: CMD = `my_func("``grep -l foo *.c``")` contains the Python expression: my_func("`grep -l foo *.c`") Note that a following attribute is only attached to the last item resulting from the Python expression. SOURCE = `glob('*.c')` {check = md5} Can be equivalent to: SOURCE = foo.c bar.c {check = md5} To apply it to all items, use parenthesis SOURCE = (`glob('*.c')`) {check = md5} Can be equivalent to: SOURCE = (foo.c bar.c) {check = md5} or: SOURCE = foo.c {check = md5} bar.c {check = md5} Backtick expressions and $VAR can be used inside a string: DIR = /home/foo /home/bar :print "`DIR + "/fi le"`" :print "$DIR/fi le" Results in: `echo /home/me` In the result of the Python expression a $ is changed to $$, so that it's not used for a variable name. The $$ is reduced to $ again when evaluating the whole expression. Python functions available: sort_list() sorts a list and returns the list. Example: INP = `sort_list(glob("*.inp"))` PYTHON BLOCK: A block of Python commands is started with a ":python" command. Optionally a terminator string may be specified. This cannot contain white space. A comment may follow. If no terminator string is specified the python code ends where the indent is equal to or less than the ":python" command. Otherwise the Python block continues until the terminator string is found in a line by itself. It may be preceded and followed by white space and a comment may follow. SOURCE = foo.c bar.c :python for i in items: SOURCE = SOURCE + " " + i ... :python <<< INCLUDE = glob("include/*.c") <<< AAP COMMANDS: :rule tpat ... : spat ... commands Define a rule to build files matching the pattern "tpat" from a file matching "spat". There can be several "tpat" patterns, the rule is used if one of them matches. There can be several "spat" patterns, the rule is used if they all exist (or no better rule is found). When "commands" is missing this only defines that "tpat" depends on "spat". Can only be used at the toplevel. The "skip" attribute on 'tpat' can be used to skip certain matches. $target and $source can be used in "commands" for the actual file names. $match is what the "%" in the pattern matched. :update target ... Update "target" now. one or more targets can be specified, each will be updated. When this appears at the top level, a dependency or rule for the target must already have been specified, there is no lookahead. :refresh file ... Refresh the files mentioned according to their "refresh" or "commit" attribute. When a file does not have these attributes or refreshing fails you will get an error message. Files that exist and have a "refresh" attribute with value "no" are skipped. the name "." can be used to update the current directory: :refresh . {refresh = cvs://$CVSROOT} :refresh {attribute} ... file ... Like above, apply {attribute} to all following items. :publish file ... :publish {attribute} ... file ... Publish the files mentioned according to their "publish" or "commit" attribute. :publishall file ... Like ":publish" but also remove files that are not an argument. Careful! :commit :checkout :checkin :unlock :add :remove :verscont Version control commands, see below. :copy [options] from ... to Copy files or directory trees. "from" and "to" may be URLs. This means :copy can be used to upload and download a file, or even copy a file from one remote location to another. Examples: :copy file_org.c file_dup.c :copy -r onedir twodir :copy *.c backups :copy http://vim.sf.net/download.php download.php :copy $ZIP ftp://upload.sf.net/incoming/$ZIP :copy ftp://foo.org/README ftp://bar.org/mirrors/foo/README OPTIONS -f forcefully overwrite an existing file or dir (default) -i before overwriting a local file, prompt for confirmation (currently doesn't work for remote files) -L when used with -r, don't copy a symlink, make a copy of the file or dir it links to -p preserve file permissions and timestamps as much as possible -r -R recursive, copy a directory tree. "to" is created and should not exist yet. -v verbose, mention every copied file (default) Wildcards in local files are expanded. This uses Unix style wildcards. When there is no matching file the command fails (also when there are enough other arguments). When (after expanding wildcards) there is more than one "from" item, the "to" item must be a directory. For "to" only local files and ftp:// can be used. For ftp the ~/.netrc file is used if possible (unfortunately, the Python netrc module has a bug that prevents it from understanding many netrc files). Alternatively, login name and password can be specified just before the machine name: user+password@foo.com When "+password" is omitted, you will be prompted for entering the password. Either way: ftp sends passwords literally over the net, thus this is not secure! Should use "scp://", but that's not implemented yet. Attributes for "from" and "to" are currently ignored. :move [options] from ... to Move files or directories. Mostly like ":copy", except that the "from" files/directories are renamed or, when renaming isn't possible, copied and deleted. OPTIONS -f forcefully overwrite an existing file or directory (default) -i before overwriting a local file, prompt for confirmation -v verbose, mention every copied file (default) :del [options] file ... :delete [options] file ... Delete files and/or directories. OPTIONS -r -R delete directories and their contents recursively -f don't fail when a file doesn't exist -q quiet: don't report deleted files Wildcards in local files are expanded. This uses Unix style wildcards. When there is no matching file the command fails (also when there are enough other arguments). :mkdir dir ... Create directory. This fails with the directory (or a file with the same name) already exists. Note: automatic creation of directories can be done by adding the {directory} attribute to a source item. :touch [-f] name ... Update timestamp of file or directory "name". If "name" doesn't exist and -f is not present this fails. If "name" doesn't exist and -f is present an empty file will be created. :sys cmds :system cmds Execute "cmds" as system (shell) commands. Example: :system filter bar Warning: This probably makes your recipe non-portable. FILETYPE DETECTION A-A-P detects the type of a file automatically. This is used to decide what tools can be used for a certain file. To manually set the file type of an item add the "filetype" attribute. This overrules the automatic detection. Example: foo.o : foo.x {filetype = cpp} Most detection is already built in. If this is not sufficient for your work, filetype detection instructions can be used to change the way file type detection works. These instructions can either be in a file or directly in the recipe: :filetype filename :filetype suffix p pascal script .*bash$ sh The advantage of using a file is that it will also be possible to use it when running the filetype detection as a separate program. The advantage of using the instructions directly in the recipe is that you don't have to create another file. For the syntax of the file see filetype.txt. PIPE COMMANDS These commands can be used in a pipe. A pipe is a sequence of commands separated by '|', where the output of one command is the input for the next command. Example: :cat foo | :filter re.sub('this', 'that', %s) | :assign bar Unix tradition calls the output that can be redirected or piped "stdout". Reading input from a pipe is called "stdin". In the commands below [redir] indicates the possibility to redirect stdout. These three items can be used: > fname Write stdout to file "fname". If "fname" already exists this fails. >! fname Write stdout to file "fname". If "fname" already exists it is overwritten. >> fname Append stdout to file "fname". If "fname" does not exist yet it is created. The white space before the file name may be omitted. White space before the ">" and "|" is required. To avoid recognizing the ">" and "|" for redirection and pipes, put quotes around them. When a command produces text on stdout and no redirection or pipe is used, the stdout is printed to the terminal. :print [redir] [arg] ... Writes the arguments to stdout. Without arguments a line feed is produced. :cat [redir] fname ... Concatenate the files and write the result to stdout. The files are handled like text files. Note that when redirecting to a file, you cannot use the same file as input, it is truncated before it is read. :assign varname Assign stdin to a variable. :tee [redir] fname ... Write stdin to each file in the argument list and also write it to stdout. This works like a T shaped connection in a water pipe. Example: :cat file1 file2 | :tee totfile | :assign foo :filter [redir] python-expression Filter stdin using a Python expression. The Python expression is evaluated as specified in the argument, with "%s" replaced by the variable name that holds stdin as a string. The expression must result in the filtered string or something that can be converted to a string with str(). This becomes stdout. The result may be empty. Example: :print $foo | :filter re.sub('<.*?>', '', %s) > tt Note that the expression must not contain a "|" preceded by white space, it will be recognized as a pipe. Also there must be no ">" preceded by white space, it will be recognized as redirection. AAP FUNCTIONS: These functions can be used in Python code: aap_sufreplace(from, to, expr) Returns "expr" with the suffix "from" changed to "to". Example: @OBJECT = aap_sufreplace(".o", ".c", SOURCE) DEPENDENCIES: A dependency can have several targets. How this is interpreted depends on whether commands are defined. Without commands the dependency is used like each target depends on the list of source files. Thus this dependency: t1 t2 : s1 s2 s3 Can be rewritten as: t1 : s1 s2 s3 t2 : s1 s2 s3 Thus when t1 is outdated to s1, s2 or s3, this has no consequence for t2. When commands are specified, the commands are expected to produce all these targets. Example: t1 t2 : s1 s2 s3 :sys produce s1 s2 s3 > t1 :sys filter < t1 > t2 When either t1 or t2 is outdated relative to s1, s2 or s3, the commands are executed and both t1 and t2 are updated. Only when no dependency is specified for a target, the rules defined with ":rule" are checked. All the matching rules without commands are used. TRICK: For the rules with commands, only the matching one with the longest pattern is used. If there are two with an equally long pattern, this is an error. TRICK: When the source and target in a rule are equal, it is skipped. This avoids that a rule like this becomes cyclic: :rule %.jpg : path/%.jpg :copy $source $target When a target is virtual and no dependency on source files is specified, it is also executed when it's up-to-date. Example: clean: :del -rf temp/* However, when dependencies are specified, a virtual target is only updated when it is outdated relative to its sources. This is different from "make". To force updating anyway use the "force" attribute: version.txt {force} : version.txt.in :print $VERSION | :cat - $source >! $target The sources for a dependency are searched for in the directories specified with $SRCPATH. The default is ". $BDIR", which means that the sources are searched for in the current directory and in the build directory. The "srcpath" attribute overrules using $SRCPATH for an item. To avoid using $SRCPATH for a source, make the "srcpath: attribute empty: foo.o : src/foo.c {srcpath=} When setting $SRCPATH to include the value of other variables, you may want to use "$=", so that the value of the variable is not expanded right away but when $SRCPATH is used. This is especially important when appending to $SRCPATH before a ":variant' command, since it changes $BDIR. Example: SRCPATH $+= include Warning: Using the search path means that the first encountered file will be used. When old files are lying around the wrong file may be picked up. Use the full path to avoid this. When a target depends on the existence of a directory, it can be specified this way: foodir/foo : foodir {directory} :print >$target this is foo The directory will be created if it doesn't exist. The normal mode will be used (0777 with umask applied). When a different mode is required specify it with an octal value: {directory = 0700}. This must start with a zero. IMPLIED DEPENDENCIES For various file types A-A-P can scan a source file for header files it includes. This implies that when a target depends on the source file, it also depends on the header files it includes. For example, a C language source file uses #include lines to include code from header files. The object file generated from this C source needs to be rebuild when one of the header files changes. Thus the inclusing of the header file has an implied dependency. Aap defines a series of standard dependency checks. You don't need to do anything to use them. To avoid all automatic dependency checks, set the variable "autodepend" to "off": autodepend = off To avoid automatic dependencies for a specific file, set the attribute "autodepend" to "off": foo.o : foo.c {autodepend = off} You can add your own dependency checks. This is done with the ":autodepend" command. Its arguments are the file types for which the check works. A block of commands follows, which is expected to inspect $source and produce the detected dependencies in $target, which has the form of a dependency. Example: :autodepend c cpp :sys $CC $CFLAGS -MM $source > $target The build commands of the ":autodepend" are expected to generate a file that specifies the dependency: foo.o : foo.c foo.h The first item (before the colon) is ignored. The items after the colon are used as implied dependencies. The source file itself may appear, this is ignored. Thus these results have the same meaning: foo.xyz : foo.c foo.h foo.o : foo.h Comments starting with "#" are ignored. Line continuation with "\" is supported. Only the first (continued) line is read. Aap will take care of executing the dependency check when the source changes or when the command changes (e.g., the value of $CFLAGS). This can be changed with a "buildcheck" attribute before the file type arguments. :autodepend {buildcheck = $CFLAGS} c :sys $CC $CFLAGS -MM $source > $target Aap expects the dependency checker to only inspect the source file. If it recursively inspects the files the source files include, this must be indicated with a "recursive" attribute. That avoids Aap will take care of this and do much more work than is required. Example: :autodepend {recursive} c cpp :sys $CC $CFLAGS -MM $source > $target INCLUDING AND CHILDREN :child name Read recipe "name" as a child. The "refresh" attribute can be used to specify a list of locations where the recipe can be refreshed from. When "name" is in another directory, change to that directory and accept all items in it relative to that directory. Build commands defined here are also executed in this directory. Thus it works as if executing the child recipe in the directory where it is located. :include name Read recipe "name" as if it was included in the current recipe. Does not change directory and file names are considered to be relative to the current recipe, not the included recipe. The "refresh" attribute is supported like with ":child". :export varname When used at the toplevel, export variable "varname" with its current value to the recipe that uses the current recipe as a child. When used in build commands, export variable "varname" to the toplevel. Useful to pass a value to the "finally" target. When the variable was assigned a value with "$=" the argument will be evaluated now. :recipe {refresh = URL ... } Location of this recipe. The "refresh" attribute is used like with ":child": a list of locations. The first one that works is used. When aap was started with the --check argument, refresh the recipe and restart reading it. Using the "refresh" or "update" target causes this as well. The commands before ":recipe" have already been executed, thus this may cause a difference from executing the new recipe directly. The values of variables are restored to the values before executing the recipe. Refreshing the recipe is done only once per session. EXECUTING COMMANDS A dependency and a rule can have a list of commands. For these commands the following variables are available: $source The list of input files as a string. $source_list The list of input files as a Python list. $source_dl Only for use in Python commands: A list of dictionaries, each input item is one entry. $target The list of output files as a string. $target_list The list of output files as a Python list. $target_dl Only for use in Python commands: A list of dictionaries, each output item is one entry. $buildtarget The name of the target for which the commands are executed. $match For a rule: the string that matched with % Example: prog : "main file.c" :print building $target from $source Results in: building prog from "main file.c" Note that quoting of expanded $var depends on the command used. The Python lists $source_list and $target_list can be used to loop over each item. Example: $OUT : foo.txt @for item in target_list: :print $source > $item The list of dictionaires can be used to access the attributes of each item. Each dictionary has an entry "name", which is the (file) name of the item. Other entries are attributes. Example: prog : file.c {check = md5} @print sourcelist[0]["name"], sourcelist[0]["check"] Results in: file.c md5 RATIONALE In an assignment and other places a Python expression can be used in backticks. Expanding this is done before expanding $VAR items, because this allows the possibility to use the Python expression to result in the name of a variable. Example: foovaridx = 5 FOO = $SRC`foovaridx` Equal to: FOO = $SRC5 In the result of the Python expression $ characters are doubled, to avoid it being interpreted as the start of a variable reference. Otherwise Python expressions with arbitrary results would always have to be filtered explicitly. Variables can still be obtained without using the $, although rc-style expansin is not done then. Example: FOO = `glob("*.tmp")` The backticks for a Python expression are also recognized inside quotes, because this makes the rule for doubling backticks consistent. Example: FOO = "this``file" that``file Doubling the backtick to avoid it being recognized as a Python expression makes it impossible to have an empty Python expression. There appears to be no reason to use an empty Python expression. FINALLY The "finally" target is always executed after the other targets have been successfully build. Here is an example that uses the "finally" target to copy all files that need to be uploaded with one command. SOURCE = `glob("*.html")` TARGET = remote/$SOURCE CFILE = :rule remote/% : % CFILE += $source :export CFILE finally: @if CFILE: :copy $CFILE ftp://my.org/www Warning: When the ":copy" command fails, aap doesn't know the targets were not made properly and won't do it again next time. REFRESHING AND UPDATING: A convention about using the "update" and "refresh" targets makes it easy for users to know how to use a recipe. The main recipe for a project should be able to be used in three ways: 1. Without specifying a target. This should build the program in the usual way. Files with an "refresh" attribute are obtained when they are missing. 2. With the "refresh" target. This should obtain the latest version of all the files for the program, without building the program. 3. With the "update" target. This should refresh all the files for the program and then build it. It's like the previous two ways combined. Here is an example of a recipe that works this way: STATUS = status.txt SOURCE = main.c version.c INCLUDE = common.h TARGET = myprog $TARGET : $SOURCE :cat $STATUS :sys $CC $CFLAGS $LDFLAGS -o $target $source $TARGET : $STATUS # specify where to refresh the files from :attr {refresh = cvs://:pserver:anonymous@cvs.myproject.sf.net:/cvsroot/myproject} $SOURCE $INCLUDE :attr {refresh = ftp://ftp.myproject.org/pub/%file%} $STATUS When the "refresh" target is not specified in the recipe or its children, it is automatically generated. Its build commands will refresh all nodes with the "refresh" attribute, except once with a "constant" attribute set (non-empty non-zero). To do the same manually: refresh: :refresh $SOURCE $INCLUDE $STATUS NOTE: When any child recipe defines a "refresh" target no automatic refreshing is done for any of the recipes. This may not be what you expect. When there is no "update" target it is automatically generated. It will invoke the "refresh" target and the first target in the recipe. To do something similar manually: update: refresh $TARGET Although the automatically generated "update" target uses one of $TARGET, "all" or the first encountered dependency. THE REFRESH ATTRIBUTE The "refresh" attribute is used to specify a list of locations where the file can be refreshed from. The word at the start defines the method used to refresh the file: ftp from ftp server http from http (www) server scp secure copy cvs from CVS repository For a module that was already checked out the part after "cvs://" may be empty, CVS will then use the same server (CVSROOT) as when the checkout was done. other user defined These kinds of locations can be used: ftp://ftp.server.name/full/path/file http://www.server.name/path/file scp://host.name/path:path/file cvs://:METHOD:[[USER][:PASSWORD]@]HOSTNAME[:[PORT]]/path/to/repository To add a new method, define a Python function with the name "refresh_method", where "method" is the word at the start. The function will be called with two arguments: name the url with "method://" removed dict a dictionary with all attributes used of the URL, dict["name"] is the full URL node a node object. Useful items: node.name short name node.absname full name of the file node.recipe_dir directory in which node.name is valid node.attributes dictionary with attributes The function should return a non-zero number for success, zero for failure. Here is an example: :python def refresh_foo(from_name, dict, node): to_name = node.absname try: foo_the_file(from_name, to_name, dict["foo_option"]) except: return 0 return 1 VERSION CONTROL The generic form of version control commands is: :command file ... Or: :command {attr = val} ... file ... These commands can be used: :commit Update the repository for each file that was changed. No-op for file that didn't change. Do checkout/checkin when checkout is required. Don't change locking of the file. Uses "logentry" attribute when a log entry is to be done. Otherwise a standard message is used. :checkout Like refresh and additionally lock for editing when possible. :checkin Like commit and leave the file unlocked. :unlock Remove lock on file, don't change file in repository or locally. :add Add file to repository. File does exist locally. :remove Remove file from repository. File may exist locally. Additionally, there is the generic command: :verscont action {attr = val} ... file ... This calls the Version control module as specified in the "commit" attribute for "action" with the supplied arguments. What happens is specific for the VCS. The commands use the "commit" attribute to define the kind of version control system and its location. For example: {commit = cvs://:ext:$CVSUSER_AAP@cvs.a-a-p.sf.net:/cvsroot/a-a-p} :removeall [-lr] [directory] ... :removeall [-lr] {attr = val} ... [directory] ... Inspect directories and remove items that do not really exist or do not have a "commit" attribute. Careful: Only use this command when it is certain that all files that should be in the VCS do have a "commit" attribute! When no directory argument is given, the current directory is used. It is inspected recursively, unless the "-l" option was given. When directory arguments are given, each directory is inspected. Recursively when the "-r" option was given. Related to these commands are targets that are handled automatically: aap commit Normally uses the files you currently have to update the version control system. This can be used after finished making changes. aap checkout Update all files from the VCS that have a "commit" attribute. When the VCS supports locking all files will be locked. Without locking this does the same as "aap refresh". aap checkin Do ":checkin" for all files that have been checked out of the VCS. For a VCS that doesn't do file locking, this is the same as "aap commit". aap unlock Unlock all files that are locked in the VCS. Doesn't change any files in the VCS or locally. aap add Do ":add" for all files that appear in the recipe with a "commit" attribute that do not appear in the VCS. aap remove Do ":removeall": remove all files that appear in the VCS but do not exist in the recipe with a "commit" attribute or do not exist in the local directory. This works in the current directory and recursively enters all subdirectories. Careful: files with a search path may be accidentally removed! PUBLISHING Publishing means distributing a version of your project and giving it a version number. :publish This uses the "publish" attribute on each of the files. When it is missing the "commit" attribute is used. If both are missing this is an error. To publish all files with a "publish" attribute use the command: aap publish If the "publish' target is defined explicitly it will be executed. Otherwise, all files with the "publish" attribute are given to the ":publish" command. USING CVS A common way to distribute sources and working together on them is using CVS. This requires a certain way of working. The basics are explained here. For more information on CVS see http://www.cvshome.org. Obtaining a module The idea is to hide the details from a user that wants to obtain the module. This requires making a toplevel recipe that contains the instructions. Here is an example: CVSROOT = :pserver:anonymous@cvs.myproject.sf.net:/cvsroot/myproject :child mymodule/main.aap {refresh = cvs://$CVSROOT} refresh: :refresh {refresh = cvs://$CVSROOT} mymodule Executing this recipe will use the "refresh" target. The ":refresh" command takes care of checking out the whole module "mymodule". Note that this toplevel recipe cannot be obtained from CVS itself, that has a chicken-egg problem. Refreshing The child recipe "mymodule/main.aap" may be totally unaware of coming from a CVS repository. If this is the case, you can build and install with the recipe, but not refresh the files or send updates back into CVS. You need to use the toplevel recipe above to obtain the latest updates of the files. This will then update all the files in the module. However, the toplevel recipe itself will never be refreshed. To be able to refresh only some of the files of the module, the recipe must be made aware of which files are coming from CVS. This is done by using an "refresh" attribute with a URL-like specification for the CVS server: {refresh = cvs://servername/dir}. Since CVS remembers the name of the server, leaving out the server name and just using "cvs://" is sufficient. Example: SOURCE = foo.c version.c INCLUDE = common.h TARGET = myprogram :attr {refresh = cvs://} $SOURCE $INCLUDE If you now do "aap refresh" with this recipe, the files foo.c, version.c and common.h will be updated from the CVS repository. The target myprogram isn't updated, of course. Note: When none of the used recipes specifies a "refresh" target, one will be generated automatically. This will go through all the nodes used in the recipe and refresh the ones that have an "refresh" attribute. The recipe itself may also be refreshed from the CVS repository: :recipe {refresh = cvs://} When using files that include a version number in the file name, refreshing isn't needed, since these files will never change. To reduce the overhead caused by checking for changes, give these files a "constant" attribute (with a value non-empty non-zero value). Example: PATCH = patches/fix-1.034.diff {refresh = $FTPDIR} {constant} To update a whole directory, omit the "refresh" attribute from individual files and use it on the directory. Example: SOURCE = main.c version.c TARGET = myprog :attr {refresh = cvs://} . Alternatively, a specific "refresh" target may be specified. The automatic updates are not used then. You can specify the "refresh" attribute right there. refresh: :refresh {refresh = cvs://} $SOURCE If you decided to checkout only part of a module, and want to be able to get the rest later, you need to tell where in the module to file can be found. This is done by adding a "path" attribute to the cvs:// item in the refresh attribute. Example: refresh: :refresh {refresh = $CVSROOT {path = mymodule/foo}} foo.aap What will happen is that aap will checkout "mymodule/foo/foo.aap", while standing in two directories upwards. That's required for CVS to checkout the file correctly. Note: this only works as expected if the recipe is located in the directory "mymodule/foo"! If the "path" attribute is omitted, A-A-P will obtain the information from the "CVS/Repository" file. This only works when something in the same directory was already checked out from CVS. Checking in When you have made changes to your local project files and want to upload them all into the CVS repository, you can use this command: :cvsdist {file} ... {server} The list of files must include _ALL_ the files that you want to appear in CVS for the current directory and below. Files that were previously not in CVS will be added ("cvs add file") and missing files are removed ("cvs remove file"). Then all files are committed ("cvs commit file"). To be able to commit changes you made into the CVS repository, you need to specify the server name and your user name on that server. Since the user name is different for everybody, you must specify it in a recipe in your ~/.aap/startup/ directory. For example: CVSUSER_AAP = foobar The name of the variable starts with "CVSUSER" and is followed by the name of the project. That is because you might have a different user name for each project. The method to access the server also needs to be specified. For example, on SourceForge the "ext" method is used, which sends passwords over an SSH connection for security. The name used for the server then becomes: :ext:$CVSUSER_AAP@cvs.a-a-p.sf.net:/cvsroot/a-a-p You can see why this is specified in the recipe, you wouldn't want to type this for commiting each change! DISTRIBUTING YOUR PROJECT WITH CVS This is a short how-to that only explains how to distribute a set of files (and directories) using CVS. 1. Copy the files you want to distribute to a separate directory Mostly you have various files in your project for your own use that you don't want to distribute. These can be backup files and snippets of code that you want to keep for later. Since CVS imports all files it can find, best is to make a copy of the project. On Unix: cp -r projectdir tempdir Then delete all files you don't want to distribute. Be especially careful to delete "aap" directories and hidden files (starting with a dot). It's better to delete too much than too few: you can always add files later. 2. Import the project to the CVS repository Move to the newly created directory ("tempdir" in the example above). Import the whole tree into CVS with a single command. Example: cd tempdir cvs -d:ext:myname@cvs.myproject.sf.net:/cvsroot/myproject import mymodule myproject start Careful: This will create at least one new directory "mymodule", which you can't delete with CVS commands. This will create the module "mymodule" and put all the files and directories in it. If there are any problems, read the documentation available for your CVS server. 3. Checkout a copy from CVS and merge Move to a directory where you want to get your project back. Create the directory "myproject" with this example: cvs -d:ext:myname@cvs.myproject.sf.net:/cvsroot/myproject checkout mymodule You get back the files you imported in step 2, plus a bunch of "CVS" directories. These contain the administration for the cvs program. Move each of these directories back to your original project. Example: mv myproject/CVS projectdir/CVS mv myproject/include/CVS projectdir/include/CVS If you have many directories, one complicated command does them all: cd myproject find . -name CVS -exec mv {} ../projectdir/{} \; 4. Publish changes After making changes to your project and testing them, it's time to send them out. In the recipe you use for distribution, add one command that will publish all the files by updating the module in CVS server. Example: FILES = $SOURCE $INCLUDE main.aap :publishall {publish = :ext:$CVSUSER_MYPROJECT@cvs.myproject.sf.net:/cvsroot/myproject} $FILES Careful: $FILES must contain all files that you want to publish in this directory and below. If $FILES has extra files they will be added in CVS. Files missing from $FILES will be removed from CVS. However, a file that exists, is not in $FILES and does exist in CVS will cause an error. You must assign $CVSUSER_MYPROJECT your user name on the CVS server. Usually you do this in one of your personal A-A-P startup files, for example "~/.aap/startup/main.aap". USING SOURCEFORGE If you are making open source software and need to find a place to distribute it, you might consider using SourceForge. It's free and relatively stable. They provide http access for your web pages, a CVS repository and a server for downloading files. There are news groups and maillists to support communication. Read more about it at http://sf.net. Since you never know what happens with a free service, it's a good idea to keep all your precious work on a local system and update the files on SourceForge from there. If several people are updating the SourceForge site, either make sure everyone keeps copies, or make backup copies (at least weekly). You can use A-A-P recipes to upload your files to the SourceForge servers. To avoid having to type passwords each time, use an ssh client and put your public keys in your home directory (for the web pages) or on your account page (for the CVS server). Read the SourceForge documentation for how to do this. For uploading web pages you can use a recipe like this: FILES = index.html download.html news.html logo.gif all : remote_file/$FILES {virtual} COPY = :rule remote_file/% : % COPY += $source :export COPY finally: @if COPY: :copy $COPY scp://myname@myproject.sf.net:/home/groups/m/my/myproject/htdocs To avoid opening an scp connection for each file, the outdated files are collected in $COPY and all copied at once. One disadvantage of this is that it only works for files in one directory. For sourceforge, set environment variable CVS_RSH to "ssh". Otherwise you won't be able to login. Do "touch ~/.cvspass" to be able to use "cvs login" Upload your ssh keys to your account to avoid having to type your password each time.