For Yacas developers


A crash course in Yacas maintenance for developers

This document intends to give a concise description of the way Yacas is maintained. There are a few parts to maintenance to take into account:

http://www.xs4all.nl/~apinkus/backups/


The autoconf/automake system

The important thing to remember with the autoconf/automake system is that there are files enumerating the files and directories that are part of the distribution. The Makefile.am files contain the lists of files that are part of the distribution, and are in the respective directory. If a file is added by a developer, but it is not added to the list in the file Makefile.am, it will not be added to the tar ball which is uploaded to the back up repository. It has the nice side effect that you can have local files which don't automatically get added to the distribution.

Adding a directory requires adding the directory name to various places also, in configure scripts etcerera.


Maintaining Yacas through a cvs repository

CVS provides an efficient way for developers to work together, automatically merging changes various developers make, and at the same tile is a back up system (uploading your changes to another computer from which you can easily obtain it at a later time). After a little effort setting it up it becomes very easy to use. This section describes the few commands needed for keeping your version and the version in the Yacas repository up to date.


How does cvs work?

CVS has a copy of the files in the repository somewhere in a directory on some system, possibly your computer. Then there is such a thing as a cvs server which you can talk to to synchronize your version of the source code with the version on the server.

CVS uses a diff-like scheme for merging differences: it looks at two text files, determines the different lines, and merges accordingly. It discovers the changes you made by looking at the version you checked out last and the version you have now, to discover which lines changed (it maintains an automatic version number for each file).

If the version of a file on your system and the version in the cvs repository has a line that has been changed by both you and some one else, the cvs repository will obviously not know what to do with that, and it will signal a 'collision' which you will have to solve by hand (don't worry, this rarely happens). More on that later.

In commands to be described in this document are in short:


Checking out an initial version of Yacas

There are two ways to check out a version of Yacas: as anonymous user and as maintainer. Anonymous users don't need to log in, but also have no right to commit changes. Maintainers first need to get an account (at sourceforge), and their account needs to be enabled so they are allowed by the maintainer to make changes. A maintainer needs to log in with every command. To be able to log in, you need ssh1 installed (ssh2 will not work). You can find this at http://www.ssh.org/download.html.

To check out Yacas as anonymous user, type:

cvs -d:pserver:anonymous@cvs.yacas.
  sourceforge.net:/cvsroot/yacas login
cvs -z3 -d:pserver:anonymous@cvs.yacas.
  sourceforge.net:/cvsroot/yacas co yacas

To check out as a maintainer, type:

export CVS_RSH=ssh1

This will tell CVS to use ssh1 for communication. Then, in order to get yacas, type

cvs -d:ext:loginname@cvs.yacas.sourceforge.
  net:/cvsroot/yacas co yacas

where loginname is your name on the sourceforge system. This then creates a directory yacas/ with the full most recent distribution. You need to enter your password there, but other than that, that's it!

Those lines typed above are long and intimidating, but it is also the last time you need to type them. From now on, if you want to do anything with cvs, just go into the yacas/ directory you just checked out, and type the cvs command without the -d... flag. This flag just tells cvs where to find the repository. But future cvs commands will know where to find them, which is why you don't need that flag.


Use case scenario 1 : getting the latest version of Yacas

You haven't looked at Yacas for a while (shame on you!) and want to check out the latest version. Just type

cvs update -d

on the command line in the yacas directory, and that should essentially download the latest version for you in that directory (just the changes). The -d option here states that you are also interested in new directories that were added to the repository. Oddly enough, cvs will only get you changed and added files, not added directories, by default.

A command

cvs -q update -d

will print messages only about changed files.


Use case scenario 2 : you made changes to Yacas

You got the latest version, but saw this huge, glaring omission in Yacas, and start hacking away to add it yourself. After a while, after playing with the code you wrote, and if you think you are finished with it, you decide you like to add it to the cvs repository. First, you added a few files, which you need to add:

cvs add [list of file names of ascii text files]

which adds ascii text files. If you added binary files (GIF images in the documentation directory, or something like that), you can add it with

cvs add -kb [list of file names of binary files]

Note that, when adding files to the CVS, you should normally also add them to the distribution list. This is done by adding the file name to the EXTRA_DIST variable in the file Makefile.am in the directory where you adding the file.

In case files need to be removed, there are two options:

There seems to be no easy way to rename or move files; you would have to remove them at their old location and add them at a new location.

Now, when finished with that, you might want to 'commit' all changes with

cvs commit

If the commit succeeds, an email is sent out to the maintainers, who can then scan the diff files for changes, to see if they agree with the changes, and perhaps fix mistakes made (if any).

If there is a collision, the commit fails (it will tell you so). In case of a collision, you need to invoke cvs update twice. The cvs update outputs a list of file names with a character in front of them. The important ones are the files with a 'C' before them. They have a collision. You can go into the file, and see the collision, which the cvs system conveniently marks as:

<<<<<<
old version
===========
new version
>>>>>>

You can edit the file by merging the two versions by hand. This happens very rarely, but it can happen. Use cvs commit afterwards to commit.

The commit and update commands can be performed in specific directories, and on specific files, if necessary, by stating them on the command line. Or you can go into a sub directory and do a cvs commit or cvs update there, if you are confident that is the only place that changed or whose changes you are interested in.

That is basically it, a quick crash course cvs. It is actually very convenient in that usually all that is needed is a cvs commit to fix small bugs. You type that in, and your version gets merged with the changes others made, and they get your changes, and you backed up your changes at the same time (all with that little command!).

You can find more information about cvs at http://cvsbook.red-bean.com/ .


Preparing and maintaining Yacas documentation


Introduction

Yacas documentation in HTML and PS/PDF formats is generated by Yacas scripts from Yacas source files. However, it is cumbersome to write those source files in the Yacas language. The script txt2yacasdoc.pl makes it possible to create and maintain the documentation in an easy-to-read form.

The "source" form of all documentation is maintained in a special plain text format. The format is such that it is clearly readable without any processing and is easy to edit. To compile the documents, the system processes the plain text docs with a script to prepare Yacas-language files and then runs the Yacas scripts to produce the final documentation.

The source text must be formatted in a certain fashion to delimit sections, code examples, and so on, but the format is easy enough to enter in a plain text editor. Text is marked up mostly by TAB characters, spaces, and asterisks "*" at the beginning of a line. The script txt2yacasdoc.pl converts this markup into Yacas code.

There is also a utility book2txt that attempts to convert existing documentation in the Yacas format into the plain text form suitable for txt2yacasdoc.pl. After conversion, the plain text docs normally require further editing.


Organization of the Yacas documentation

All documentation source files are kept in the subdirectory manualmaker/in. During compilation, Yacas language files as well as HTML, LaTeX and PS/PDF files are automatically generated in the manualmaker subdirectory. Contributors should only need to edit files in manualmaker/in.

Currently, documentation consists of five "books" (introductory tutorial, programming tutorial, miscellaneous essays, user's reference manual, and programmer's reference manual). The first three books are meant as stand-alone texts, while the last two are "reference" books and are meant to be used together. The Yacas Help() command will show a reference article from either of the two reference books.

Books may be divided into "chapters", "sections" and "subsections". Reference documentation contains each command description in a separate "section" which is marked by a special *CMD label (see below).

Stand-alone books are free-form, but reference books must be written with a certain template that allows online hyperlinking. The file manualmaker/in/dummies is an example template for a reference manual section.

At the beginning of each book must be a short book description (labeled *BLURB) which does not appear in printed documentation but only in the HTML top-level book index.

At the beginning of each chapter there may be a "chapter introduction" labeled *INTRO which is also a short description of the contents of that chapter. It may be one paragraph only. Note that the chapter introduction is only really useful for reference manual because it is the text that appears at the very top of a reference manual section. In the HTML version of reference docs, each chapter contains a list of all functions described in it, and this list goes right after the first paragraph of "chapter introduction". For the printed (PS/PDF) documentation the "introduction" is indistinguishable from any other text.


Using the script txt2yacasdoc.pl

The script txt2yacasdoc.pl is used to transform plain text markup into the Yacas language. The script acts as a stream filter:

perl txt2yacasdoc.pl < file.txt \
  > file.chapt

In this example, file.txt contains some formatted plain text (source text) and the resulting file file.chapt will be produced in Yacas-language documentation format.

There is a single option for txt2yacasdoc:

perl txt2yacasdoc.pl -debug < file.txt \
  > file.chapt
This option is to be used for debugging, i.e. when the resulting file does not compile in Yacas. The effect of this option is to introduce many more breaks between text strings in the generated file, so that the Text() function is called more often. It is then easier to locate the source of the problem in the Yacas-language file (Yacas will tell you the last line in the Yacas-language file at which a syntax error occurred). This option is largely obsolete because the Text() function is called frequently enough by default.


Formatting of source text files

Formatting of source text files uses TAB symbols; if your editor does not support them and converts them to spaces, you should convert the results back to contain real TAB symbols using the standard Unix unexpand utility or a custom perl script.

You may want to examine the source of this file (manualmaker/ in/ YacasDocs.chapt.txt) to see how various features of the markup are used. Currently the following markup is implemented:

*	Item
*	Another item

Note that item text continues until the next itemized line is given or until end of paragraph.

Nesting of enumerated or itemized environments is not yet supported, except for a fringe case of nesting just one itemized list at the very end of an enumerated list or vice versa.

Enumerated environment is currently only implemented in LaTeX docs; HTML docs render them as itemized.

Mathematical expressions must be valid Yacas expressions, with balanced parentheses, no hanging periods and so on, or else the Yacas script that formats the docs will fail! (This limits the scope of mathematical formulae but is hopefully not critical.) Currently, when creating online HTML documentation, mathematics is kept in Yacas notation and set in boldface font. (This may change in the future.) Of course, LaTeX typesets printed documentation with correct mathematical symbols.

Another feature of the LaTeX exporter is that it will try to show all functions and infix operators according to their mathematical meaning, and if no such meaning is defined in Yacas, then it will show them exactly as they are written in Yacas. For infix operators to work, they have to be declared in the standard library, or else an error will occur when processing the manual. For example, Yacas operators = and == are both represented by and equals sign "=", the operator := becomes "identically equal" ( a:=b), and the cosmetic operators <> and <=> become a<>b and a<=>b. But you cannot use an undefined infix operator such as ":=*".

*INCLUDE ../essays/howto.chapt

Note that the included document must be a Yacas-language file. (This will become the IncludeFile() call in the Yacas document -- an alias to Load().)


Formatting text for the reference manual

The formatting explained in the previous section is enough to create most of the user guide and tutorial documentation. The script txt2yacasdoc.pl implements some additional markup features to help create reference manual sections.

The additional elements all start with an asterisk "*" in the first position on a line, followed by an uppercase keyword. A typical reference manual subsection documenting a certain function may look like this in plain text:

*CMD PrintList --- print list with padding
*STDLIB

*CALL
{PrintList}(list)
{PrintList}(list, padding);

*PARMS
{list} -- a list to be printed
{padding} -- (optional) a string

*DESC
Prints {list} and inserts the {padding} ...

*E.G.

	In> PrintList({a,b,{c, d}}, " .. ")
	Out> " a ..  b .. { c ..  d}";

*SEE Write, WriteString
Compare this with the reference manual section on the function PrintList to see how this plain text markup is rendered in the finished documentation.

Notes:

In a subsection there may be either one function documented or several at once: for example, it may make sense to document Sin, Cos and Tan together. In this case, all function names should be simply listed in the *CMD header, for example:

*CMD Sin, Cos, Tan --- Trigonometric ...

In addition, there is a tag *INTRO to denote a "reference chapter introduction" corresponding to the ChapterIntro() function, *BLURB for book introduction (only enters the HTML book index but not LaTeX docs), *A which creates an HTML link to a manual section corresponding to AddBody(HtmlAnchor()...), *AA which creates AddAnchor(), and *HEAD which creates a small heading (Topical()). For instance,
*PARMS
results in the same text as
*HEAD Parameters:
Usage of the *A, *AA, *HEAD tags is for compatibility only and should not be necessary. The *CMD tag generates all necessary HTML anchors.

The *INTRO and *BLURB tags only work for one paragraph. There must be no empty line between *INTRO/*BLURB and that paragraph. Also, there must be no empty line between the "blurb" and the book title (for technical reasons). There must be one and only one "blurb" paragraph in a "book" and no more than one "chapter intro" paragraph per chapter.

This markup should be sufficient for creating reference documentation in plain text.

If a manual book does not compile after you made some changes, examine the Yacas-language files (*.chapt, *.book), find the line at which an error occurred, and try to see what happened. Most probable cause of error is unbalanced brackets, parentheses, or dollar signs; or expressions which should not be split between lines were accidentally split after reformatting the text. Mathematical expressions cannot contain a trailing period (e.g. you cannot say $x+a.$ -- the period should go outside dollar signs.


book2txt: Conversion of existing documentation to plain text

Currently, most but not all of the Yacas documentation markup functionality is implemented in the simple plaintext filter; also, documentation includes some extra HTML files. However, almost all of the reasonable markup needed to write documentation is present. Therefore it is possible to maintain most of the documentation in the plain text format described above. To convert existing Yacas documentation back to the plain text format, a script book2txt.ys/book2txt.sh can be used.

(Note: as of version 1.0.49, all Yacas documentation is converted to plaintext format. This section is left for reference only.)

By using a command such as
book2txt.sh file.chapt
one can create a source text file file.chapt.txt corresponding to the Yacas documentation file file.chapt. For example:

12:51pm scriabin> book2txt.sh intro.book
[editvi.ys] [gnuplot.ys] 
True;
Out> True;
Quitting...
File 'intro.book.txt' was created.
12:51pm scriabin>

In the above example, the shell commands in book2txt.sh executed the following Yacas commands,
Use("book2txt.ys");
ToFile("file.chapt.txt")
  Load("file.chapt");
This requires that the Yacas script book2txt.ys be available in the current directory. The shell script book2txt.sh assumes that book2txt.ys is stored in the same directory as book2txt.sh.

Of course, it is possible that some features of Yacas documentation were not implemented in the script and in that case the resulting file must be edited by hand. But the purpose of the book2txt script is exactly this: to make a plain text source file to be edited and maintained.

Several files can be converted at once, for example:
book2txt.sh f1.chapt f2.chapt file3.book
Each file is processed by an independent Yacas session. Any errors of processing are printed on the screen.


book2TeX: preparing typeset documentation

The script book2TeX.sh prepares a TeX file out of a Yacas-language documentation book. Usage is similar to book2txt.sh, except that only one file is processed at a time. For example:
book2TeX.sh intro.book intro.book.tex
will create a TeX-formatted version of the introductory tutorial. It can be processed with standard tools, for example
latex intro.book.tex
latex intro.book.tex
dvips -o intro.book.ps intro.book dvi
will prepare a Postscript version, while
latex intro.book.tex
pdflatex intro.book.tex
will prepare a PDF version.

The shell commands in book2txt.sh execute the following Yacas commands:
Use("book2TeX.ys");
ToFile("file.chapt.tex")
  Load("file.chapt");
This requires that the Yacas script book2TeX.ys be available in the current directory. The shell script book2TeX.sh assumes that book2TeX.ys is stored in the same directory as book2TeX.sh.

Not all features of Yacas documentation are compatible with TeX typesetting. To prevent errors, documentation should avoid certain things. In general, it is a good idea to check the typeset appearance of documentation, since it helps detect errors.

For example, the symbols %, { }, < >, #, \, _ and & are special to TeX. They should not normally be used in plain text; it is okay to use them in "typerwriter" text or code samples -- but not in section or chapter heads, because it makes it difficult to export to TeX correctly. TeX commands may be entered but will not be correctly rendered in HTML online documentation. Sometimes fixed-font text will hang over the right edge; a workaround is to break the fixed-font text into shorter fragments or to rephrase the text.

Another concern is that code examples are typeset in a fixed-width font and may not fit into the width of the page and will be hanging over it. To avoid this, the code in the code examples should not be longer than about 44 characters.

The current implementation uses a "report" style which allows chapters, sections, subsections and includes an automatically generated table of contents. ( LaTeX is automatically run twice to produce the TOC when executing make texdocs.)

Some complicated mathematical expressions may not correctly render in TeX. This is because Yacas uses its internal function TeXForm() to transform Yacas expressions to TeX. Mathematical expressions are entered in the plain text documentation source using Yacas syntax, then transformed to a special non-evaluating call TeXMath() in the Yacas-language documentation, which formats into HTML using a Write() call or into TeX using a TeXForm() call, as necessary. Testing should be performed on documentation before releasing it. The most stringent limitation is that the expression between dollar signs should evaluate in Yacas (perhaps to itself) and not cause syntax errors. In case of doubt, try whether the expression evaluates without errors and then try to use TeXForm on that expression and see if that evaluates without errors.