Documenting new rules

Why document your rules?

There are several ways in which you can improve a newly created rule, making it both more capable and easier to use:

You can describe the types of values the rule accepts as input and produces as output. Documenting your rule's "user interface" in this way not only enables kiwi's documentation system to generate a detailed syntax description, it also allows kiwi to determine whether the values supplied by the user are appropriate for your rule and generate helpful error messages when they are not.
You can include a full user manual within your rule's definition. This manual may be a one-line summary of your rule's purpose, or a detailed multi-section guide to every aspect of your rule's operation and use, complete with example code, technical notes, known limitations, and links to other resources.
You can indicate which category (or categories) the rule should appear under when HTML rule documentation is generated.
You can indicate which kind(s) of objects (kiwi values, pacu text frames, saki template objects, etc.) can be manipulated by that rule. Kiwi's documentation system automatically includes this information in the user documentation for quick reference, while kiwi's auto-apply rule can use this information to apply rules to all such objects automatically (e.g. to check for text overflows on all tagged text frames in a pacu template).

Describing values' types

A powerful feature of kiwi's rule system is its ability to coerce and check rules' input and output values to ensure those values are both appropriate and ready for use. When defining a composite rule, you can use type rules to describe the required type and constraints of each value received by your rule, as well as any values that your rule produces.

While describing value types is completely optional, it has several advantages:

greater control over what values are acceptable
more precise error messages when an unacceptable value is given
more helpful auto-generated documentation.

For example, try applying the following beware rule command to see what happens when an undesirable value is given:

R> beware ("")
┌──────────────────────────────────────────────────────────┐
 11:06:27.05 WARNING: Beware the !
└──────────────────────────────────────────────────────────┘

Here the beware rule successfully logs a message, but because the text value supplied by the rule command was empty the message itself does not make sense.

Let's try another, this time to see what happens when the value is omitted completely:

R> beware
┌──────────────────────────────────────────────────────────┐
 10:50:05.24 ERROR:

The 'log' rule's 'message' value must be expanded text, but
it received the following text:

"Beware the "{$scary thing}"!"

Can't expand the following text:

"Beware the "{$scary thing}"!"

Can't coerce the following tag value to text:

{$scary thing}

The following tag produced an empty tag, but text was
required:

{$scary thing}

Can't coerce the following empty tag value to text:

{}
└──────────────────────────────────────────────────────────┘

This time, the beware rule fails. Because the value was missing, kiwi automatically substituted an empty tag as placeholder. This empty tag was then stored under the name scary thing as normal. However, once the {$scary thing} tag retrieved this value, kiwi was unable to expand the resulting "Beware the "{}"!" value as empty tags cannot be coerced to text. However, while the user's bad input was successfully detected and prevented from causing any further problems, the generated error message offers the user little help in identifying her original mistake: her beware rule command needs to supply an additional text value.

[TO DO: does it help or hinder to introduce dedicated jargon here? Also, what term is more self-explanatory, 'interface', 'interface description' or 'signature'?]

Let's require the beware rule's scary thing value to be text. To do this, we need to include a text type rule in its name (i.e. its interface description). beware (scary thing). First, space out the value names list to make it easier to read:

define rule (
  beware (
    scary thing
  ), 
  ...
)

Next, replace the name scary thing with a two-item list. The first item is the value's name (scary thing) and the second item is a type rule that describes the value's required type (in this case, text):

R> ^m
define rule (
  beware (
    (scary thing, text)
  ), 
  log ("Beware the "{$scary thing}"!", warning)
)
^m

Once that's done, test the rule to make sure it works as intended:

R> beware (Dog)
┌──────────────────────────────────────────────────────────┐
 13:01:44.92 WARNING: Beware the Dog!
└──────────────────────────────────────────────────────────┘
R> beware ("")
┌──────────────────────────────────────────────────────────┐
 13:02:23.29 WARNING: Beware the !
└──────────────────────────────────────────────────────────┘
R> beware
┌──────────────────────────────────────────────────────────┐
 13:02:51.12 ERROR:

The 'beware' rule's 'scary thing' value must be expanded
text, but it received the following empty tag:

{}

Can't coerce the following empty tag value to text:

{}
└──────────────────────────────────────────────────────────┘
R> ^Z beware
────────────────────────────────────────────────────────────
beware (scary thing)

  scary thing : text

────────────────────────────────────────────────────────────

As you can see, the rule still produces the intended output when a suitable text value is given, while supplying the wrong type of value now provides a much more helpful error message. The auto-generated rule documentation is also improved so users can see at a glance that the beware rule requires its scary thing value to be text.

The only use case that still produces an unsatisfactory result is when the given text is empty. However, we can address that too by using a type rule that both coerces the given value to text and ensures its content is appropriate, or rejects it if not. There are a couple of ways you can do this: one option is to supply the text type rule with a text pattern to match; another is to use the non-empty text rule included in kiwi's standard library. For this exercise, let's just use the non-empty text rule:

R> ^m
define rule (
  beware (
    (scary thing, non-empty text)
  ), 
  log ("Beware the "{$scary thing}"!", warning)
)
^m

Now when the beware rule receives empty text, it rejects it too:

R> beware ("")
┌──────────────────────────────────────────────────────────┐
 13:17:32.48 ERROR:

The 'beware' rule's 'scary thing' value must be expanded 
non-empty text, but it received the following text:

...
└──────────────────────────────────────────────────────────┘

The rule's documentation also provides an improved description of the value required:

R> ^Z beware
────────────────────────────────────────────────────────────
beware (scary thing)

  scary thing : non-empty text

────────────────────────────────────────────────────────────

While the beware rule is more robust, there are still ways in which the user might "fool" it, whether accidentally or deliberately, into displaying nonsensical messages. For example, while empty text is rejected, a text value containing nothing but space/tab/return characters could still slip through. (In this case, using a pattern-checking text ("[^\s]") type rule would ensure the given text contains at least one non-white space character.) When developing and testing rules, it's a good idea to consider all the things that might go wrong, and decide whether or not it is worth building additional protection into the rule itself, or merely adding a note or warning to the rule's documentation so that users understand its limitations and know to take care.

Type rules can also be used to make a value optional, or supply a default value instead. For instance, to use Spider as the default scary thing value:

R> ^m
define rule (
  beware (
    (scary thing, default ("Spider", non-empty text))
  ), 
  log ("Beware the "{$scary thing}"!", warning)
)
^m
R> beware (Raccoon)
┌──────────────────────────────────────────────────────────┐
 13:02:35.09 WARNING: Beware the Raccoon!
└──────────────────────────────────────────────────────────┘
R> beware
┌──────────────────────────────────────────────────────────┐
 13:02:35.09 WARNING: Beware the Spider!
└──────────────────────────────────────────────────────────┘

Or to make the scary thing value completely optional:

R> ^m
define rule (
  beware (
    (scary thing, optional (non-empty text))
  ), 
  if value ({$scary thing},
    log ("Beware the "{$scary thing}"!", warning),
    log (Nothing scares me!)
  )
)
^m
R> beware (Boojum)
┌──────────────────────────────────────────────────────────┐
 13:36:57.46 WARNING: Beware the Boojum!
└──────────────────────────────────────────────────────────┘
R> beware
┌──────────────────────────────────────────────────────────┐
 13:37:19.72 INFO: Nothing scares me!
└──────────────────────────────────────────────────────────┘

If your rule requires a particular input value or produces a new output value, you can describe these requirements using the reserved names input and output. For example, here is our earlier uc rule again:

R> define rule (uc, case (upper))

This rule is fully functional, but neither its error reporting nor documentation are very user friendly:

R> ^^ {}
R> uc
┌──────────────────────────────────────────────────────────┐
 14:22:16.89 ERROR:

The 'case' rule's input value must be expanded text, but it
received the following empty tag:

{}

Can't coerce the following empty tag value to text:

{}
└──────────────────────────────────────────────────────────┘
R> ^Z uc
────────────────────────────────────────────────────────────
uc

────────────────────────────────────────────────────────────

Adding input and output names to the uc rule's value descriptions slightly improves the documentation by indicating that the rule takes an input value and produces an output value:

R> define rule (uc (input, output), case (upper))
R> ^Z uc
────────────────────────────────────────────────────────────
uc

  input : anything [optional]
  output : anything [optional]

────────────────────────────────────────────────────────────

We can also add type rules to input and output descriptions:

R> ^m
define rule (
  uc (
    (input, text),
    (output, text)
  ), 
  case (upper)
)
^m

This provides significantly better documentation that now states exactly what type of value is required as input and what type of value will be produced as output:

R> ^Z uc
────────────────────────────────────────────────────────────
uc

  input : text
  output : text

────────────────────────────────────────────────────────────

Error messages are also much improved, making it clear to the user exactly where they went wrong:

R> ^^ (a, b, c, d, e)
R> uc
┌──────────────────────────────────────────────────────────┐
 14:31:59.53 ERROR:

The 'uc' rule's input value must be expanded text, but it
received the following list:

(a, b, c, d, e)

Can't coerce the following list value to text:

(a, b, c, d, e)
└──────────────────────────────────────────────────────────┘

Adding a user manual

When developing rules for long-term use by multiple users, it is well worth including full documentation on how to use those rules. Along with the required name and commands values, the define rule rule accepts up to three documentation-related values - description, targets, and categories:

define rule

Summary

Define a composite rule handler.

Syntax

define rule (name, commands, description, targets, categories)

name : single rule — the new rule handler's name, optionally followed by a list of value definitions
commands : rule — the rule command(s) to perform
description : text [default: ""] — description of this rule, for documentation purposes
targets : list of text [default: ()] — the object(s) this rule applies to, for documentation purposes
categories : list of text [default: ()] — categorization(s) for this rule, for documentation purposes

…

Let's start with a simple case: adding a one-line summary to the uc rule:

R> ^m
define rule (
  uc (
    (input, text),
    (output, text)
  ),
  case (upper),
  "Convert the input text to all-upper case."
)
^m

As you can see, the description value consists of plain text. When the rule's short documentation is viewed in iki, this summary appears below the rule's syntax and value descriptions:

R> ^z uc
────────────────────────────────────────────────────────────
uc

  input : text
  output : text

Convert the input text to all-upper case.

────────────────────────────────────────────────────────────

If the rule's documentation is exported to HTML, the same summary will appear directly beneath the rule's name on both the rule's own manual page and on the category page(s) under which the rule is listed.

Be aware that kiwi's documentation system treats the description as Markdown-formatted text. This allows you to indicate which parts of the text should be appear as headings, lists, kiwi code, and so on.

See the Markdown reference appendix for a quick introduction to formatting rule documentation.

In addition, when generating HTML documentation, kiwi automatically separates the description text into two parts: everything up to the first heading (written as ## ... ##) is treated as the summary text which should appear at the top of the manual page, while that heading and everything beneath it is inserted into the page below the Syntax section. This ensures that the Summary and Syntax sections always appear at the top of the page.

[TO DO: describe how following rule is documented; also mention arg descriptions]

define rule (
  beware (

    (scary thing, optional (non-empty text), 
        "whatever scares you the most")
  ), 
  if value ({$scary thing @ is empty tag},
    log (Nothing scares me!),
    log ("Beware the "{$scary thing}"!", warning)
  ),
  "Log a warning about something scary.

  ## Examples ##

  To log a warning about the Ides of March:

    R> beware (""Ides of March"")
    ┌────────────────────────────────────────────────┐
     14:41:06.33 WARNING: Beware the Ides of March!
    └────────────────────────────────────────────────┘
  ",
  (),
  user interaction
)

When writing documentation text, take extra care to escape all double quote characters within the text by typing them twice. For example, note how the Examples section above uses R> beware (""Ides of March""), not R> beware ("Ides of March"). If you forget to escape double quote characters correctly, the double quote marks won't appear when the rule's documentation is displayed, or kiwi may report a syntax error when it tries to parse the code between the quote marks.

For best results when authoring description text for your rules:

Limit the summary to a single paragraph that states in one or two sentences what your rule does.
Immediately beneath the summary text, provide a short Examples section that shows one or two simple examples of your rule in use. If the rule can be run directly in iki, you can often run a simple test there then copy and paste those lines directly into your Examples section (remember to indent each line with an extra tab so that it is formatted as a code block, not as a plain paragraph).
If you need to provide a more detailed description of your rule's purpose, behavior, and/or usage, add Description and/or Usage sections as appropriate.
If users will benefit from addition examples of more advanced use, these may be included in a More examples section at the end of the manual text.

Adding target and category information

The last two optional values accepted by define rule are a list of target names and a list of rule categories.

Target names are most useful when developing rules that will be applied automatically by kiwi's auto-apply rule. (Currently, only pacu template tags support this feature.) However, it is also useful to include one or more target names where appropriate as these will be displayed in the Target section of the rule's HTML documentation for reference.

Let's add targets and categories values to our uc rule. Since the uc rule simply uses kiwi's case rule to modify its input value, we'll just reuse the case rule's own target (text value) and category (manipulate text):

define rule (
  uc (
    (input, text),
    (output, text)
  ),
  case (upper),
  "Convert the input text to all-upper case."
  text value,
  manipulate text
)

When deciding how best to categorize your own rule, consider using category names already employed in the kiwi library's own documentation. If you don't specify any categories, kiwi's rule system will automatically list the rule under the "uncategorized" section. If you don't want the rule to be shown in the rule documentation at all, include the hidden category name in your categories list. This is handy for hiding rules intended for use by other rules only, as well as rules you are currently developing that are not yet ready for anyone else to use.)

As an exercise, add appropriate targets and categories values to the beware rule. (Tip: as the beware rule isn't used to manipulate objects, just use an empty tag or empty list for its targets value.)