★ wanayoo — archive 1999 http://www.clari.net/tech/webformat.html

ClariNet ClariNews Article Format

ClariNet's ClariNews (formatted e.News) articles have our full USENET headers and a body of HTML. One extra feature in the headers is the addition of an HTML comment surrounding most of the header data. These headers look like this: X-HTML: <HTML> This causes most of the header data to be ignored by programs that only understand HTML data and not USENET articles. The processing of the headers by various news systems may move one or more headers outside these comment lines, however this is typically one line out of a couple dozen.

The body of each article is HTML. It contains an <HTML> tag, a <HEAD> section (including a <TITLE>) and a <BODY> section. In the BODY are embedded tags to allow various bits of data to be parsed and/or modified. These are useful to have in addition to the article headers because the headers can be stripped off to turn the articles into pure HTML. These extra tags are simply ignored by a web browser without any ill effects.

All the ClariNet data tags have the form <CLARI-ITEM TYPE> along with the corresponding closing tag of </CLARI-ITEM>. The TYPE varies depending on what we're marking (see below for a full list).

This tag is used in two different ways: a single-line form and a multi-line form. The single-line form looks like this:

<H1><CLARI-ITEM HEADLINE>Mars Teeming With Life</CLARI-ITEM></H1> There is never more than one CLARI-ITEM per line, but other HTML tags and non-tagged data may be on the same line. The single-line form never contains other CLARI-ITEMs and never includes the TYPE in the closing tag.

The multi-line form looks like this:

<CLARI-ITEM CAPTION> Little Blue Men arrive from Mars and demand that we give them females to revive their dying race. </CLARI-ITEM CAPTION> In the multi-line form the CLARI-ITEM tag is the last thing on the line prior to the data, though not necessarily the only thing. The closing tag is always the first thing on the line following the data, and is always marked with its TYPE. Other CLARI-ITEMs may be contained within a multi-line item (such as a CAPTION being contained by a PHOTO-TABLE).

CLARI-ITEM tags respect the nested structure of HTML -- you'll never see a block element that is opened in a tag but not closed or closed that wasn't opened.

The following CLARI-ITEM types are currently defined:

ANCHOR
Used in some specialty stories to mark anchor items of significance.
BACK-LINK
(NewsDrill only) The HTML for the back reference inside a LINKS-INDIRECT block.
BOUNDLINKS
(NewsDrill only) Links that are bound to specific words within the story. (Multi-line)
CAPTION
The photo's caption text (part of the PHOTO-TABLE). (Multi-line)
COPYRIGHT
The story's Copyright.
DATE
Taken directly from the Date header.
FOOTNOTE
Foot-note text. (Multi-line)
FROM
Just the comment portion of the From header.
HEADER
A block of header information for the story. (Multi-line)
HEADLINE
Typically the same as the <TITLE>, but may have some data stripped (such as a trailing date).
IMAGESTORY
We now pack our comics and cartoons in this tag (these stories don't have a STORY tag). (Multi-line)
LINK
(NewsDrill only) Each link of a link/description pair will soon be marked with this tag. (Multi-line)
LINK-DESC
(NewsDrill only) Each description of a link/description pair will soon be marked with this tag. (Multi-line)
LINK-STATUS
(NewsDrill only) If a particular link has special status (such as "[Reviewed]"), it will be noted with this tagged item.
LINKS-DIVIDER
(NewsDrill only) Bits of HTML that separate links or trail the last link in the BOUNDLINKS and STORYLINKS sections. (Multi-line)
LINKS-INDIRECT
(NewsDrill only) A tagged block of link information that one of the links in the article references. (Multi-line)
NEWSGROUPS
Newsgroup information for this story in tabular form. (Multi-line)
KEYWORDS
A list of keywords for this story separated by commas.
PHOTO-TABLE
A table used to display an in-line photo. (Multi-line)
SLUGWORD
The story's slug word.
STORY
The text of the story. (Multi-line)
STORYLINKS
(NewsDrill only) Links that releate to the story, but are not bound to specific words within it. (Multi-line)
TAGLINE
(Coming soon) The bottom-of-the-story links.
TRAILER
The bottom-most HTML, usually used to display some handy links. (Multi-line)
TYPE
The story's type (currently "story" or "feature").
XINFO
A block of "extra info" about the article, such as the NEWSGROUPS clari-info. Since this info is typically shown by a newsreader, this data is commented out of the news articles, but might get uncommented by the web extraction software. (Multi-line)

Not all of these items are present in every story. Just ignore any unknown tags (the normal thing to do when parsing HTML). Also remember that it is easy to tell programatically if you're dealing with a multi-line tag because the opening tag will be the last item prior to a newline.

There are also some special sub-tags for certain tags. SomeIMG tags have CLARI-XFN after the "SRC", some Anchor tags have CLARI-XGN after the "HREF", and various tags (such as HTML and BODY) can have a CLARI-STYLE tag. The purpose of these tags is to allow the news-hosted version to be easily converted into a web-hosted version. For instance, all external references to newsgroups and other articles (including images) are in news-relative links. If the articles are translated into a web-based format using our extraction technique, the conversion is done using these tags.

Note that if you are parsing extracted files, these tags will have already been processed, and in the case of the CLARI-XFN and CLARI-XGN tags, these values will be swapped with the value of the preceding SRC or HREF sub-tag. When this happens the previous news reference is placed in the sub-tag CLARI-MID (for message IDs) and CLARI-GID (for group references).

Here's an example article. It contains one in-line photo (there can be 0, 1 at the top, or 1 at the top and 1 at the bottom) and 3 icon images.

You can view how it looks when it is formatted, if you like.