SWI-Prolog -- Parsing Primitives

Creates a new parser. A parser can be used one or multiple times for parsing documents or parts thereof. It may be bound to a DTD or the DTD may be left implicit, in which case it is created from the document prologue or parsing is performed without a DTD. Options:

dtd(?DTD): If specified with an initialised DTD, this DTD is used for parsing the document, regardless of the document prologue. If specified using as a variable, a reference to the created DTD is returned. This DTD may be created from the document prologue or build implicitely from the document's content.

free_sgml_parser(+Parser)

Destroy all resources related to the parser. This does not destroy the DTD if the parser was created using the dtd(DTD) option.

set_sgml_parser(+Parser, +Option)

Sets attributes to the parser. Currently defined attributes:

file(File)

Sets the file for reporting errors and warnings. Sets the line to 1.

line(Line)

Sets the current line. Useful if the stream is not at the start of the (file) object for generating proper line-numbers.

linepos(LinePos)

Sets notion of the current column in the source line.

charpos(Offset)

Sets the current character location. See also the file(File) option.

position(Position)

Set source location from a stream position term as obtained using stream_property(Stream, position(Position)).

dialect(Dialect)

Set the markup dialect. Known dialects:

sgml: The default dialect is to process as SGML. This implies markup is case-insensitive and standard SGML abbreviation is allowed (abreviated attributes and omitted tags).
html
html4: This is the same as sgml, but implies shorttag(false) and accepts XML empty element declarations (e.g., <img src="..."/>).
html5: In addition to html, accept attributes named data- without warning. This value initialises the charset to UTF-8.
xhtml
xhtml5: These document types are processed as xml. Dialect xhtml5 accepts attributes named data- without warning.
xml: This dialect is selected automatically if the processing instruction <?xml ...> is encountered. See section 3.3 for details.
xmlns: Process file as XML file with namespace support. See section 3.3.1 for details. See also the qualify_attributes option below.

xmlns(+URI)

Set the default namespace of the outer environment. This option is provided to process partial XML content with proper namespace resolution.

xmlns(+NS, +URI)

Specify a namespace for the outer environment. This option is provided to process partial XML content with proper namespace resolution.

qualify_attributes(Boolean)

How to handle unqualified attribute (i.e. without an explicit namespace) in XML namespace (xmlns) mode. Default and standard compliant is not to qualify such elements. If true, such attributes are qualified with the namespace of the element they appear in. This option is for backward compatibility as this is the behaviour of older versions. In addition, the namespace document suggests unqualified attributes are often interpreted in the namespace of their element.

space(SpaceMode)

Define the initial handling of white-space in PCDATA. This attribute is described in section 3.2.

number(NumberMode)

If token (default), attributes of type number are passed as a Prolog atom. If integer, such attributes are translated into Prolog integers. If the conversion fails (e.g. due to overflow) a warning is issued and the value is passed as an atom.

encoding(Encoding)

Set the initial encoding. The default initial encoding for XML documents is UTF-8 and for SGML documents ISO-8859-1. XML documents may change the encoding using the encoding= attribute in the header. Explicit use of this option is only required to parse non-conforming documents. Currently accepted values are iso-8859-1 and utf-8.

doctype(Element)

Defines the toplevel element expected. If a <!DOCTYPE declaration has been parsed, the default is the defined doctype. The parser can be instructed to accept the first element encountered as the toplevel using doctype(_). This feature is especially useful when parsing part of a document (see the parse option to sgml_parse/2.

get_sgml_parser(+Parser, -Option)

Retrieve infomation on the current status of the parser. Notably useful if the parser is used in the call-back mode. Currently defined options:

file(-File)

Current file-name. Note that this may be different from the provided file if an external entity is being loaded.

line(-Line)

Line-offset from where the parser started its processing in the file-object.

charpos(-CharPos)

Offset from where the parser started its processing in the file-object. See section 6.

charpos(-Start, -End)

Character offsets of the start and end of the source processed causing the current call-back. Used in PceEmacs to for colouring text in SGML and XML modes.

source(-Stream)

Prolog stream being processed. May be used in the on_begin, etc. callbacks from sgml_parse/2.

dialect(-Dialect)

Return the current dialect used by the parser (sgml, html, html5, xhtml, xhtml5, xml or xmlns).

event_class(-Class)

The event class can be requested in call-back events. It denotes the cause of the event, providing useful information for syntax highlighting. Defined values are:

explicit: The code generating this event is explicitely present in the document.
omitted: The current event is caused by the insertion of an omitted tag. This may be a normal event in SGML mode or an error in XML mode.
shorttag: The current event (begin or end) is caused by an element written down using the shorttag notation (<tag/value/>.
shortref: The current event is caused by the expansion of a shortref. This allows for highlighting shortref strings in the source-text.

doctype(-Element)

Return the defined document-type (= toplevel element). See also set_sgml_parser/2.

dtd(-DTD)

Return the currently used DTD. See dtd_property/2 for obtaining information on the DTD such as element and attribute properties.

context(-StackOfElements)

Returns the stack of currently open elements as a list. The head of this list is the current element. This can be used to determine the context of, for example, CDATA events in call-back mode. The elements are passed as atoms. Currently no access to the attributes is provided.

allowed(-Elements)

Determines which elements may be inserted at the current location. This information is returned as a list of element-names. If character data is allowed in the current location, #pcdata is part of Elements. If no element is open, the doctype is returned.

This option is intended to support syntax-sensitive editors. Such an editor should load the DTD, find an appropriate starting point and then feed all data between the starting point and the caret into the parser. Next it can use this option to determine the elements allowed at this point. Below is a code fragment illustrating this use given a parser with loaded DTD, an input stream and a start-location.

        ...,
        seek(In, Start, bof, _),
        set_sgml_parser(Parser, charpos(Start)),
        set_sgml_parser(Parser, doctype(_)),
        Len is Caret - Start,
        sgml_parse(Parser,
                   [ source(In),
                     content_length(Len),
                     parse(input)       % do not complete document
                   ]),
        get_sgml_parser(Parser, allowed(Allowed)),
        ...

sgml_parse(+Parser, +Options)

Parse an XML file. The parser can operate in two input and two output modes. Output is either a structured term as described with load_structure/2 or call-backs on predefined events. The first is especially suitable for manipulating not-too-large documents, while the latter provides a primitive means for handling very large documents.

Input is a stream. A full description of the option-list is below.

document(-Term)

A variable that will be unified with a list describing the content of the document (see load_structure/2).

source(+Stream)

An input stream that is read. This option must be given.

content_length(+Characters)

Stop parsing after Characters. This option is useful to parse input embedded in envelopes, such as the HTTP protocol.

cdata(+Representation)

Specify the representation of cdata elements. Supported are atom (default), and string. See load_structure/3 for details.

parse(+Unit)

Defines how much of the input is parsed. This option is used to parse only parts of a file.

file: Default. Parse everything upto the end of the input.
element: The parser stops after reading the first element. Using source(Stream), this implies reading is stopped as soon as the element is complete, and another call may be issued on the same stream to read the next element.
content: The value content is like element but assumes the element has already been opened. It may be used in a call-back from call(on_begin, Pred) to parse individual elements after validating their headers.
declaration: This may be used to stop the parser after reading the first declaration. This is especially useful to parse only the doctype declaration.
input: This option is intended to be used in conjunction with the allowed(Elements) option of get_sgml_parser/2. It disables the parser's default to complete the parse-tree by closing all open elements.

max_errors(+MaxErrors)

Set the maximum number of errors. If this number is exceeded further writes to the stream will yield an I/O error exception. Printing of errors is suppressed after reaching this value. The default is 50. Using max_errors(-1) makes the parser continue, no matter how many errors it encounters.

error(limit_exceeded(max_errors, Max), _)

syntax_errors(+ErrorMode)

Defines how syntax errors are handled.

quiet: Suppress all messages.
print: Default. Pass messages to print_message/2.
style: Print dubious input such as attempts for redefinitions in the DTD using print_message/2 with severity informational.

xml_no_ns(+Mode)

Error handling if an XML namespace is not defined. Default generates an error. If quiet, the error is suppressed. Can be used together with call(urlns, Closure) to provide external expansion of namespaces. See also section 3.3.1.

call(+Event, :PredicateName)

Issue call-backs on the specified events. PredicateName is the name of the predicate to call on this event, possibly prefixed with a module identifier. If the handler throws an exception, parsing is stopped and sgml_parse/2 re-throws the exception. The defined events are:

begin: An open-tag has been parsed. The named handler is called with three arguments: Handler(+Tag, +Attributes, +Parser).
end: A close-tag has been parsed. The named handler is called with two arguments: Handler(+Tag, +Parser).
cdata: CDATA has been parsed. The named handler is called with two arguments: Handler(+CDATA, +Parser), where CDATA is an atom representing the data.
pi: A processing instruction has been parsed. The named handler is called with two arguments: Handler(+Text, +Parser), where Text is the text of the processing instruction.
decl: A declaration (<!...>) has been read. The named handler is called with two arguments: Handler(+Text, +Parser), where Text is the text of the declaration with comments removed.
This option is expecially useful for highlighting declarations and comments in editor support, where the location of the declaration is extracted using get_sgml_parser/2.
error: An error has been encountered. the named handler is called with three arguments: Handler(+Severity, +Message, +Parser), where Severity is one of warning or error and Message is an atom representing the diagnostic message. The location of the error can be determined using get_sgml_parser/2
If this option is present, errors and warnings are not reported using print_message/3
xmlns: When parsing an in xmlns mode, a new namespace declaraction is pushed on the environment. The named handler is called with three arguments: Handler(+NameSpace, +URL, +Parser). See section 3.3.1 for details.
urlns: When parsing an in xmlns mode, this predicate can be used to map a url into either a canonical URL for this namespace or another internal identifier. See section 3.3.1 for details.

3.6.1 Partial Parsing

In some cases, part of a document needs to be parsed. One option is to use load_structure/2 or one of its variations and extract the desired elements from the returned structure. This is a clean solution, especially on small and medium-sized documents. It however is unsuitable for parsing really big documents. Such documents can only be handled with the call-back output interface realised by the call(Event, Action) option of sgml_parse/2. Event-driven processing is not very natural in Prolog.

The SGML2PL library allows for a mixed approach. Consider the case where we want to process all descriptions from RDF elements in a document. The code below calls process_rdf_description(Element) on each element that is directly inside an RDF element.

:- dynamic
        in_rdf/0.

load_rdf(File) :-
        retractall(in_rdf),
        open(File, read, In),
        new_sgml_parser(Parser, []),
        set_sgml_parser(Parser, file(File)),
        set_sgml_parser(Parser, dialect(xml)),
        sgml_parse(Parser,
                   [ source(In),
                     call(begin, on_begin),
                     call(end, on_end)
                   ]),
        close(In).

on_end('RDF', _) :-
        retractall(in_rdf).

on_begin('RDF', _, _) :-
        assert(in_rdf).
on_begin(Tag, Attr, Parser) :-
        in_rdf, !,
        sgml_parse(Parser,
                   [ document(Content),
                     parse(content)
                   ]),
        process_rdf_description(element(Tag, Attr, Content)).

3.6 Parsing Primitives

3.6.1 Partial Parsing