chilon::parser tutorial - part 1

In all these examples the declaration `using namespace chilon::parser' has been assumed.

The first example parser can store only a simple javascript variable declaration. The variable name parser is too permissive in this example, it will be extended to support the correct syntax in tutorials 3 and 4. The test data:



var lonely

The code to parse it:



// The contents of this file are discussed at the end of this tutorial page.
// It provides the `test' function called below.
#include <test_parser.hpp>

int main(int argc, char *argv[]) {
    // This example can store a single variable definition only.
    return test<
        sequence<
            // This will not be stored as it always matches the same thing.
            // I call parsers like these that do not store `constant' parsers.
            char_<v,a,r>,
            // Whitespace can appear between elements in a sequence, in this
            // case it is handled by the cplusplus_stream which skips whitespace
            // and C++ style comments (which also matches javascript's
            // comments). many<char-type-parser> always stores a string.
            many<non_whitespace>
        >
    >(argc, argv);
}

The output of this program is a simple string. As the sequence only stores one element the result of the above parser is a range (essentially a string, represented by storing a pointer to the beginning of the variable name, and a pointer to one past the last character in the variable name. The range type works with C++ stream operators and can be used pretty much like an STL string).



"lonely"

If you are willing to accept the test function works and translates the input file given (based on argv) to an AST, then prints this AST you can move on to tutorial 2.

I'll explain a few things used in all the examples so you might easily compile these examples yourself. The following function is available inside chilon/parser/test.hpp that is used in all these examples to parse source code and print a tree with only the type of the parser given through `Parser'.



#include <chilon/parser.hpp>
#include <chilon/print.hpp>

namespace chilon { namespace parser {

// chilon::printers can pretty print tuples, variants, vectors and the other
// types which chilon uses to create ASTs. It is also easy to hook your own
// types into chilon::printer (which is shown in the tutorials).
template <class Parser, class Stream, class Callback = chilon::printer<>>
bool test(Stream& stream, Callback c = chilon::make_printer(std::cout)) {
    // store creates an AST type based on the Parser type at compile time
    // and has a member of this type which can be accessed through `value_'.
    store<Parser> storer;
    stream.skip_whitespace();

    // storer's operator() uses the data in the stream to populate the
    // AST in its value_ member. This storer returns true if any of the input
    // was matched otherwise false.
    if (storer(stream)) {
        c(storer.value_);
        return true;
    }
    else return false;
}

} }

In addition these tutorials make use of a wrapper which creates the parser and instantiates it to parse a file based on the command line arguments.



// The file in the code block above.
#include <chilon/parser/test.hpp>
// This file includes every file in the chilon::parser namespace.
#include <chilon/parser.hpp>

// The following line allows alpha characters to be used by name e.g.
// char_<a> vs char_<'a'>
using namespace chilon::parser::ascii;

using namespace chilon::parser;

// The type of cplusplus_stream is essentially a representation of a file
// in memory that can be parsed and will be discussed later. It is important
// to know that streams must know how to skip whitespace as this ability makes
// it much easier to generate rules that can automatically build ASTs.
// `cplusplus_stream' skips C++ style comments and whitespace.  The parser
// type Parser is responsible for turning this whitespace skipping stream into
// an Abstract Syntax Tree.
template <class Parser, class Stream = cplusplus_stream>
int test(int argc, char *argv[]) {
    Stream stream;
    if (argc != 2) {
        std::cout << "program requires a single argument\n";
        return 1;
    }

    if (! stream.load(argv[1])) {
        std::cerr << "could not load file\n";
        return 1;
    }

    if (test<Parser>(stream))
        return 0;
    else {
        std::cerr << "invalid contents\n";
        std::cerr << stream.begin();
        return 1;
    }
}

previous | next