v0.5

Changelog

• General
  • Improved Tokay parser newline behavior to support Windows and classic Mac line ending as well
  • Moved tests from test.rs into their particular locations, so tests are written next to their implementation
  • Read from stdin when no input stream but a program that is consuming was specified
  • Moved ccl.rs into separate charclass crate
  • Equipped Reader struct with better tools for scanning (Reader::take(), Reader::span())
• Values
  • New Object trait
  • Splitting enum Value into separate objects using Box<dyn Object>
  • unary_op() and binary_op() for RefValue, with fast-lanes for some operations
  • Replacement of separate binary- and unary-VM instructions into Op::BinaryOp and Op::UnaryOp
  • Implemented all primary operations by builtin methods
• Builtins
  • Entire redesign of builtins using proc-macro
  • Builtin function registry generated by a build-script, substituting inventory/linkme-crates
  • Builtin functions
    • repr() to get string with Tokay object representation
    • type() to get string of Tokay value type
    • bool methods: bool()
    • int methods: int()
    • float methods: float(), float.ceil(), float.fract(), float.trunc()
    • addr methods: addr()
    • dict methods: dict(), dict.len(), dict.update()
    • list methods: list(), list.add(), list.iadd(), list.len(), list.push(), list.pop()
    • str methods: str(), str.add(), str.byteslen() str.endswith(), str.mul(), str.join(), str.len(), str.lower(), str.replace(), str.startswith(), str.substr(), str.upper()
  • Builtin tokens
    • Float allowing for parsing floating point numbers into float values
    • Ident renamed (from Identifier)
    • Int renamed (from Integer) and accepting parameters like a radix base
    • Word skipping words not in the wanted word size, but not rejecting
• New example files
  • Self-hosted Tokay parser in examples/tokay.tok
  • Simple JSON parser in examples/json.tok