move to a direct byte code compiler (#8)

* chore: refactor to keep only the new parser and bytecode vm

* scanner: remove Token.value field

* scanner: remove scanner.kind field

* chore: move language specification in lang package

This avoid a cyclic dependency in scanner_test which can now use
the golang/GoSpec language specification for Go.

* clean code

* scanner: export scanner fields

Also parser now generate function calls, including externals.

* chore: fix lint issues

* parser: handle strings

* wip

* parser: implement support for 'if, else, else if' statements

Resolving labels in the compiler still in progress.

* parser: support if statements, improve compiler

* improve handling of functions

* improve support of local variables

* scanner: trim leading and trailing spaces

* fixes to make fibonacci work

* parser: improve README, fix function parameters parsing

1 files changed, 43 insertions, 0 deletions

diff --git a/lang/spec.go b/lang/spec.go
new file mode 100644
index 0000000..c5b50a4
--- /dev/null
+++ b/lang/spec.go
@@ -0,0 +1,43 @@
+package lang
+
+const (
+	CharIllegal = 1 << iota
+	CharOp
+	CharNum
+	CharAlpha
+	CharSep
+	CharLineSep
+	CharGroupSep
+	CharStr
+	CharBlock
+	StrEsc
+	StrNonl
+	ExcludeEnd  // exclude end delimiter from content
+	EosValidEnd // end of input string terminates block or string token
+)
+
+const ASCIILen = 1 << 7 // 128
+
+type TokenProp struct {
+	TokenId
+	SkipSemi bool // automatic semicolon insertion after newline
+}
+
+type Spec struct {
+	CharProp   [ASCIILen]uint       // special Character properties
+	End        map[string]string    // end delimiters, indexed by start
+	BlockProp  map[string]uint      // block properties
+	TokenProps map[string]TokenProp // token properties
+	DotNum     bool                 // true if a number can start with '.'
+	IdAscii    bool                 // true if an identifier can be in ASCII only
+	Num_       bool                 // true if a number can contain _ character
+}
+
+// HasInit stores if a statement may contain a simple init statement
+var HasInit = map[TokenId]bool{
+	Case:   true,
+	For:    true,
+	If:     true,
+	Select: true,
+	Switch: true,
+}