tokenize - split string into words .EX include "sys.m"; sys := load Sys Sys->PATH; tokenize: fn(s, delim: string): (int, list of string); Tokenize breaks s into words separated by characters in delim . The returned tuple holds the number of words and an ordered list of those words (whose hd gives the leftmost word from s ).

Words are delimited by the maximal sequences of any character from the delim string. Tokenize skips delimiter characters at the beginning and end of s , so each element in the returned list has non-zero length.

If s is nil or contains no words, tokenize returns a count of zero and a nil list.

Delim may be nil or the empty string, specifying no delimiter characters. The resulting word list will be nil (if s is nil or the empty string) or a single-item list with a copy of s .

sys-intro (2)