base.StringTokenizer.createExt()

createExt() is an extended version of create() which provides the user with some additional parsing options. The method requires 4 arguments.

Syntax:

base.StringTokenizer.CreateExt(string, delimiter, esc, nulls)

• string is a string to be parsed.
• delimiter is a quoted string or a variable specifying the symbol which will be treated as a delimiter.
• esc is a character to escape a delimiter.
• nulls indicates if NULL tokens must be returned.

Usage example:

MAIN
	DEFINE i TINYINT
	DEFINE tok BASE.StringTokenizer

	LET tok = BASE.StringTokenizer.createExt('||#|tok1||tok2|','|','#',TRUE)
	DISPLAY "1. BASE.StringTokenizer.createExt('||#|tok1||tok2|','|','#',TRUE)"
	LET i = 1
	DISPLAY "BASE.StringTokenizer.countTokens() - " , tok.countTokens() #shows six tokens with nulls
	WHILE tok.hasMoreTokens()
		DISPLAY i, ". >", tok.nextToken(),"<"
		LET i = i + 1
	END WHILE
	DISPLAY ""

	LET tok = BASE.StringTokenizer.createExt('||#|tok1||tok2|','|','#',FALSE)
	DISPLAY "2. BASE.StringTokenizer.createExt('||#|tok1||tok2|','|','#',FALSE)"
	LET i = 1
	DISPLAY "BASE.StringTokenizer.countTokens() - " , tok.countTokens() #shows two tokens without nulls
	WHILE tok.hasMoreTokens()
		DISPLAY i, ". >", tok.nextToken(),"<"
		LET i = i + 1
	END WHILE
	DISPLAY ""
END MAIN

If a delimiter symbol is passed to the esc parameter, it will be escaped during the string tokenizing.

When the nulls argument is set to TRUE, the empty tokens are not ignored. As the result, nextToken() can return a NULL string. The leading, trailing and consequent delimiters between tokens influence the number of resulting tokens.