CSTR.iTokenize Method

CSTRiTokenize Method

Tokenize a string based on any characters.

Available since Oasis montaj version: 5.0.0
License: Available to anyone.

Namespace: GeoEngine.Core.GXNet
Assembly: geoengine.core.gxnet (in geoengine.core.gxnet.dll)

Notes

This uses a finite state machine to tokenize on these rules: 1. Any one character following an escape delimiter is treated as a normal character. 2. Any characters inside a quote string are treated as normal characters. 3. Any number of Soft delimiters in sequence without a hard delimiter are treated as one hard delimited. 4. Any number of soft delimiters can preceed or follow a hard delimiter and are ignored. EXAMPLE Soft = [ ] Hard = [,] Escape = [\] Quote = ["] [this is a , , the "test," of , \,\" my delimite fi,] Results in: [this] [is] [a] [] [the] ["test,"] [of] [\,\"] [my] [delimite] [fi] [] NOT use this function except in GXC code. The corresponding etToken_STR function will not operate correctly in GX.Net code.

Syntax

C++

Copy

public static int iTokenize(
	ref string str1,
	string str2,
	string str3,
	string str4,
	string str5
)

Public Shared Function iTokenize ( 
	ByRef str1 As String,
	str2 As String,
	str3 As String,
	str4 As String,
	str5 As String
) As Integer

public:
static int iTokenize(
	String^% str1, 
	String^ str2, 
	String^ str3, 
	String^ str4, 
	String^ str5
)

static member iTokenize : 
        str1 : string byref * 
        str2 : string * 
        str3 : string * 
        str4 : string * 
        str5 : string -> int

Parameters

str1: Type: SystemString
str - String containing token(s)
str2: Type: SystemString
szSoft - Soft delimiters (spaces/tabs)
str3: Type: SystemString
szHard - Hard delimiters (commas)
str4: Type: SystemString
szEsc - Escape delimiters (back-slash)
str5: Type: SystemString
szQuote- Quote delimiters (quote characters)

Return Value

Type: Int32
number of tokens

Reference

CSTR Class

GeoEngine.Core.GXNet Namespace