# ACE_Tokenizer Class Reference

Tokenizer. More...

`#include <SString.h>`

Collaboration diagram for ACE_Tokenizer:

[legend]
List of all members.

## Public Types

enum | { MAX_DELIMITERS = 16, MAX_PRESERVES = 16 }

## Public Member Functions

| ACE_Tokenizer (ACE_TCHAR *buffer)
int | delimiter (ACE_TCHAR d)
int | delimiter_replace (ACE_TCHAR d, ACE_TCHAR replacement)
int | preserve_designators (ACE_TCHAR start, ACE_TCHAR stop, int strip=1)
ACE_TCHAR * | next (void)
| Returns the next token.

## Protected Member Functions

int | is_delimiter (ACE_TCHAR d, int &replace, ACE_TCHAR &r)
int | is_preserve_designator (ACE_TCHAR start, ACE_TCHAR &stop, int &strip)

## Protected Attributes

ACE_TCHAR * | buffer_
int | index_
Preserve_Entry | preserves_ [MAX_PRESERVES]
| The application can specify MAX_PRESERVES preserve designators.
int | preserves_index_
| Pointer to the next free spot in preserves_.
Delimiter_Entry | delimiters_ [MAX_DELIMITERS]
| The tokenizer allows MAX_DELIMITERS number of delimiters.
int | delimiter_index_
| Pointer to the next free space in delimiters_.

## Detailed Description

Tokenizer.

Tokenizes a buffer. Allows application to set delimiters and preserve designators. Does not allow special characters, yet (e.g., printf ("\"like a quoted string\"")).

## Member Enumeration Documentation

anonymous enum

Enumeration values:

  MAX_DELIMITERS |
  MAX_PRESERVES |

## Constructor & Destructor Documentation

ACE_Tokenizer::ACE_Tokenizer | ( | ACE_TCHAR * | buffer | ) |

buffer will be parsed. Notice that ACE_Tokenizer will modify buffer if you use `delimiter_replace` or `preserve_designators` to do character substitution.

Note:
  You should NOT pass a constant string or string literal to this constructor, since ACE_Tokenizer will try to modify the string.

See also:
  preserve_designators

  preserve_designators

## Member Function Documentation

int ACE_Tokenizer::delimiter | ( | ACE_TCHAR | d | ) |

d is a delimiter.

Returns:
  Returns 0 on success, -1 if there is no memory left.

Example:

         char buf[30];
         ACE_OS::strcpy(buf, "William/Joseph/Hagins");

         ACE_Tokenizer tok (buf);
         tok.delimiter ('/');
         for (char *p = tok.next (); p; p = tok.next ())
          cout << p << endl;

This will print out:

         William/Joseph/Hagins
          Joseph/Hagins
          Hagins

int ACE_Tokenizer::delimiter_replace | ( | ACE_TCHAR | d,
|  | ACE_TCHAR | replacement
| ) |

d is a delimiter and, when found, will be replaced by replacement.

Returns:
  0 on success, -1 if there is no memory left.

Example:

         char buf[30];
         ACE_OS::strcpy(buf, "William/Joseph/Hagins");

         ACE_Tokenizer tok (buf);
         tok.delimiter_replace ('/', 0);
         for (char *p = tok.next (); p; p = tok.next ())
           cout << p << endl;

This will print out:

           William
           Joseph
           Hagins

int ACE_Tokenizer::is_delimiter | ( | ACE_TCHAR | d,
|  | int & | replace,
|  | ACE_TCHAR & | r
| ) | `[protected]`

| Returns 1 if <d> is a delimiter, 0 otherwise. If <d> should be replaced with <r>, <replace> is set to 1, otherwise 0.

int ACE_Tokenizer::is_preserve_designator | ( | ACE_TCHAR | start,
|  | ACE_TCHAR & | stop,
|  | int & | strip
| ) | `[protected]`

| If <start> is a start preserve designator, returns 1 and sets <stop> to the stop designator. Returns 0 if <start> is not a preserve designator.

ACE_TCHAR * ACE_Tokenizer::next | ( | void |  | ) |

| Returns the next token.

int ACE_Tokenizer::preserve_designators | ( | ACE_TCHAR | start,
|  | ACE_TCHAR | stop,
|  | int | strip = `1`
| ) |

Extract string between a pair of designator characters. For instance, quotes, or '(' and ')'. start specifies the begin designator. stop specifies the end designator. strip If strip == 1, then the preserve designators will be stripped from the tokens returned by next.

Returns:
  0 on success, -1 if there is no memory left.

Example with strip = 0:

         char buf[30];
         ACE_OS::strcpy(buf, "William(Joseph)Hagins");

         ACE_Tokenizer tok (buf);
         tok.preserve_designators ('(', ')', 0);
         for (char *p = tok.next (); p; p = tok.next ())
           cout << p << endl;

This will print out:

          William(Joseph)Hagins
          (Joseph)Hagins
          )Hagins

Example with strip = 1:

         char buf[30];
         ACE_OS::strcpy(buf, "William(Joseph)Hagins");

         ACE_Tokenizer tok (buf);
         tok.preserve_designators ('(', ')', 1);
         for (char *p = tok.next (); p; p = tok.next ())
           cout << p << endl;

This will print out:

          William
          Joseph
          Hagins

## Member Data Documentation

ACE_TCHAR* ACE_Tokenizer::buffer_`[protected]`

int ACE_Tokenizer::delimiter_index_`[protected]`

| Pointer to the next free space in delimiters_.

Delimiter_Entry ACE_Tokenizer::delimiters_[MAX_DELIMITERS]`[protected]`

| The tokenizer allows MAX_DELIMITERS number of delimiters.

int ACE_Tokenizer::index_`[protected]`

Preserve_Entry ACE_Tokenizer::preserves_[MAX_PRESERVES]`[protected]`

| The application can specify MAX_PRESERVES preserve designators.

int ACE_Tokenizer::preserves_index_`[protected]`

| Pointer to the next free spot in preserves_.

The documentation for this class was generated from the following files:

- SString.h
- SString.cpp