This section of the archives stores flipcode's complete Developer Toolbox collection, featuring a variety of mini-articles and source code contributions from our readers.

| Parsing Console Commands Submitted by

The C function strtok (defined in string.h) is great for writing basic text parsers. For those who haven't encountered it before, strtok splits a string into tokens delimited by user defined separators, eg. whitespace, newlines or perhaps commas.
Sadly, it's not quite up to the task of parsing in-game console commands that take strings containing spaces as arguments, eg. <code>say_team "A string with spaces"</code> as it cannot be made to understand that delimiters should be ignored when enclosed by quotes.
I love the simplicity of using strtok, so I decided to code 'ConsoleStrTok' which works in a similar fashion to strtok, and makes the whole process of parsing console commands dead easy. ConsoleStrTok tokenizes the input string, returning each token as a string. A token is either a string of non-whitespace characters, or a string of arbitrary characters that were enclosed by quotes in the input string. This way you can treat each token the same, with no need for special case code.
Parsing is similar to using strtok - for the first call you pass the pointer to the string to be tokenized, and for subsequent calls you pass in NULL. Each time you get back a pointer the current token or NULL if no more can be found.
As an example, your console command parsing routine could be:

    void Console::ParseConsoleCommand( char* inputBuffer )
    {
        // Find the first token, ie. the command name
        char *commandName = ConsoleStrTok( inputBuffer );

        // Check to see if the command exists...

        // Get the first argument token
        char *commandArg = ConsoleStrTok( NULL );
        int numArgs = 0;

        while ( commandArg != NULL ) {
            ++numArgs;
            // ...do your per-argument processing, type checking etc.
            // Get the next argument token
            commandArg = ConsoleStrTok( NULL );
        }

        // build argument list and call command handler...
    }

Important Notes:

> - The input string will have null characters inserted at the end of each token. Keep a copy if you need the original again!
> - Token delimiters are hard coded as whitespace (tab, space, newline)
> - Parsing multiple strings at once is not possible, as I use an internal static variable. For most purposes this won't be a problem, but if you really need to parse multiple strings at once, I recommend placing the routine into its own class, with pNext as a non-static member variable.
> - As a bonus side effect, parsing `"say_team" "hello"` will return the same tokens as `say_team "hello"` So you could even have console commands containing spaces, if you're feeling different! ;)

Download Associated File: strtokreplace.txt (4,272 bytes)

    /*

    COTD ENTRY
    ==========

    A Simple Routine for Parsing Console Commands

    Chris Cookson (cjcookson@hotmail.com)

    The C function strtok (defined in string.h) is great for writing basic text
    parsers. For those who haven't encountered it before, strtok splits a string
    into tokens delimited by user defined separators, eg. whitespace, newlines
    or perhaps commas.

    Sadly, it's not quite up to the task of parsing in-game console commands
    that take strings containing spaces as arguments, eg. <code>say_team "A
    string with spaces"</code> as it cannot be made to understand that
    delimiters should be ignored when enclosed by quotes.

    I love the simplicity of using strtok, so I decided to code 'ConsoleStrTok'
    which works in a similar fashion to strtok, and makes the whole process of
    parsing console commands dead easy. ConsoleStrTok tokenizes the input
    string, returning each token as a string. A token is either a string of
    non-whitespace characters, or a string of arbitrary characters that were
    enclosed by quotes in the input string. This way you can treat each token
    the same, with no need for special case code.

    Parsing is similar to using strtok - for the first call you pass the pointer
    to the string to be tokenized, and for subsequent calls you pass
    in NULL. Each time you get back a pointer the current token or NULL if
    no more can be found.

    As an example, your console command parsing routine could be:

    */

    void Console::ParseConsoleCommand( char* inputBuffer )
    {
        // Find the first token, ie. the command name
        char *commandName = ConsoleStrTok( inputBuffer );

        // Check to see if the command exists...

        // Get the first argument token
        char *commandArg = ConsoleStrTok( NULL );
        int numArgs = 0;

        while ( commandArg != NULL ) {
            ++numArgs;
            // ...do your per-argument processing, type checking etc.
            // Get the next argument token
            commandArg = ConsoleStrTok( NULL );
        }

        // build argument list and call command handler...
    }

    /*

    Important Notes:

    * The input string will have null characters inserted at the end of each
      token. Keep a copy if you need the original again!
    * Token delimiters are hard coded as whitespace (tab, space, newline)
    * Parsing multiple strings at once is not possible, as I use an internal
      static variable. For most purposes this won't be a problem, but if you
      really need to parse multiple strings at once, I recommend placing the
      routine into its own class, with pNext as a non-static member variable.
    * As a bonus side effect, parsing <code>"say_team" "hello"</code> will
      return the same tokens as <code>say_team "hello"</code> So you could
      even have console commands containing spaces, if you're feeling
      different! ;)

    The Routine:

    */

    char * ConsoleStrTok( char *strToken )
    {
    	// Where to start searching next
    	static char *pNext;
    	// Start of next token
    	char *pStart;

    	// If NULL is passed in, continue searching
    	if ( strToken == NULL ) {
    		if ( pNext != NULL ) {
    			strToken = pNext;
    		} else {
    			// Reached end of original string
    			return NULL;
    		}
    	}

    	// Zero length string, so no more tokens to be found
    	if ( *strToken == 0 ) {
    		pNext = NULL;
    		return NULL;
    	}

    	// Skip leading whitespace before next token
    	while (	(*strToken == ' ') || (*strToken == '\t')
    			|| (*strToken == '\n') ) {

    		++strToken;
    	}

    	// It's a quoted literal - skip the first quote char
    	if ( *strToken == '\"' ) {
    		++strToken;

    		pStart = strToken;

    		// Find ending quote or end of string
    		while ( (*strToken != '\"') && (*strToken != 0) ) {
    			++strToken;
    		}

    		if ( *strToken == 0 ) {
    			// Reached end of original string
    			pNext = NULL;
    		} else {
    			// More to find, note where to continue searching
    			*strToken = 0;
    			pNext = strToken + 1;
    		}

    	} else {
    		// Unquoted token
    		pStart = strToken;

    		// Find next whitespace delimiter or end of string
    		while ( (*strToken != 0) && (*strToken != ' ') &&
    				(*strToken != '\t') && (*strToken != '\n' ) ) {
    			++strToken;
    		}

    		// Reached end of original string?
    		if ( *strToken == 0 ) {
    			pNext = NULL;
    		} else {
    			*strToken = 0;
    			pNext = strToken + 1;
    		}
    		return pStart;
    	}

    	// Return ptr to start of token
    	return pStart;
    }

    /*

    Cheers,

    Chris Cookson
    ---
    cjcookson@hotmail.com

    */

The zip file viewer built into the Developer Toolbox made use of the zlib library, as well as the zlibdll source additions.

Copyright 1999-2008 (C) FLIPCODE.COM and/or the original content author(s). All rights reserved.
Please read our Terms, Conditions, and Privacy information.