Implement strtok() function in c/c++

In this article, we will see how to implement strtok() function in C/C++

What is strtok

Strtok is a StringTokenizer function that takes the string and delimiter as input parameters and splits the string w.r.t. the delimiter and provides the output.

Following is the implementation:

// Author: Murali Krishna
// Email: bmknaidu555@gmail.com
// Message: Comment below if you find any mistakes in this program

#include <iostream>

#pragma warning(disable: 4996)

char * mystrtok(char * string,
  const char * delim) {

  /* Static storage class is used to keep 
   track of the string's index in each function call */
  static char * index;

  // Initial case, where user passes the actual string in strtok
  if (string != NULL) {
    index = string;
  } else {
    // further cases, where NULL is passed
    string = index;
  }

  // Final case where the index will be '\0'
  if ( * index == '\0') {
    return NULL;
  }

  while ( * index != '\0') {
    // Iterate over each delimeter and check if any delimiter matches to the character
    for (int i = 0; delim[i] != '\0'; i++) {
      if ( * index == delim[i]) {

        // We are not intrested in the following case where there is 
        // no character available to print before delimiter.
        // This case occurs when two delimiters are side by side.
        if (index == string) {
          index++;
          string++;
        } else {
          * index = '\0';
          break;
        }
      }
    }

    // return the token
    if ( * index == '\0') {
      index++;
      return string;
    }

    index++;
  }
  return string;

}

int main(void) {

  // Declare a character array and initialize it with the input string.
  char string[100] = "a=b;a=c;a=d;a=f;a=g;a=h;a=i;a=j;a=k;";
  
  // String delimiter - To split the string
  const char * delimiter = ";";
  
  char * returnS = mystrtok(string, delimiter);

  // Call the API in a loop till the return String in NULL
  while (returnS != NULL) {
    std::cout << returnS << std::endl;
    returnS = mystrtok(NULL, delimiter);
  }

  return 0;
}

Output:

a=b
a=c
a=d
a=f
a=g
a=h
a=i
a=j
a=k

Explanation:

The above program is an implementation of the strtok function, which is used to split a string into tokens based on a specified delimiter. The implementation of strtok is done in the mystrtok function.

A static storage class is used to keep track of the string’s index in each function call. When first time the function is called, the actual string is passed to the function. On subsequent calls, NULL is passed. If the index of the string is equal to ‘\0‘, the function returns NULL, indicating the end of the string.

The function iterates over the string and checks each character against the characters in the delimiter. If a match is found, the character is replaced with a null terminator and returns the token formed before the delimiter.

The pragma warning(disable: 4996) line disables a warning that may be generated when compiling the code with a C++ compiler. The warning is related to the use of the strtok function, which is considered unsafe in C++.