StringTokenizer and the Missing Token

If we have a comma delimited string like “token0,token1,token2,,token4” that has some token(s) missing, and we try to use java.util.StringTokenizer to place each delimited token in a slot of an array A such that:

 A[0]="token0"; A[1]="token1"; A[2]="token2"; A[3]=""; A[4]="token4"; 

we might write code like:

 String s = "token0,token1,token2,,token4"; java.util.StringTokenizer stringTokenizer=    new java.util.StringTokenizer(s,","); int index = 0; int numTokens = stringTokenizer.countTokens(); System.out.println("num tokens: "+numTokens); String[] A = new String[numTokens]; if(numTokens>0) {    while(stringTokenizer.hasMoreTokens())    {       String token = stringTokenizer.nextToken();       A[index++] = token;    } } 

Notice that array A contains:


We get no indication of the missing token. In fact, the variable numTokens will have the value of 4, representing the number of tokens that are actually in a tokenized string.

The way around this behavior, if we want an empty string indication of the missing token, lies in one of the constructors of the java.util.StringTokenizer class, namely:

 public StringTokenizer(String str,                        String delim,                        boolean returnTokens) 

Here, the boolean returnTokens indicates to the java.util.StringTokenizer instance that we’d like to have, in addition to the actual tokens, the delimiters to be returned as tokens.

The following method uses the above mentioned StringTokenizer constructor, and returns a java.util.Vector whose elements are the tokens of a tokenized string and empty strings for any missing tokens.

 Vector split(String input,String delimiter) {    boolean wasDelimiter=true;    String token=null;    Vector v=new Vector();    StringTokenizer st=new StringTokenizer(input,delimiter,true);    while(st.hasMoreTokens())    {       token=st.nextToken();       if(token.equals(delimiter))       {          if(wasDelimiter)          {             token="";          }          else          {             token=null;          }          wasDelimiter=true;       }       else       {          wasDelimiter=false;       }       if(token!=null)       {          v.addElement(token);       }    }    return v; } 
Share the Post:
Share on facebook
Share on twitter
Share on linkedin


The Latest

technology leadership

Why the World Needs More Technology Leadership

As a fact, technology has touched every single aspect of our lives. And there are some technology giants in today’s world which have been frequently opined to have a strong influence on recent overall technological influence. Moreover, those tech giants have popular technology leaders leading the companies toward achieving greatness.

iOS app development

The Future of iOS App Development: Trends to Watch

When it launched in 2008, the Apple App Store only had 500 apps available. By the first quarter of 2022, the store had about 2.18 million iOS-exclusive apps. Average monthly app releases for the platform reached 34,000 in the first half of 2022, indicating rapid growth in iOS app development.

microsoft careers

Top Careers at Microsoft

Microsoft has gained its position as one of the top companies in the world, and Microsoft careers are flourishing. This multinational company is efficiently developing popular software and computers with other consumer electronics. It is a dream come true for so many people to acquire a high paid, high-prestige job