From: johan.nilsson_at_[hidden]
Date: 2001-09-07 03:09:41
Thanks,
I've been off for a while now. I'll look into it and try it out asap.
// Johan
> -----Original Message-----
> From: jbandela_at_[hidden] [mailto:jbandela_at_[hidden]]
> Sent: den 31 augusti 2001 20:22
> To: boost_at_[hidden]
> Subject: [boost] Re: Tokenizer and empty tokens
>
>
> Johan
> Updated
> http://groups.yahoo.com/group/boost/files/token_functions.hpp
>
> I am now thinking that char_delimiters_separator is not the place to
> do empty tokens. Since emtpy tokens almost always involve fields, I
> think escaped_list_separator is a better place for it. To support
> problems like yours I did 2 things
>
> 1. Made sure it returned trailing empty tokens
> 2. Allow you to specify multiple characters as escape,c,quote
>
> Here is an example
>
> string s = "$GPGSA,A,3,25,04,16,06,18,24,,,,,,,1.86,0.92,1.62*0E,";
> typedef escaped_list_separator<char> sep_type;
> sep_type sep("","$,*","");
> tokenizer<sep_type> t(s,sep);
> copy(t.begin(),t.end(),ostream_iterator<string>(cout,"\n"));
>
> Regards
>
> John R. Bandela
>
>
> --- In boost_at_y..., johan.nilsson_at_e... wrote:
> > First of all: yes, now it works in my particular case.
> >
> > Couple of notes though:
> >
> > * [when allowing empty tokens] char_delimiters_separator outputs
> leading,
> > but not trailing empty tokens, e.g.:
> >
> > tokenizing ",test," outputs ("" "test") instead of ("" "test" ""),
> which at
> > least I would expected.
> >
> > * Off the top of my head, it seems like escaped_list_separator
> should be
> > able to do the same thing (output empty tokens). Which leads to the
> question
> > on _where_ the policy on outputting empty tokens should be adapted,
> or?
> >
> > // Johan
> >
>
>
>
> Info: http://www.boost.org Unsubscribe:
<mailto:boost-unsubscribe_at_[hidden]>
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/