Subject: Re: [Boost-users] regex iterator question
From: John Maddock (john_at_[hidden])
Date: 2009-03-30 11:47:51
> I've got a need for regex and I would like to use it to extract tokens
> matching a regular expresssion from a file stream. Seems like this would
> be a common desire.
>
> So first shot is
>
> boost::regex_token_iterator<some_input_iterator>
>
> This doesn't work since regex_token_iterator requires a bidirectional
> iterator. Seems reasonable enough.
>
> So next I comb through boost and find
>
> #include <boost/spirit/iterator/multi_pass.hpp>
>
> and try
>
> boost::regex_token_iterator<
> boost::spirit::multi_pass<some_input_iterator>
>>
>
> I'm thinking this is veeeeeery cool - maybe a 1000 lines of free code
> included for the price of one.
>
> But, doesn't work. multi_pass is a forward_trasversal iterator while
> regex_token_iterator requires a bidirectional_trasversal_iterator. A huge
> disappoint to come soooo close.
>
> Thinking about it, this problem must come very often. How is it usually
> addressed? There must be a simple bridge across this. In a pinch, I'll
> just have to load the whole file into some sort collection, but I prefer
> the ultimate unlimited file size solution.
If you check the regex examples there are some "load_file" routines than
dump a files contents into memory, but I agree it's not an ideal solution.
I did experiment with some adapters to solve this issue in the early days of
regex but never got a really good solution, and folks weren't demanding it
so it got dropped :-(
But.. how about a memory mapped file? Boost.Interprocess has support for
that:
http://www.boost.org/doc/libs/1_38_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.mapped_file
although I admit it's not quite a one liner...
HTH, John.