Subject: [Boost-commit] svn:boost r62707 - trunk/libs/tokenizer
From: kbelco_at_[hidden]
Date: 2010-06-09 22:24:32
Author: noel_belcourt
Date: 2010-06-09 22:24:31 EDT (Wed, 09 Jun 2010)
New Revision: 62707
URL: http://svn.boost.org/trac/boost/changeset/62707
Log:
Fixed ticket 2672 by applying (and slightly modifying
the attached patch. The page now seems to validate
correctly.
Text files modified:
trunk/libs/tokenizer/introduc.htm | 30 ++++++++++++++----------------
1 files changed, 14 insertions(+), 16 deletions(-)
Modified: trunk/libs/tokenizer/introduc.htm
==============================================================================
--- trunk/libs/tokenizer/introduc.htm (original)
+++ trunk/libs/tokenizer/introduc.htm 2010-06-09 22:24:31 EDT (Wed, 09 Jun 2010)
@@ -16,9 +16,9 @@
<h1 align="center">Introduction</h1>
- <p align="left">The boost Tokenizer package provides a flexible and easy to
- use way to break of a string or other character sequence into a series of
- tokens. Below is a simple example that will break up a phrase into
+ <p align="left">The Boost Tokenizer package provides a flexible and
+ easy-to-use way to break a string or other character sequence into a series
+ of tokens. Below is a simple example that will break up a phrase into
words.</p>
<div align="left">
@@ -40,19 +40,19 @@
</pre>
</div>
- <p align="left">You can choose how the string gets broken up. You do this
- by specifying the TokenizerFunction. If you do not specify anything, the
- default TokenizerFunction is char_delimiters_separator<char> which
+ <p align="left">You can choose how the string gets parsed by using the
+ TokenizerFunction. If you do not specify anything, the default
+ TokenizerFunction is <em>char_delimiters_separator<char></em> which
defaults to breaking up a string based on space and punctuation. Here is an
- example of using another TokenizerFunction called escaped_list_separator.
- This TokenizerFunction parses a superset of comma separated value (csv)
- lines. The format looks like this</p>
+ example using another TokenizerFunction called
+ <em>escaped_list_separator</em>. This TokenizerFunction parses a superset
+ of comma-separated value (CSV) lines. The format looks like this:</p>
<p align="left">Field 1,"putting quotes around fields, allows commas",Field
3</p>
<p align="left">Below is an example that will break the previous line into
- its 3 fields</p>
+ its three fields.</p>
<div align="left">
<pre>
@@ -73,13 +73,11 @@
</pre>
</div>
- <p align="left">Finally, for some TokenizerFunctions you have to pass in
+ <p align="left">Finally, for some TokenizerFunctions you have to pass
something into the constructor in order to do anything interesting. An
- example is offset_separator. This class breaks a string into tokens based
- on offsets for example</p>
-
- <p align="left">12252001 when parsed using offsets of 2,2,4 becomes 12 25
- 2001. Below is an example to parse this.</p>
+ example is the offset_separator. This class breaks a string into tokens based
+ on offsets. For example, when <em>12252001</em> is parsed using offsets of
+ 2,2,4 it becomes <em>12 25 2001</em>. Below is the code used.</p>
<div align="left">
<pre>