From: Thorsten Ottosen (nesotto_at_[hidden])
Date: 2005-04-24 11:54:02
> "Beman Dawes" <bdawes_at_[hidden]> wrote in message
> news:6.0.3.0.2.20050422110501.042b2a20_at_mailhost.esva.net...
|I'd add a tokenizer to the bunch. Not specially sexy but
|massively helpful --the typical impulse of a beginner is to
|do the tokenizing stuff by hand, many a time resulting
|in off-by-one errors, mishandling of unexpected separators
Tokenizer was rejected some years ago on the grounds of being too slow.
I think the regex part contains faster tokenization algorithms and the
remaning
can probably be found in the string library.
-Thorsten