Subject: Re: [boost] Updating the Boost Review Process Was: [GGL] Bost.Polygon (GTL) vs GGL - rationale
From: John Phillips (phillips_at_[hidden])
Date: 2009-11-17 12:13:01
Jose wrote:
> On Mon, Nov 16, 2009 at 8:30 PM, John Phillips
> <phillips_at_[hidden]> wrote:
>> ..
>
> Just to make it clear, my proposal is not based on using votes. The
> idea is that if there is clearly two different overall opinions, and
> the NOs are not going to be reversed by the changes anyway, then the
> reasoning for the acceptance has to be well justified OR the judgement
> of the review manager should be questioned. Otherwise, you're ignoring
> the No group!
Look at the Announcement that Polygon was accepted. Fernando
addresses the specific complaints of the "No" group one at a time. That
is not ignoring them, in my opinion.
>
> With broad libraries covering different application domains, it seems
> obvious that the above might happen, and it's not a question of one vs
> the other but of broadening the purpose of the library (if technically
> possible!)
The problem with broadening the purpose is that sometimes it is far
harder to do than is obvious before trying. (Think about how many bad
cross platform GUI libraries have been written.) There are at least two
valid ways to accomplish the goal. The first is to work from the top
down - design for the broad purpose from the beginning and fit the
pieces in. The second is to work from the bottom up - make libraries
that are well suited to the pieces while only worrying about having
compatible concepts, then when the pieces work well look to refactor and
combine. In the Polygon review, and now in the GGL review there has been
animated discussion of what those compatible concepts should be, so they
are following the second route.
>
> The earlier boost libraries were more fundamental and broadly useful
> but some of the newer libraries are specific to application domains,
> not to all boost/c++ users.
Take a look at the Review Schedule page. Libraries like uBLAS and
Special Functions that are tied closely to some specific domains have
been around for many years. Now look at the queue. Libraries like the
Logging proposals could be useful in many different domains. It has
always been a mix.
>
>> ..
>
> Well, maybe you made a mistake! If not, then please take action and
> FIX the situation. Also, the reviewer has to acknowledge that he has
> time to give a timely decision and engage all reviewers. This takes a
> lot of time !!
I have not excluded the possibility that we made a mistake, but I
have seen no proof that we did. If no mistakes were made, then there is
nothing to fix (whether or not it is shouted).
I am quite familiar with the time requirements of managing a Boost
review. I've done it a few times. Though this is Fernando's first time
managing, he has successfully submitted a couple of libraries and knows
the review process as a developer well. However, since he lives in the
real world where work requirements come up while you are doing other
things. As a volunteer organization, we just have to accept that this
happens.
>
> If you look at the specific case, the reviewer is very experienced and
> has given lots of advice to one author, as acknowledged in one
> boostcon paper but the other application domain has been mostly
> ignored. From a generic library viewpoint you need to try to reconcile
> views that are from different applications domains, you can not just
> ignore half of the potential users!
A frequent piece of design advice for generic libraries is to
understand a more limited case first, then expand from there. Polygon
and GGL are both limited in some ways, but both are broad enough to be
useful in real world code. (Both have established user bases, after
all.) I do not know the technical details of the problem domain well
enough to know how hard merging all the different computational geometry
sub-domains into one generic library will be, but I would expect it to
be very hard. In such a case, it is not wise to let the perfect become
the enemy of the good.
>
>> There has been talk of parallel reviews of competing libraries. This sounds
>> like a decent idea on the surface but it has been tried and it did not go
>> well. In the review of the two Thread Pool libraries, I do not think anyone
>> involved was happy with how things ran. We discussed it as a community in
>> advance and decided to review them together, but the reality was just not
>> satisfying.
My mistake, here. It was the competing Futures libraries, not Thread
Pool. Not central, but still wrong.
>
> Nobody is suggesting more competition. I think there are times only
> library proposal doesn't cut the mustard and the submitter realizes
> this and abandons the proposal.
>
> A different case, the current case, is that different proposals are
> overlapping, competition has been fostered and you actually want
> someone experienced to guide the cooperation to get a single library.
The Futures review was not a case of anyone abandoning anything.
Anthony submitted an implementation of the proposal for addition to the
standard library. He did not want to change or merge it because then it
would not implement the proposal. Oliver submitted what he believed was
a better library than the standard library proposal, and so also didn't
want to lose what he considered important extra features. One stated
goal of the joint review was to guide cooperation and a possible merger.
However, we found that the review process does not do this job well.
>
>> ...
>
> As stated above, some libraries tackle fundamental problems and
> multiple approaches make sense. But imagine multiple BGL-related
> libraries, multiple asio-related libraries, multiple GIL-related
> libraries, multiple geometry libraries .. I think that would be a huge
> mess and as a user I would be discouraged!
And, if they don't each offer something important and useful that is
not available other places, I would expect the Boost community to reject
them. However, for different text parsing tasks we have a few different
libraries in Boost. They coexist exactly because they provide different
things for different use cases. The implication is that there is no one
correct way to parse text in all circumstances, but instead there are
ways that are well suited to different tasks.
I do not know if an analogous situation is true for computational
geometry, but I am not willing to exclude the possibility out of hand.
Instead, I want to look at libraries and see what they have to offer in
the context of what is already available.
>
>> ...
>
> I don't have a clear opinion on this. It looks like the Wizard has the
> most control over the process and maybe should be acknowledged to take
> some decisions on behalf of the community. If nobody owns to problem,
> nobody comes up with a solution.
The Wizards do make some decisions on behalf of the community, as do
the people in the other named roles (such as the release team, the
moderators, and others). However, in borderline cases the only way
anyone can say that the wrong choice was made in a review is by having a
deep grasp of the technical details. To say that this is the job of the
Wizards is equivalent to saying that the Wizards must be people who have
a deep grasp of the technical details for everything that appears in or
is proposed for Boost. No such people exist, and I sure am not one.
What is the other possibility?
The Wizards monitor the review process to catch any egregious
problems and try to solve them. This is already done, though we try to
be as low profile about it as possible. When there is a subtle problem,
the Wizards look at the presented technical details when they exist and
request advice from experts if needed. In the absence of technical
information, the Wizards engage in the conversation but have no basis
for taking any action.
>
> My main point is that boost has to make an effort to attract more
> libraries and make it easier for the new authors to get the libraies
> in (if the proposal makes sense!). Sometimes I find a great c++
> library and encourage the author to consider submitting it to Boost
> but they see the cost (docs, examples, ..) but don't see the benefits
> as clearly (and the benefits are there in terms of peer-review driven
> quality, ...).
I agree that there are many libraries that would be good additions to
Boost if they were willing to meet our quality standards. However, I
don't see how to make that easier. Producing high quality code that is
extensively tested, well documented, and provides instructive examples
is just a darn lot of work. I would be against any proposal that tried
to lower these standards to make it easier for new authors.
We also have the problem that the flow of incoming libraries is
already out pacing the flow of reviews. This is my candidate for the
biggest problem in the review process. It is happening for a few
reasons. One, we don't have enough qualified review managers
volunteering. Two, since producing a review takes time and work, we
can't schedule many reviews close together or the reviewer response
drops a lot.
This also affects scheduling parallel reviews for libraries that
address the same domain, since the scaling for producing a good review
is worse than linear in the number of libraries. Not only are there the
issues of the individual review for each, but there are also comparisons
and compatibility questions.
>
> Also, it would be useful to know what libraries people think are
> needed in boost. This could guide what needs to get in, rather than
> reviewing only everything that is submitted. I think many are
> interested in geometry-related domains so a FIX to the current
> situation is important.
>
> regards
The old wiki had a section for new library requests. I don't know if
the new one does, but you can check just as easily as I can. This might
provide a nudge to a developer who was considering producing a Boost
library (If the evidence of interest was already available.), but its
not lie we can assign someone to making a specific library. That just
isn't the way Boost is organized.
John
PS - If your goal in shouting FIX is to convince me of how important it
is, you are failing. I am much more persuaded by reasoned arguments and
technical details than by shouting.