From: John Barnard (barnard_at_[hidden])
Date: 2001-01-06 00:47:55
Dave,
Dave Abrahams writes:
> John,
>
> Sorry about the delay in responding to this; for some reason I never
> got the mail message. I only saw it by chance when browsing boost
> messages on the web!
I wonder if I'm doing something wrong -- this is the second time
you've said that you didn't receive my message (I sent an unrelated
message last month). I sent both mailings to boost_at_egroups.com.
> I'm no expert on NumPy, but I would have thought that it had its own
> data representation. Is it possible to create a NumPy array that uses
> some external block of data? Even if this is possible, you run the
> risk of crashing the Python interpreter if the NumPy array is used
> after the DVArray is destroyed.
Yes, it is possible to create a NumPy array that uses some external
block of data, in this case, the data of valarray. If I increase the
reference count to the DVArray instance when I create the NumPy array
instance, the DVArray instance should exist as long as the NumPy array
instance exists.
> > if (arr == NULL) return NULL;
> > arr->flags |= OWN_DATA;
>
> Does the above line indicate to the NumPy array that it owns its
> data? If so, you are lying to it ;-) The data is still owned by the
> valarray.
No. The flag indicates that the NumPy array doesn't own the data and
shouldn't free the memory when the NumPy array is destroyed.
> >
> > How do I access the reference count of the wrapped DVArray instance?
>
> In what context?
>
> > I
> > would also like to gain access to the PyObject* of the wrapped
> > instance so I can make it the base for the NumPy array instance (and
> > subsequently have the reference count of the wrapped DVArray
> decrease
> > when the NumPy array is destroyed).
>
> Ah; I think I'm beginning to understand. You want the NumPy array to
> manage the lifetime of the DVArray, so that the former holds a
> reference count to the latter as long as it exists?
Not quite. I want to use the NumPy array interface as a convenient way
to manipulate in Python the data in DVArray. As I mentioned above,
NumPy allows you to create a NumPy array that doesn't own the
data. However, to ensure that the DVArray instance doesn't get
destroyed while the NumPy array still exists, I need to increase the
reference count of the DVArray-wrapped Python instance.
> I am not familiar with the internals of NumPy (and I can't seem to
> get to the manual over the web), so I can't tell you whether this is
> possible. The NumPy array would need to have some slot where you
> could store a pointer to the DVArray (if you only store the &self[0]
> you can't get back to the DVArray), and it would also need to give
> you a way to hook its destruction.
It has such a slot, base (a PyObject*), whose reference count gets decremented when
the NumPy array is deleted, which is exactly what I want.
> A DVArray object is-a PyObject, so you can always write:
>
> PyObject* to_array(DVArray& self) {
> PyObject* p = &self; // get the PyObject*
> Py_INCREF(p); // increment, decrement, whatever
This is exactly what I needed to know. I was confused about the usage
of DVArray in the to_array function. I thought DVArray referred to
the the valarray<double> class and not the wrapped valarray<double>
class. If I wanted a pointer to a valarray<double> (i.e., DVArray) and
not the wrapped valarray<double> class, how would I get it in this
function? In general, I'm confused about when a class is considered
wrapped and when it's not in the context of functions added as methods
of the wrapped class.
> In any case, Python usually doesn't care about the actual types of
> objects (just their interfaces), so you might find it much easier to
> build up the interface of DVArray so that it mimics a NumPy array and
> can be used in its place.
NumPy arrays have a rich interface and allows things like sin(x),
where x is an NumPy array, to be computed very quickly. To implement
that interface and functionality for valarray<> or vector<> would be a
large amount of work, hence, my use of the above strategy.
Thanks very much for your help and for BPL -- it's a fantastic piece
of work.
John
-- John Barnard Assistant Professor Department of Statistics Harvard University Phone: (617) 495-1603 Fax: (617) 496-8057 Email: barnard_at_[hidden]