|lloda d047d34ab3 Notes on newra vs Guile built in vs various array srfis||4 months ago|
|bench||5 months ago|
|docs||4 months ago|
|examples||5 months ago|
|mod||4 months ago|
|sandbox||5 months ago|
|test||5 months ago|
|.travis.yml||2 years ago|
|LICENSE||5 years ago|
|README.md||4 months ago|
|TODO||4 months ago|
newra) wants to replace the current (3.0) Guile array system, which is almost entirely implemented in C.
The new implementation should be at least as fast. I think this is feasible once the Scheme compiler goes native, because for the most part the array functions are used to call back to Scheme, and a Scheme implementation could get rid of the back and forth, optimize the type dispatches, etc.
The C API shouldn't be affected. Once you get an array handle it makes no sense to use the Scheme array functions anyway, since the memory layout is transparent.
Except for the tests and for the pair of functions
newra is independent of the old array system.
newra tracks the Guile main branch as of 2021/12/03 and might not work with earlier revisions. Run the test or the benchmark with
> $GUILE -L mod test/test.scm > $GUILE -L mod bench/bench.scm
The manual is at (lloda.github.io/guile-newra), and you can find some larger examples in
To install the library, copy
mod/newra.scm to somewhere in your Guile load path, and use it with
newra can use
guile-ffi-blis for some functions (
ra-fill! in the current version), which can provide a considerable speed up for operations with arrays of types
c64 (it can also be slower -- there isn't a good heuristic yet).
newra will test at runtime whether
guile-ffi-blis is available.
The old array compatibility layer is mostly finished, with only a naming change (
newra versions of the
for-each functions are significantly faster already, but the
-set! functions are a bit slower, and some of the functions that have a fast path in C, such as
ra->list!, can be several times slower in
newra, depending on the types of the arguments.
These issues seem fixable, and besides, the Scheme compiler is only improving.
Compared with the old arrays,
newra offers a growing list of features:
(define ra (list->ra 2 '((1 2) (3 4)))),
(ra 1 1)returns
(set! ((make-ra #f 2 3) 1 1) 99)returns
#%2:2:3((#f #f #f) (#f 99 #f)).
ra-i). These may be infinite:
((ra-iota #f 1) (- #e1e20 1))returns
(ra-map! (make-ra 'x 2 3) + (ra-i 2 3) (ra-iota 2 0 10))returns
#%2:2:3((0 1 2) (13 14 15)). Prefix matching supports undefined dimensions; the previous expression and
(ra-map! (make-ra 'x 2 3) + (ra-i #f 3) (ra-iota #f 0 10))are equivalent.
(define I (ra-iota))and
(define J (ra-transpose (ra-iota) 1)), then
(ra-map! (make-ra 'x 10 10) * I J)is a multiplication table.
ra-amend!: index arguments can have any rank, and use of lazy index vectors (of any rank!) results in a shared array. A stretch index object
(dots)is supported; e.g.
(ra-from A (dots) 0)will produce the slice
A[..., 0]for an array of any rank.
newrais written entirely in Scheme, if a
newraoperation takes too long, you can actually interrupt it, which is not always the case in the old system.
Originally I wanted
newra to be a drop-in replacement for the old array system, reusing the same function names and all. Now I think it's better to have a parallel system where some of the flaws of old system can be cleaned up. Still it's important that programs can be easily ported to the new system.
With that in mind, here is what you'd have to change. Note that the
ra- names are not final, and neither is the
#% read syntax. I'm not sure yet how the old array syntax will be absorbed — maybe old array objects will be converted transparently for a while. Some of these are bugs that will eventually be fixed.
array->ra are provided to convert between the old and the new array types. Neither of these functions copy the contents of the array, so
(let ((o (make-array 3))) (eq? (shared-array-root o) (shared-array-root (ra->array (array->ra o))))) returns
#t. Note that some of the new
ra types aren't convertible in this manner; for example
(ra->array (ra-iota 3)) is an error.
The new system matches sizes strictly. For instance
(array-map! (make-array #f 2) - (make-array #t 3)) succeeds, but
(ra-map! (make-ra #f 2) - (make-ra #t 3)) fails with a shape mismatch error.
The new system still supports non-zero base indices, but I'd advise against using them, because they aren't worth what they cost and I'm tempted to get rid of them.
For most of the old functions
array-xxx, the equivalent function in
(array-copy! src dst)is
(ra-copy! dst src). This follows
ra-fill!which both use the first argument as destination.
ra- functions try to return something useful even when the corresponding
array- functions do not. For example
(array-fill! (make-array 0 3) 4) returns
(ra-fill! (make-ra 0 3) 4) returns
#%1:3(4 4 4).
The default writer defaults to printing all the sizes of the array, so
(ra-i 3 2) prints as
#%2:3:2((0 1) (2 3) (4 5)). Note that
#2:3:2((0 1) (2 3) (4 5)) is a valid read syntax in the old system, just not the default.
The read syntax is like that of the old system except for an extra
#2f64((1 2) (3 4)) becomes
#%2f64((1 2) (3 4)). The compiler doesn't support the new literal type yet. You can work around this using the reader like
(call-with-input-string "#%2f64((1 2) (3 4))" read) or say
(list->ra 'f64 2 '((1 2) (3 4))).
The default printer and reader don't handle undefined size arrays as well as they could. For example
(ra-transpose (ra-i 2) 1) prints as
#%2:d:2((0 1)), but this cannot be read back.
(ra-iota #f) prints as
truncated-print doesn't support
newra types, so you'll get a lone
# if truncation is necessary at all.
equal? doesn't support
newra types, so it does just
eqv?. Instead, you can use