The basic Lush type for tabular data is called an INDEX or an IDX. The IDX is merely an access structure, the actual data is stored in a STORAGE associated with the IDX.

The data contained in a storage, and accessed with an IDX, can be of type double, float, int, short, byte, ubyte, gptr (generic pointer), atom (lisp object), and packed (one-byte fixed point number).

There are many IDX-related functions in Lush. Some are described in this section, but the vast majority is described in the Standard Libraries section.

creating a tensor of float is done simply with:

  ? (setq m (float-matrix 10 8 4))   ; create 3D matrix
  = ::INDEX3:<10x8x4>
  ? (m 3 4 2 45.6)       ; set value of element (3,4,2) to 45.6
  = ::INDEX3:<10x8x4>
  ? (m 3 4 2)            ; get value of element (3,4,2).
  = 45.6

• double-matrix, double-matrix-nc, real-matrix, real-matrix-nc : doubles
  
• float-matrix, float-matrix-nc, flt-matrix, flt-matrix-nc : floats
  
• int-matrix, int-matrix-nc : 32-bit ints
  
• short-matrix, short-matrix-nc : 16-bit shorts
  
• byte-matrix, byte-matrix-nc : 8-bit bytes
  
• ubyte-matrix, ubyte-matrix-nc : 8-bit unsigned bytes
  
• gptr-matrix, gptr-matrix-nc : generic pointers (void*)
  

Multiple IDXs can point to the same STORAGE and provide access to the same data in multiple ways.

an IDX with 0 dimension is called a scalar, and can be seen as a pointer to a number.

  (setq m [3 4 5])

  (setq m [[1 2 3] [3 4 5]])

  (setq m [d 1 2 3])   ; doubles
  (setq m [f 1 2 3])   ; floats
  (setq m [i 1 2 3])   ; ints
  (setq m [s 1 2 3])   ; shorts
  (setq m [b 1 2 3])   ; bytes
  (setq m [u 1 2 3])   ; ubytes
  (setq m [a "choucroute" "garnie"])  ; lisp objects

  (setq m #["choucroute" "garnie" #])

  (idx-dotm0 vector [f@ 34] output)

  (setq s [d@ 42])
  (setq s [f@ 42])
  (setq s [i@ 42])
  (setq s [s@ 42])
  (setq s [b@ 42])
  (setq s [u@ 42])
  (setq s [a@ "choucroute"])
  (setq s #[@ "choucroute" #])

  ? (de foo () (let ((v [0 1])) (v 0 (+ (v 0) 1)) v))
  = foo
  ? (foo)
  = [  1.00  1.00 ]
  ? (foo)
  = [  2.00  1.00 ]

  (de foo () (let ((v (idx-copy [0 1]))) (v 0 (+ (v 0) 1)) v))

  ? (de new-counter () (lambda () (let ((v [@ 0])) (v (1+ (v))) (v))))
  = new-counter
  ? (setq c (new-counter))
  = ::DE:8805898
  ? (c)
  = 1
  ? (c)
  = 2

More precisely, each si will be an idx with one less dimension than the corresponding idxi , and will simulataneously loop over the successive "slices" of idxi for each possible value of the first index. In other words, applying function myfunc to each element of a vector v1 and putting the result in the corresponding element in v2 can be done with:

  (idx-bloop ((x1 v1) (x2 v2)) (x2 (myfunc (x1))))

This is like idx-bloop, but it loops on the last dimension, instead of the first. For example, the matrix product operation C = A*B can be written as follows:

  (de idx-m2timesm2 (A B C) 
     (idx-eloop ((Bj B)(Cj C)) (idx-m2dotm1 A Bj Cj)))

A call to cidx-bloop as shown in the synopsis is somewhat equivalent to n nested idx-bloop s, which will loop over the first n dimensions of idxs l_1 to l_m simultaneously. The arguments i_1 to i_n are strings containing names of C local variables that will be created and set to the loop index in each of the n dimensions. At each iteration, the C variables provided in strings c_1 to c_m will point to the appropriate values in the idxs l_1 to l_m . For example, the following function will fill matrix a with cos(i+j).

    (de foo (a)
       ((-idx2- (-flt-)) a)
       (cidx-bloop ("i" "j" ("a" a)) #{ *a = cos(i+j); #}) a)

Example:

  ? (idx-dim [[2 3 4][5 6 7]])
  = (2 3)

  ? (setq m [[0 1 2 3 4][10 11 12 13 14]])
  = [[ 0.00  1.00  2.00  3.00  4.00 ]
     [10.00 11.00 12.00 13.00 14.00 ]]
  ? (select m 1 2)
  = [ 2.00 12.00 ]

  ? (setq m [[0 1 2 3 4][10 11 12 13 14]])
  = [[ 0.00  1.00  2.00  3.00  4.00 ]
     [10.00 11.00 12.00 13.00 14.00 ]]
  ? (narrow m 1 3 2)
  = [[ 2.00  3.00  4.00 ]
     [12.00 13.00 14.00 ]]

  ? (setq v [0 1 2 3 4 5 6])
  = [ 0.00  1.00  2.00  3.00  4.00  5.00  6.00 ]
  ? (setq k [1 2 1])
  = [ 1.00  2.00  1.00 ]
  ? (setq uv (unfold v 0 3 1))
  = [[ 0.00  1.00  2.00 ]
     [ 1.00  2.00  3.00 ]
     [ 2.00  3.00  4.00 ]
     [ 3.00  4.00  5.00 ]
     [ 4.00  5.00  6.00 ]]
  ? (setq r (matrix 5))
  = [ 0.00  0.00  0.00  0.00  0.00 ]
  ? (idx-m2dotm1 uv k r)
  = [ 4.00  8.00 12.00 16.00 20.00 ]

  ? (setq uv (unfold v 0 3 2))
  = [[ 0.00  1.00  2.00 ]
     [ 2.00  3.00  4.00 ]
     [ 4.00  5.00  6.00 ]]
  ? (setq r (matrix 3))
  = [ 0.00  0.00  0.00 ]
  ? (idx-m2dotm1 uv k r)
  = [ 4.00 12.00 20.00 ]

  ? (unfold [3 4 5 6 7] 0 5 1)
  = [[ 3.00  4.00  5.00  6.00  7.00 ]]
  ? (unfold [3 4] 0 1 1)      
  = [[ 3.00 ]
     [ 4.00 ]]

  (idx-transclone m '(0 2 1))

  ? (setq m [[[111 112 113][121 122 123][131 132 133]] 
             [[211 212 213][221 222 223][231 232 233]]])
  = [[[111.00 112.00 113.00 ]
      [121.00 122.00 123.00 ]
      [131.00 132.00 133.00 ]]
     [[211.00 212.00 213.00 ]
      [221.00 222.00 223.00 ]
      [231.00 232.00 233.00 ]]]
  ? (diagonal m 2)
  = [[111.00 122.00 133.00 ]
     [211.00 222.00 233.00 ]]

  ? (setq m (matrix 3 4))
  = [[ 0.00  0.00  0.00  0.00 ]
     [ 0.00  0.00  0.00  0.00 ]
     [ 0.00  0.00  0.00  0.00 ]]
  ? (m () () (range 0 11))
  = [[ 0.00  1.00  2.00  3.00 ]
     [ 4.00  5.00  6.00  7.00 ]
     [ 8.00  9.00 10.00 11.00 ]]
  ? (idx-changedim m 1 3)
  = ()
  ? m
  = [[ 0.00  1.00  2.00 ]
     [ 4.00  5.00  6.00 ]
     [ 8.00  9.00 10.00 ]]

Argument file may be a filename string or a file descriptor created with open-read . When file is a file name, a suffix ".mat" is added when needed. Binary Matrix File Format. Ascii Matrix File Format.

The first line of the file is a matrix header. It is composed of the letters ``.MAT'', the number of dimensions and the size of each dimension. The array elements are written then, separated by spaces and newlines characters.

The following lines display the content of a valid ascii matrix file.

        .MAT 2 3 4
         1 -1  1  2  -1  1 -3
         1 -1
         4  0  0

        .MAT 2 3 4
         1 -1  1
         2 -1  1
        -3  1 -1
         4  0  0

The core header is a C structure defined as follows :

struct header {
        int magic;
        int ndim;
        int dim[3];
};

The first member, magic , of this structure is a ``magic'' number, which encodes the type of the matrix. This number must be:

• 0x1E3D4C51 for a single precision matrix
  
• 0x1E3D4C53 for a double precision matrix
  
• 0x1E3D4C54 for an integer matrix
  
• 0x1E3D4C56 for a short matrix
  
• 0x1E3D4C55 for a byte matrix
  
• 0x1E3D4C52 for a packed matrix
  

The second member of this structure, ndim , is the number of dimensions of the matrix. Then come the dimensions themselves in the array dim . If there are more than three dimensions, the array dim is extended to accomodate the extra dimensions.

When the number of dimensions (specified in ndim ) is greater than 3, the header is completed by ndim - 3 further integers.

This header is followed by a binary image of the matrix. Elements are stored with the last index changing faster, i.e.

(0,0,0) (0,0,1) (0,0,2) ...  (0,1,0) (0,1,2) etc...

• as float numbers for single precision matrix
  
• as double numbers for double precision matrix
  
• as int numbers for integer matrix
  
• as short numbers for short matrix
  
• as unsigned char numbers for byte matrix
  
• and as char for packed matrix.
  

In this latter case, each byte represents a fixed point number between -8 and +8 (+8 not included), the first 4 bits (most significant nybble) contain the integral part and the remaining 4 bits contain the fractional part. It uses a two's complement format. Here are two C functions which convert a ``compacted fixed point'' number into a floating point number and back.

/* Converts a packed number to a float */
float unpack(b)
int b;
{
    if (b & 0x80)
        b |= ~0x7f;
    else
        b &= 0x7f;
    return (float)b / 16.0;
}

/* Converts a float into a single byte packed number */
unsigned char pack(x)
float x;
{
    if (x > 8.0-1.0/16.0)
        return 0x7f;
    else if (x < -8.0)
        return 0x80;
    else
        return x*16;
}

Argument mat must be a matrix freshly created with functions matrix , dmatrix , imatrix , smatrix , bmatrix or pmatrix . Submatrices are usually rejected. Argument file might be a file name or a file descriptor created with open-read . When argument offset is specified, a header of offset bytes is skipped.

After executing this function, the file descriptor file points to the first byte following the matrix data.

After executing this function, the file descriptor file points to the first non blank character following the matrix data.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;