APL at work in Africa

My dad is originally from the Cape, and I grew up in Botswana, so hopefully I can be forgiven for feeling that it is very cool to find Dyalog APL at the core of a project which aims to increase financial literacy and support the financial stability of small and medium-sized enterprises in South Africa, by delivering a “cash flow optimizer” on phones and tablets:

More Links:

CF04i Home Page
Article in the Financial Mail Corporate Essentials
Riskflow Blog

Elementary Cellular Light-Emitting Automaton

In response to an earlier post on driving an 8-bar LED using the Raspberry Pi, Roger Hui commented that 8 lights should be sufficient to display the output of a 1-d Game of Life. The code snippets displayed in this post are based on some working code that Roger was also kind enough to forward to me.

The following two functions provide the basic building blocks for the computation of the next generation of a 1-d cellular automaton:

bits←{(8⍴2)⊤⍵}           ⍝ Encode a number as 8 binary bits
neighbors←{↑¯1 0 1∘.⌽⊂⍵} ⍝ Map vector => 3-row matrix with neighbors

These allow us to perform the following transformations:

      bits 5
0 0 0 0 0 1 0 1
      bits 110
0 1 1 0 1 1 1 0
      neighbors b←0 1 0 0 1 1 1 1
1 0 1 0 0 1 1 1
0 1 0 0 1 1 1 1
1 0 0 1 1 1 1 0
      2⊥neighbors b ⍝ base-2 decode of each column
5 2 4 1 3 7 7 6

With these, it is now easy to define a function to compute the next generation of a pattern for a given Wolfram Code (and test it with a couple of arguments):

      wc←{(bits ⍺)[8-2⊥neighbors ⍵]}
      110 wc b
1 1 0 1 1 0 0 1
      110 wc 110 wc b ⍝ 3rd generation
0 1 1 1 1 0 1 1

We subtract the result of the base-2 decode from 8, because the least significant bit (bit 0) is the 8th element of the vector returned by the bits function. Next, we define an operator which returns a matrix containing a number of generations:

    ∇ r←gens(rule wcgm)pattern;i
[1] ⍝ Wolfram Code Generations
[2]
[3] r←(gens,⍴pattern)⍴pattern
[4] :For i :In 1↓⍳gens
[5]    r[i;]←rule wc r[i-1;]
[6] :EndFor
    ∇

    5 (110 wcg) b ⍝ 5 generations of code 110
0 1 0 0 1 1 1 1
1 1 0 1 1 0 0 1
0 1 1 1 1 0 1 1
1 1 0 0 1 1 1 1
0 1 0 1 1 0 0 0

Finally, we can add 1 to the boolean matrix and use it to index into a 2-element character vector, to get better “visualization” of the result (10 generations of Wolfram code 20):

      '-*'[1+10 (20 wcgm) b]
-*--****
-**-----
---*----
---**---
-----*--
-----**-
-------*
*------*
-*------
-**-----

The function also works with longer inputs – in the example below a random input of length 60 (20 generations of Wolfram code 18, this time using just blank and asterisk for the display):

     ' *'[1+20 (18 wcgm) 1=?60⍴2]
*****    * ***** *** ** *  * ***    *  *** *  ***  ** ***** 
     *  *                **     *  * **     **   **         
    * ** *              *  *   * **    *   *  * *  *        
   *      *            * ** * *    *  * * * **   ** *       
  * *    * *          *        *  * **        * *    *      
 *   *  *   *        * *      * **    *      *   *  * *     
* * * ** * * *      *   *    *    *  * *    * * * **   *    
              *    * * * *  * *  * **   *  *        * * *  *
*            * *  *       **   **    * * ** *      *     ** 
 *          *   ** *     *  * *  *  *        *    * *   *   
* *        * * *    *   * **   ** ** *      * *  *   * * *  
   *      *     *  * * *    * *       *    *   ** * *     **
* * *    * *   * **     *  *   *     * *  * * *      *   *  
     *  *   * *    *   * ** * * *   *   **     *    * * * **
*   * ** * *   *  * * *          * * * *  *   * *  *        
 * *        * * **     *        *       ** * *   ** *      *
    *      *      *   * *      * *     *      * *    *    * 
   * *    * *    * * *   *    *   *   * *    *   *  * *  * *
* *   *  *   *  *     * * *  * * * * *   *  * * * **   **   
   * * ** * * ** *   *     **         * * **        * *  * *

 

Finally, we can light the lights using the BarLED class from our GitHUB repository – and the Quick2Wire interface board and LED Bar – see the original post for an explanation. The video below shows the output resulting from the following APL expressions

      ]load /home/pi/BarLED
      bl←⎕NEW BarLED ⍬
      0.2 bl.Play ↓50(20 wcg) 0 1 0 0 1 1 1 1
      0.2 bl.Play ↓50(18 wcg) 0 1 0 0 1 1 1 1

Many thanks to Roger for the inspiration to take this one step further (and the code)! For a 2-dimensional implementation of Conway’s classic Game of Life, see the video by John Scholes at http://www.youtube.com/watch?v=a9xAKttWgP4.

Dyalog APL v14.0 Tech Preview

And now, as they say, for something completely different. Our robot is running Dyalog APL version 13.2 for the Raspberry Pi, which was released on our major platforms (Windows, Linux, AIX) in January 2013. The next release of Dyalog APL will be version 14.0, which currently has a tentative delivery date of “sometime in the 1st half of 2014”.

We are in the final stages of the design of a number of new language features, and would like to invite interested parties to install and test the “v14.0 Tech Preview” – or download and review the Release Notes. Some of the features may be controversial, so we welcome your feedback – whether you are a user of Dyalog APL or not. The key features that we hope to validate through the release of the preview include:

  1. New primitive operator rank (identical to the SHARP APL implementation and the ISO/IEC 13751 Extended APL Standard)
  2. New primitive operator key (similar to the J definition)
  3. New primitive function tally (identical to the J definition)
  4. Function trains (similar to trains of length 3 in J)
  5. An extension to dyadic index of (iota), to work on higher-rank arrays (for example, to search for rows of a matrix)
  6. An extension to mix with axis, providing greater compatibility with IBM APL2
  7. New user commands to control boxed / truncated session output
  8. Experimental changes to the binding of function definitions

A number of these new features were discussed by John Scholes and Roger Hui at the Dyalog’12 conference – the video can be viewed on the Dyalog Conference YouTube feed, and is also embedded below (but beware, some of the design details have changed since this recording was made):

“Tech Preview II”

The current invitation is in fact for the second release of the preview; the first release was distributed to a small number of well-known (and loved) troublemakers, and has already resulted in one important change to the definition of the proposed Key operator. I would like to thank Phil Last and Paul Mansour for feedback which has resulted in the operand function being passed the relevant key value as a left argument on each invocation.

Note to users of the first preview version: The default “loco” (1060 I-Beam) setting is 2 (trains, “late” binding) in this release, as this is the setting that we are most keen to have applications tested with. It was 0 (no trains, traditional binding) in the first preview.

How You Can Help

If you are able to commit to setting some time aside to test the new language features and providing us with feedback, please write to sales at dyalog dot com and let us know why YOU should be included in the evaluation team. It would be particularly valuable to us if you have time to run an existing application on v14.0; one of the proposed language extensions is a change to name binding, which will impact the behaviour of certain language constructs. Although most of these would be labelled “exotic”, there is a chance that they are used in existing applications, and we are very interested in learning about the degree to which these changes actually cause problems for existing applications. If serious “adverse reactions” are experienced, we may be forced to abandon some of the new features.

If you do not have time to install and test the preview, you are welcome to read the Release Notes and write to us with feedback, either in response to this blog post or by writing to language at dyalog dot com.

Thanks in advance to everyone who is able to participate!

 

Dyalog APL now available for the Raspberry Pi!

Although the news had not yet appeared on the Dyalog webpage when this was written, the CTO blog has access to exclusive sources and is therefore able to present this scoop: The big day has finally come – Dyalog APL version 13.2 is now available to anyone with a Raspberry Pi, and can be downloaded immediately from http://packages.dyalog.com! We will of course be making official announcements via various channels over the next few days, so keep an eye on our web page – but remember that you saw it here first!

In the above clip, you can see that the Dyalog C3Pi got a little over-excited: while celebrating (and testing) the new release with an autonomous drive on my kitchen floor (in the middle of dinner preparations), the robot got a bit too close to an obstacle and dragged the on/off switch along a pillow, switching itself off in the process! A single infra-red sensor doesn’t give much information for autonomous driving, but we have placed orders for a high-definition sonar, which should arrive next week. Stay tuned for further developments!

A Full Implementation of Dyalog APL

Note that, although the Pi version of Dyalog APL is free for educational and non-commercial use, is not technically restricted in any way – it has exactly the same features as any other 32-bit Linux-based (Unicode) version of Dyalog APL.

You are also welcome to take a look at the User Guide before installing the software – it also contains useful links to other resources that you can use to learn about Dyalog APL. If you would like to take a look at APL but do not (yet) have Raspberry Pi, educational and non-commercial licenses are also available for Linux/x86 and Windows – and you can also try APL online at http://tryapl.org (apologies in advance if you have a tablet, good support for tablets is coming soon to TryAPL).

Fun with APL on the Raspberry Pi (without a Robot)

The official release of Dyalog APL for the Raspberry Pi now looks as if it going to happen on Friday! In preparation for this, we have been working on some examples to demonstrate things you can do on your Pi without a set of wheels attached – like making lights blink!

 

Quick2Wire Interface Boards

It is possible to connect input and output devices directly to your Pi. However, we have elected to do our experiments using the interface boards from Quick2Wire – to be precise the “Port Expander Combo“, which protects you from damaging your Pi by making wiring/soldering mistakes, and makes a number of interfaces more easily accessible. The photo below shows the LED bar used in the above video, attached to the combo:

pi-with-quick2wire-boards

Raspberry Pi with Quick2wire expansion and bar LED boards

The LED Pattern Generation Language (LEDPGL)

The Bar LED has 8 controllable LED’s. APL is actually a neat language for generating boolean patterns, for example:

      3↑1    ⍝ "3 take 1"
1 0 0
      8⍴3↑1 ⍝ "8 reshape 3 take 1"
1 0 0 1 0 0 1 0
      ¯1⌽8⍴3↑1 ⍝ "negative 1 rotate" of the above
0 1 0 0 1 0 0 1 
      ⍪¯1 ¯2 ¯3⌽¨⊂8⍴3↑1 ⍝ "columnise the neg 1, 2 and 3 rotations..."
 0 1 0 0 1 0 0 1
 1 0 1 0 0 1 0 0
 0 1 0 1 0 0 1 0

Once you get into the habit of working with APL, you quickly learn to create small, functional “Embedded Domain Specific Notations” to work with your data. The LEDPGL namespace contains a small “pattern generation language”, which allows expressions like:

       LEDPGL.(0 4 shift 4 / 1 0)
 1 1 1 1 0 0 0 0   0 0 0 0 1 1 1 1
       LEDPGL.(show 0 4 shift 4 / 1 0)
 ⎕⎕⎕⎕....
 ....⎕⎕⎕⎕

Finally, the namespace contains a Demo function which takes a time interval in seconds as its argument (or 0 to use the “show” method to display the patterns in the session log). This is what was used to create the video at the top of this post:

       LEDPGL.Demo 0.1
 0 4 shift 4/1 0 ⍝ Left Right
 cycle 4 4/1 0   ⍝ Barber pole
 4 cycle 8 repeat head 4 ⍝ Ripple
 binary ⍳256             ⍝ Counter
 mirror cycle head 4     ⍝ Halves-in
 6 repeat 3 repeat x∨ reverse x←mirror cycle head 4 ⍝ Half-flip
 12 repeat ¯1↓x,1↓reverse x←mirror cycle head 4     ⍝ Out-In-Out

 

 

LED Morse Code

The Quick2wire expansion board (largest, closest board to the Pi in the first picture) has a single LED mounted, which can be controlled via GPIO. Our first example, which is included in the “Getting Started” guide which is included with Dyalog APL for the Pi – and available on GitHub – is an encoder for Morse Code. For example, the video below was generated by passing a character vector containing the only morse message that most people would be able to read:

GPIO.Morse 'SOS'

Visualising Sensor Data using APL on the Robot

As described in a recent post, our robot now has an Infra-Red distance sensor, which allows us to measure the distance from the front of the robot to the nearest obstacle. With respect to the autonomous navigation code that we wish to write, this will be the cornerstone! In order to evalute the perfomance of the sensor, we surrounded the robot with obstacles and commanded it to rotate slowly in an anti-clockwise direction, while IR data was collected 20 times per second:

RotatingC3Pi

Surrounded by obstacles, C3Pi rotates anti-clockwise and returns IR distance data every 0.05 seconds

Collecting the Data

We initialized the workspace by loading first the “RainPro” graphics package (which is included with Dyalog APL on the Pi), and then the robot code:

    )load rainpro                   Loads the graphics workspace
    ]load /home/pi/DyaBot           Loads the robot control code

The following function loops 300 times (once every 0.05 seconds), repeatedly collecting the value of the robot’s IRange property (which contains the current distance measured by the IR sensor). The call to the UpdateIRange method of the bot ensures that a fresh sample has just been taken (otherwise, the robot will update the value automatically every 100ms).

      ∇ r←CollectIRangeData bot;i;rc
 [1]    r←⍬
 [2]    :For i :In ⍳300
 [3]        :If 0=1⊃rc←bot.UpdateIRange
 [4]            r←r,bot.Irange
 [5]            ⎕DL 0.05 ⍝ Wait 1/20 sec
 [6]        :Else
 [7]            ∘∘∘ ⍝ Intentional error if update fails
 [8]        :EndIf
 [9]    :EndFor
      ∇

We can now perform the experiment as follows:

      iBot←⎕NEW DyaBot ⍬           ⍝ Instance of the robot class
      iBot.Speed←40 0              ⍝ 40% power on right wheel only 
      ⍴r←CollectIRangeData iBot    ⍝ Check shape of collected values 
300
      iBot.Speed←0                 ⍝ Let the robot rest its batteries
      1⍕10↑r                       ⍝ First 10 observations to 1 decimal 
15.3 12.8 13.7 12.7 13.7 11.4 9.9 11.1 10.4 9.4

Now that we have the data, the following function calls will create our first chart:

     ch.Set 'head' 'IR Sensor Data' ⍝ Set Chart Header
     ch.Set 'ycap' 'Distance (cm)'  ⍝     Y caption
     ch.Set 'xcap' 'Time (s)'       ⍝     X caption
     ch.Set 'xfactor'(÷0.05)        ⍝ Scale the x-axis to whole seconds
     ch.Plot data                   ⍝ Create the chart
     '/home/pi/irline.svg' svg.PS ch.Close ⍝ Render it to SVG

ir-sensor-data-raw

Removing the Noise with a Moving Average

The chart above suggests that the robot performed a complete rotation every 5 seconds or so with just under 100 observations per cycle. The signal seems quite noisy, so some very simple smoothing would probably make it easier to understand. The following APL function calculates a moving average for this purpose – it does this by creating moving sums with a window size given by the left argument, and dividing these sums by the window size:

       movavg←{(⍺ +/ ⍵) ÷ ⍺}  ⍝ Define the function 
       3 movavg 1 2 3 4 5     ⍝ Test it
 2 3 4

We can re-use the existing chart settings and plot smoothed data as follows:

      #.ch.Plot 7 movavg data      
      '/home/pi/irline.svg' svg.PS ch.Close

ir-sensor-data-smooth

Making Sense of It

The pattern is now nice and clear – but how does the map compare to the territory? We can use a “Polar” chart of the distance to see how the measured distances compare to reality:

     ∇ filename PolarDistance data;⎕PATH;mat;deg;window;smoothed;angles;movavg
[1]   ⍝ Polar IR Distance plot - "data" is one cycle of observations
[2]   ⍝ Note the -ve right margin to get the chart off-centre!
[3]
[4]    ⎕PATH←'#.ch'      ⍝ Using RainPro ch namespace
[5]    window←7          ⍝ Smoothing window size
[6]    deg←⎕UCS 176      ⍝ Degree symbol
[7]    movavg←{(⍺+/⍵)÷⍺} ⍝ Moving average with window size ⍺
[8]
[9]    angles←360×(⍳⍴data)÷⍴data ⍝ All the way round
[10]   smoothed←window movavg data,(¯1+window)↑data
[11]   data←(⌈window÷2)⌽data     ⍝ Rotate data so centre of window is aligned with moving average
[12]
[13]   Set'head' 'Infra-Red;Distance;Measurement'
[14]   Set('hstyle' 'left')('mleft' 12)('mright' ¯60)
[15]   Set'footer' 'Distance measured;every 0.05 seconds;while 3Pi was rotating'
[16]   Set'style' 'lines,curves,xyplot,time,grid,hollow'
[17]   Set'lines' 'solid'
[18]   Set'nib' 'medium,broad'
[19]   Set('yr' 0 60)('ytick' 10)
[20]   Set('xr' 0 360)('xtick' 15)('xpic'('000',deg))
[21]   Set('key' 'Measured' '7 MovAvg')('ks' 'middle,left,vert')
[22]
[23]   Polar angles,data,⍪smoothed
[24]   filename svg.PS Close  
     ∇

To align the chart with the picture, we need to:

  1. Extract the first 99 observations – corresponding to one rotation
  2. Reverse the order of the data, because the robot was rotating anti-clockwise
  3. Finally, rotate the data by 34 samples, to align the data with the photograph (the recording started with the robot in the position shown on the photograph)

We can do these three operations using the expression on the next line, and then pass this as an argument to the PolarDistance function, which creates another SVG file:

     onerotation←¯34⌽⌽99↑r
     '/home/pi/irpolar.svg' PolarDistance onerotation

ir-sensor-data-polar.jpgIf we compare the red line to the picture we started with, and take into account the fact that the robot was rotating quite fast and the IR sensor probably needs a little time to stabilise, it looks quite reasonable. The accuracy isn’t great, but with a little smoothing it does seem we should be able to stop little C3Pi from bumping into too many things!

Stay tuned for videos showing some autonomous driving, and the code to do it…