Category Archives: Videos

Dyalog ’18 Videos, Week 5

Posted on by

We are releasing four talks this week from Dyalog ’18 in Belfast. Three of them focus on techniques for deploying APL applications across all the platforms supported by Dyalog APL, in particular Microsoft Windows, Apple macOS, and GNU Linux on x64 platforms (many tools will also work under IBM AIX and on the Raspberry Pi).

If that isn’t your sort of thing, we invite you to relax with Dr. Charles Brenner’s entertaining talk on The Manly Chromosome – How Simplicity can be Confusing. Charles is a “purveyor of forensic mathematics”; he participates in CSI (Crime Scene Investigation) using mathematics. When he actually needs to compute a number for a court case, he often does it in APL (did you know how useful ×.! is?). Although the underlying subject is a serious matter – the mis-interpretation of DNA data related to the Y Chromosome in criminal cases – the talk is full of humour, although you will need to concentrate as some of jokes are rather understated!

Returning to the theme of the week, my own second presentation follows on from my opening keynote. In the keynote, I whizzed through a flurry of scenarios in which a new user of APL was able to run, edit and debug APL code under Linux without actually installing APL, and then deploy the code as a Web Service using JSONServer, and finally as a Web Application using MiServer. The keynote intentionally did not explain much about the technology. In this week’s talk, entitled Cloud Computing, I reveal some of the details of how the smoke was generated and the mirrors were polished for the opening talk.

As mentioned, Dyalog APL is available on a variety of platforms. Our goal is to allow you to freely select a platform for development, and independently select a platform for deployment. For example, you might develop and test the application under Windows or macOS, and deploy it to the cloud under Linux. If your application has a user interface, you may be interested in Brian Becker’s talk on Cross Platform User Interfaces, where he presents a project with the internal code name “DUI” (Dyalog User Interface). We’ve been working on DUI for several years as a component of our Web Server framework MiServer. Brian demonstrates the results of this endeavour: the new HTMLRenderer, which embeds the Google Chrome Browser Engine (known as Chromium) inside Dyalog APL for Windows, macOS and Linux. The same code can now be used for MiServer and HTMLRenderer, allowing it to drive UI without a Web Server using the same principles as Node JS.

If your application is – or can expose – an engine without a user interface, one of the options for deployment is to run it as a Web Service. This makes it callable from anywhere that can connect to it via a network, for example using the JSONServer as demonstrated in my opening keynote. Marko Vranic’s talk on Severless APL demonstrates the latest fashion in deployment technology, where individual functions – rather than applications – can be deployed to the cloud (or a collection of internal servers). Obviously, the name is a little misleading since the functions do run on servers. The point is that you do not need to define a new server or even a container for your application; once the Kubeless environment is set up, developers can continuously deploy new or modified functions into the “serverless” framework from the command line. The framework takes care of encryption of network traffic, allows you to control access to your functions, and lets you manage scaling and load balancing. This talk probably contains a bit more technology and more moving parts than many APL users will be comfortable with, but the important point that Marko makes is that we have all the pieces required to use APL in scenarios where extreme security and scalability are a requirement.

Summary of this week’s videos:

Email, RSS Follow

Dyalog ’18 Videos, Week 4

Posted on by

This week is mostly a deep dive into the new world of storing source code in text files rather than workspaces and other “binary” formats. However, in case that is not your cup of tea yet, we can offer you another talk by Marshall Lochbaum, who presents more amazing algorithms to make the very widely used primitive search functions ∊, ⍳ and ⍸ run faster than ever before. By combining non-branching algorithms with vector instructions and a technique known as Robin Hood Hashing, Marshall is able to drive a modern CPU close to the theoretical maximum throughput, and in many cases spend less than one nanosecond searching for each item of an array.

Source code in text files is the dominant theme, and we are fortunate enough to have three pioneers to show us the way: Paul Mansour, Gilgamesh Athoraya and Kai Jaeger.

Paul has been working on – and using – source code management systems for decades. Recently, his team have implemented a lightweight version of the Acre project management system, named Acre Desktop, based entirely on textual source files. Apart from having to start your day by ]Open-ing a project, rather than by )LOAD-ing a workspace, there are very few changes to how you would actually use APL – but now you have access to a huge collection of professional tools developed for programmers using other programming languages, such as GitHub.

One of the very significant advantages of the APL community starting to use common structures for source code – and projects – is that it becomes realistic to share tools and utilities.
Following on from Paul’s talk, Gilgamesh Athoraya demonstrates a prototype of an APL Package Manager (APM). The APM connects to a repository of packages written in APL and allows you to declare package depenedencies from a public or private repository. It also keeps tabs on the availability of new versions of dependencies, and allows you to easily update them when the time is right.

A package manager can only be successful if there are packages to be managed. Kai Jaeger has been an APL Toolsmith for a very long time, and made much of his work available via the APLTree. Now, Kai has transferred the contents of the APLTree to GitHub, making everything available as textual source. With a bit of luck, once the APM finds its legs, we’ll all be able to use Acre Desktop to define projects, Git[Hub/Lab] to manage the source, and APM to search for Kai’s tools and manage our dependencies on them!

Summary of this week’s videos:

Email, RSS Follow

Dyalog ’18 Videos, Week 3

Posted on by

The four presentations from Dyalog’18 that we are releasing this week address both the visible (user interface) and invisible (performance) parts of application design. Starting with performance:

“You don’t have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy.” – former Formula One racing driver Sir John Young “Jackie” Stewart, OBE


This quote was at the heart of the talk by our invited keynote speaker Martin Thompson. In order to write software which performs well, you need to have a basic understanding of how the underlying machinery works. Understanding basic mathematical models for the theoretical throughput of software and hardware helps us take the step from being alchemists to scientists, as we endeavour to write high-performance systems.

Martin takes us for an entertaining stroll through the evolution of modern processors, and some of the maths behind high performance systems. The good news is that systems which make sequential and predictable memory accesses are likely to find sympathy with modern hardware…

Marshall Lochbaum, the most recent addition to the core interpreter team at Dyalog, followed up with a talk on a number of his ideas for increasing the mechanical sympathy of Dyalog APL, to take maximum advantage of branch prediction and other features of modern processors. Some strategies take advantage of runtime inspection of the arguments, something that is more natural in an interpreter with the ability to dynamically select data types, as opposed to strongly typed strategies typically employed by compilers.


TamStat is an application which helps students Tame Statistics. In two talks at Dyalog’18, Stephen Mansour and Michael Baas focus on two different aspects of the user experience. In the first talk, Stephen focuses on the notation available to users of TamStat. Where many statistical libraries contain dozens of strangely named functions with a variety of switches and parameters, TamStat uses a small set of functions, combined with another small set of operators, to provide a very simple but extremely elegant notation for computing probabilities based on a wide variety of distributions. For example:

⍝ Probability that 7 coin flips (0.5 specifying a "fair" coin) will result 
⍝ in at least 3 heads:
7 0.5 binomial probability ≥ 3

⍝ Probability that a number from a normal distribution with a mean of 0 and 
⍝ standard deviation of 1 will be ≤ 3:
0 1   normal   probability ≤ 3

I almost wish I could go back to University and start Statistics 101 again 😊.


Notation is a powerful tool of thought, but graphs make it easier to visualise the results. Following Stephen’s talk, Michael Baas describes work that Dyalog is doing in collaboration with Stephen, with the goal of wrapping TamStat in a modern, HTML/JavaScript based frontend. Current TamStat is based on the ⎕WC (Window Create) library function and is therefore restricted to running on Microsoft Windows. However, many of Stephen’s students use Mac or Linux laptops. The new interface also makes it possible to run TamStat as a web-based service with a web site. We expect that this work will make TamStat accessible to a much wider audience.

Summary of this week’s videos:

Email, RSS Follow

Dyalog ’18 Videos, Week 2

Posted on by

Each week until early January, we will be releasing a selection of recordings of presentations from Dyalog’18, which was held in Belfast at the end of October 2018. Last week we kicked off with the opening keynote talks and the prize ceremony and acceptance speech by the winner of our annual problem-solving competition.

Just under half of the presentations at Dyalog User Meetings are by users who have volunteered – or sometimes been commandeered – to share stories about how they have used APL for fun or profit. These user stories provide significant motivation to the Dyalog team for future direction.

Aaron Hsu’s talk on “High Performance Tree Wrangling, the APL Way” is a pearl. Back in 2015 I gave a talk at Google on APL. One of the Google engineers asked about working with trees in APL and I was unable to give him a useful answer. Aaron is working on a compiler for APL, and trees that represent the code that is being compiled are his most important type of data structure.

In this talk Aaron demonstrates that APL is an elegant – and highly efficient – notation for working with trees, if you just pick the right representation!


Most of the talks at Dyalog User Meetings are fairly technical. The subject at the core of Ilaria Piccirilli’s talk – the fair pricing of financial instruments and subsequent evaluation of portfolios – is no exception. Mercifully, Ilaria spares us the details of the calculations – as she dryly notes, there is no “Fair Pricing for Dummies”. Instead, she offers humorous insights into the way her team used APL to deal with the explosion of computations required by regular additions to legislation requiring health checks – and the day that negative interest pulled the rug out from under most standard pricing calculations.

The other, slightly larger half of the talks at Dyalog Users meetings are by members of the Dyalog Team, talking about work that has recently been done on our products or presenting designs for future enhancements.


Adám Brudzewsky’s talk, titled Array Notation Mk III, is about a potential future extension to the APL language, which will make it possible to easily and clearly describe arrays of high rank, or with deeply nested structure, without using APL primitives to “construct” them, as is common practice today. In addition to making application code easier to read and write, a literal notation for data structures will make it easy to use text files to describe data structures which are essentially part of the source code of an application, and should be managed by a source code management system. As the name suggests, this work has been ongoing for some time, with the initial inspiration coming from a user presentation by Phil Last, back at Dyalog ’15 in Sicily. Watch the presentation and give us feedback on whether you think this idea is now sufficiently baked to become part of Dyalog APL, or we’ll need a “Mk IV” talk next year!


With the growth in usage of Dyalog APL under macOS and Linux – especially in server or cloud environments – the Dyalog Remote Integrated Development Environment is becoming a “mainstream” tool, rather than the curiosity that it was during the first few years of development. Our partners at Optima Systems are developing RIDE on Dyalog’s behalf, and Gilgamesh Athoraya is now the lead developer. In his talk on “RIDE 4.1 and Next Generation Integrations”, Gil talks first about significant new features and performance improvements to RIDE in 4.1 – and then continues to talk about how components of the RIDE technology may be re-purposed to provide APL add-ins for popular development frameworks like the new Microsoft VS Code.

Summary of this week’s videos:

Email, RSS Follow

Welcome to the Dyalog ’18 Videos!

Posted on by

Three weeks have gone by since we waved goodbye to the last Dyalog ’18 delegates in Belfast. We’ve had time to catch up on sleep, half of us have had colds and recovered from them. Jason Rivers and Richard Park have started mixing and improving the audio and video recordings, and we are ready to release the first group of processed videos.


Our plan is to release batches of 3-5 videos, with enough variety for everyone to find at least one topic of interest each week. We have not reviewed all of the material yet; there are always one or two where something went wrong and we are unable to publish the recordings (or the presenter asks that we refrain from making the talk public), but we do expect to be able to make the vast majority of the talks available over the next couple of months.

Each week, I’ll be doing my best to introduce each set with a blog entry: The first batch contains cleaned-up versions of the presentations that were streamed live from Belfast. The audio and video quality is significantly enhanced compared to the live stream, and the most confusing gaffes in my own live demo have been removed 😊.


As usual, the user meeting opened with the traditional trio of keynotes by Dyalog’s CEO Gitte Christensen, the CXO Morten Kromberg (that’s me) and CTO Jay Foad. Gitte introduces a couple of new faces at Dyalog, and the contest winners, so everyone can plan to buy the winners drinks during the week. Gitte then discusses high level direction – announcing our intention to make the Linux version available to download, and included in public Docker containers and Cloud VM images, with no questions asked.


My own session mostly consists of a live demo of the potential consequences of making Linux licences really easy to get hold of. In an imaginary conversation with a data scientist, I demonstrate the use of Dyalog APL to implement an (admittedly silly) analytical function, and subsequently make it available as a web service and via a web site, finally deplying it to the cloud using a set of Public Docker containers, without once installing Dyalog APL itself.


Jay Foad rounded Monday’s live stream off with a review of the features of the recently released version 17.0, before moving on to talk about the work that the development team is planning for versions 17.1 and 18.0, scheduled for the spring of 2019 and 2020, respectively.


In accordance with tradition, we also streamed the Prize Ceremony for the International Problem-Solving Competition and – often the most interesting talk of the year – the acceptance speech where this year’s winner talked about his code, and the experience of learning APL. This year’s winner did not let us down; it is amazing how quickly you can learn to write really, really good APL code!

Summary of this week’s videos:

Email, RSS Follow

Response to Feedback on Cut, Under and Merge

Posted on by

At Dyalog ’15 John Scholes and I presented proposals for future operators cut, under, and merge. Following the release of the video of this presentation, we received some feedback from a user. Our response to the feedback may be of wider interest.

  1. It’s early days yet for cut, under, and merge (they are tentatively planned for release in version 16.0), and one of the reasons for that is so that we can receive feedback from the user community. So thank you for your comments.
  2. We take our inspiration from many sources, and while it’s true that J has cut, under, and merge, cut saw the light of day in Rationalized APL (1983), under in Operators and Functions (1978), and merge in Rationalized APL (1983) again. And even before that, the ideas have been in mathematics or functional programming for many years.Wherever the inspiration came from, we don’t take them just on faith, just because they are in J.Even now, even in these early days, cut, under, and merge have undergone a vigorous internal debate. The discussion is informed by the experience in other contexts, J or otherwise.
  3. We are interested in what you think are the “few fundamental ideas expressed by a handful of mathematical symbols”, what you say is the definition of APL. From our perspective we have not violated and will not violate what we think are the fundamental ideas. But perhaps our set of fundamental ideas differ from yours?
  4. No one here is thinking of adopting the J spelling scheme (with the dots and colons).
  5. The Dyalog development team are in unanimous agreement that dfns, ⍺⍵-functions, are A Good Thing. As we see it, the recent and proposed additions to Dyalog APL have only enhanced the elegance and utility of dfns. For example, {(+⌿⍵)÷≢⍵}⍤r to compute the mean of not just vectors but subarrays of a specified rank, {⍺⍵}⌸ to compute the unique keys and the corresponding indices, {⍺(+⌿⍵)}⌸ to compute the unique keys and corresponding sums, and {⊂Cut ¯2⊢⍵,','}Cut ¯2 ⊢x~⎕av[4] to partition CSV x into its columns and cells.
  6. Cut, under, and merge are described and documented in discussion papers accompanying the video. If you have not already read them, they can be found in the Dyalog ’15 webpage and specifically here. Please tell us what more can be done as far as documentation, and what parts specifically are unclear.
  7. We have previously (and recently) examined the question of adding quaternions to Dyalog. Currently there are no plans to add them.
  8. Finally, examples of under from everyday life. Of course the processes involved can be composed, so that for example “dinner” can be: 
    open the fridge
    take the food
close the fridge

    but it can also be a more elaborate:

    take plate from cupboard
    open the fridge
        take the food
    close the fridge
 
    put food on plate
        open mouth
            put food in mouth
        close mouth
    clean plate
put plate back into cupboard

    As remarked in the Dyalog ’15 presentation, once you are attuned to it you can see under everywhere, sometimes in subtle ways: Ashes to ashes, dust to dust.

Email, RSS Follow