Although run-time performance is rarely the most important reason for selecting APL, good performance often becomes important during the lifetime of an application (especially if it is successful and, therefore, has to deal with growing data volumes and numbers of users). Array-oriented programming naturally encourages Subject Matter Experts to use dense and pointer-free structures, which allow APL-based solutions to do things like balancing thousands of portfolios in a fraction of the time that more traditional solutions need.

The “set functions” – membership (∊), index of (⍳), intersection (∩), union (∪), and without (~) – are already some of the most highly-tuned primitives in the history of APL because they are critical to the performance of very many APL applications. New parallel instruction sets keep appearing in modern processors, and the balance between processor and memory performance is in a constant state of flux. Also, computer scientists continually improve algorithms for searching. Karta Kooner’s talk on the performance of Set Functions describes the approach that he is taking as we embark on another round of optimisations.

Sometimes, the best way to improve application performance or reliability is to split the application into multiple processes that can run independently and be scaled up by adding more processes as required. Apache Kafka is a widely-used tool for connecting such processes and reliably forwarding streams of messages between them. Stefan Kruger presents the benefits of Kafka and his initial experiments on what a Dyalog interface to Kafka might look like.

Application performance increasingly depends on how much memory you use, and how efficiently you move data around. If you want to help ensure that your APL algorithms have the best possible mechanical sympathy with Dyalog APL, Richard Smith’s “Introduction to the Workspace” will help you understand how the interpreter manages the memory that holds your arrays in the workspace.

An APL compiler promises to help APL users take advantage of highly parallel hardware like General-Purpose Graphics Processing Units (GPGPUs). If the compiler is self-hosted, it also makes it practical to quickly port APL to virtual environments, or base an APL implementation on other programming languages such as Python or JavaScript. Aaron Hsu presents an update on recent enhancements to the Co-dfns compiler, and plans for the near future.

Although I have been an APL developer for more than four decades, it is only recently that I understood how APL can be used to efficiently and elegantly handle tree structures using simple arrays. Brandon Wilson has been studying techniques developed by Aaron Hsu that make it possible to parallelise compilation of APL by the Co-dfns compiler. YAML parsers are notoriously difficult to write accurately, and Brandon hopes to find an effective description of YAML through APL that can help the community better understand its edge cases.

Next week the final videos from Dyalog ’23 will be published (along with my final blog post on the subject) – a great way to end the year!

——————————————

This week’s videos:

• D11: Dyalog + Kafka = True? (Stefan Kruger)
• D07: Future Performance Improvements in Set Functions (Karta Kooner)
• D06: An Introduction to the Workspace (Richard Smith)
• D15: Co-dfns Update 2023 (Aaron Hsu)
• U11: A YAML Parser in APL – Leveraging a Linear Dataflow Model to Unleash APL’s Expressiveness (Brandon Wilson)

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Follow