Whether you are celebrating a winter holiday, looking forward to the days getting longer from tomorrow, or enjoying summer south of the equator, we hope you have time to enjoy the final collection of light-hearted presentations from Dyalog ’23.

Andy Shiers has overall responsibility for making sure that Dyalog gets correctly built and tested before it reaches the users – and that it is supported once it has been released. In his “Fireside Chat”, Andy covers a wide range of topics that he regularly encounters but feels might have not been sufficiently emphasised in other talks or in our documentation – and that you might need to know about. Andy is back by popular demand from user meeting delegates, after a gap of a few years.

The development team at SimCorp Italiana have been regular contributors to Dyalog user meetings, with insightful and amusing anecdotes about the relationship between humans who work with technology. This year, in “Once Upon A File”, the stories are mostly about the trials and tribulations of importing data.

Ray Cannon has been working on synthesising music on various computing devices. From a humble start where he tapped notes out himself using “A Pointy Stick”, he learned how to generate chords, add harmonics, attack, decay, and reverberate – and store the result in .WAV files. The result is a rendering of J.S. Bach’s Tocatta and Fugue in D Minor, BMV565 – complete with animations – on multiple organ pipes, all done in APL.

We hope that you enjoyed the presentations from Dyalog ’23, that you have a Happy New Year ahead, and that we will see many of you at Dyalog ’24 in Glasgow (15-19 September 2024).

——————————————

This week’s videos:

• U07: Once Upon A File (Elena Paviotti and Gianna Folladore)
• U04: Music: APL vs APS (A Pointy Stick) (Ray Cannon)
• D14: The Return of Uncle Andy’s Fireside Chat (Andy Shiers)

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Follow

Although run-time performance is rarely the most important reason for selecting APL, good performance often becomes important during the lifetime of an application (especially if it is successful and, therefore, has to deal with growing data volumes and numbers of users). Array-oriented programming naturally encourages Subject Matter Experts to use dense and pointer-free structures, which allow APL-based solutions to do things like balancing thousands of portfolios in a fraction of the time that more traditional solutions need.

The “set functions” – membership (∊), index of (⍳), intersection (∩), union (∪), and without (~) – are already some of the most highly-tuned primitives in the history of APL because they are critical to the performance of very many APL applications. New parallel instruction sets keep appearing in modern processors, and the balance between processor and memory performance is in a constant state of flux. Also, computer scientists continually improve algorithms for searching. Karta Kooner’s talk on the performance of Set Functions describes the approach that he is taking as we embark on another round of optimisations.

Sometimes, the best way to improve application performance or reliability is to split the application into multiple processes that can run independently and be scaled up by adding more processes as required. Apache Kafka is a widely-used tool for connecting such processes and reliably forwarding streams of messages between them. Stefan Kruger presents the benefits of Kafka and his initial experiments on what a Dyalog interface to Kafka might look like.

Application performance increasingly depends on how much memory you use, and how efficiently you move data around. If you want to help ensure that your APL algorithms have the best possible mechanical sympathy with Dyalog APL, Richard Smith’s “Introduction to the Workspace” will help you understand how the interpreter manages the memory that holds your arrays in the workspace.

An APL compiler promises to help APL users take advantage of highly parallel hardware like General-Purpose Graphics Processing Units (GPGPUs). If the compiler is self-hosted, it also makes it practical to quickly port APL to virtual environments, or base an APL implementation on other programming languages such as Python or JavaScript. Aaron Hsu presents an update on recent enhancements to the Co-dfns compiler, and plans for the near future.

Although I have been an APL developer for more than four decades, it is only recently that I understood how APL can be used to efficiently and elegantly handle tree structures using simple arrays. Brandon Wilson has been studying techniques developed by Aaron Hsu that make it possible to parallelise compilation of APL by the Co-dfns compiler. YAML parsers are notoriously difficult to write accurately, and Brandon hopes to find an effective description of YAML through APL that can help the community better understand its edge cases.

Next week the final videos from Dyalog ’23 will be published (along with my final blog post on the subject) – a great way to end the year!

——————————————

This week’s videos:

• D11: Dyalog + Kafka = True? (Stefan Kruger)
• D07: Future Performance Improvements in Set Functions (Karta Kooner)
• D06: An Introduction to the Workspace (Richard Smith)
• D15: Co-dfns Update 2023 (Aaron Hsu)
• U11: A YAML Parser in APL – Leveraging a Linear Dataflow Model to Unleash APL’s Expressiveness (Brandon Wilson)

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Follow

An increasing number of APL systems serve business logic as services, in addition to providing a user interface. Some recent APL applications have no user interface at all, and are only available as services.

For this reason, Dyalog’s web service framework, Jarvis, features prominently in Brian Becker’s overview of the current state of tool development at Dyalog Ltd. Brian is the APL Tools Architect, and the author of both Jarvis and HttpCommand, our tool for making web requests (Jarvis provides server functionality, while HttpCommand provides client functionality).

Jarvis is the foundation of many new APL projects, two of which are featured this week. First, Finn Flug from Dittrich & Partner Consulting GmbH (DPC) tells the story of an application that started its life as a COM service implemented in APL+Win, was converted to a web service using Jarvis, and was then deployed as a docker container. Docker containers are also the execution platform for Claus Madsen’s talk. For decades, Claus has been writing software in APL for pricing financial instruments; he is one of Denmark’s leading experts in this field. Claus is not a software engineer, so when a new client wanted a pricing application as a web service, he needed a little help from his friends at Dyalog Ltd. He wanted to replace the binary workspaces that he had been using to hold his source code with docker containers, and for these docker containers to be rebuilt and published on Amazon Web Services every time he committed a change to one of his APL source files (managed on GitHub). To his surprise, since making this change he has found that he feels more in control of his APL development workflow than ever before – without having lost any flexibility.

The only talk that doesn’t mention Jarvis is Stig Nielsen’s talk on “Worker Bees”, where he demonstrates a mechanism which is similar to Dyalog APL’s isolates. The difference is that where the isolate workspace launches separate processes to host isolates, Stig makes use of a .NET class that allows a single process to launch multiple Dyalog interpreters that all run within the same process. This mechanism has only been made available to selected clients because some features of Dyalog (like file holds and locks or the use of DLLs) are restricted in this mode – but it has the advantage that data can be shared in-process, between multiple APL interpreters and components written in C#.

——————————————

This week’s videos:

• D09: Dyalog Tools Update (Brian Becker)
• U05: Converting a COM Server to a Jarvis-based Web Service (Finn Flug)
• U01: APL Worker Bees – Another Way to Run APL Tasks in Parallel (Stig Nielsen)
• U06: Dyalog, AWS, Jarvis, Docker…What’s Not to Like? (Claus Madsen)

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Follow

This week, the focus is on the use of tools and interfaces in applications. Mark Wolfson from BIG integrates data from 1,000 retailers to provide business intelligence to both the retailers and manufacturers. Over the last couple of years, Mark has migrated his application from IBM APL2 to Dyalog APL. He tells the story of how Dyalog’s tools and features (including SQAPL, Jarvis, HttpCommand, .NET integration, ⎕JSON, and ⎕CSV) have enabled BIG to respond to customer needs faster and more flexibly than ever before.

Kai Jaeger has a long history of developing tools and utilities for Dyalog APL. He is also the main developer of the new Tatin Package Manager. In addition to developing the package manager itself, he has been working on moving his tools and small “APL-cations” to Tatin and GitHub, where Kai is now responsible for 90% of the existing packages. His presentation also briefly introduces several new packages and recent enhancements to existing tools.

Following immediately after Kai, I attempted to demonstrate how much easier packages can make your life by creating a small APL application within the new Cider project management system, using one Tatin package and one NuGet package. The Tatin package is Dyalog’s HTTP Client utility, HttpCommand (a rough equivalent of cURL), which is used to retrieve data from the web, and the NuGet package is MailKit, which reads e-mail using the POP3 protocol.

The new .NET eco-system is a potential source of many very useful packages that will work in the same way under Microsoft Windows, Linux, and macOS. Modern .NET APIs make widespread use of generic classes and methods, which means that they work the same way with different data types. Although this is how we have always worked in APL, Dyalog’s .NET bridge assumes that .NET expects specific data types. In his “Part 2” talk, John presents his thoughts on how Dyalog can embrace .NET generics.

The final two talks this week are about potential future packages. First, Josh David takes a look at open-source statistical libraries available to the APL developer, including KokoStats and TamStat. Rich Park follows with a demonstration of a simple interface for producing graphs from APL data using Vega-lite, a declarative language for describing visual presentations using JSON. Charts are rendered using a JavaScript library and, thanks to the HTMLRenderer, data can be provided directly from APL to produce graphics ready for publication on the web.

It’s great to see that the APL ecosystem is not only alive and well, but expanding!

——————————————

This week’s videos:

• U02: Transforming and Streamlining a Complex Development Process using Dyalog (Mark Wolfson)
• U03: The Evolution of APLTree and APL-cation (Kai Jaeger)
• D08: Using Packages (Morten Kromberg)
• D12: Dyalog Version 20.0 – Part 2 (John Daintree)
• D16: Statistical Libraries for Dyalog (Josh David)
• D18: Vega Charts with Dyalog (Rich Park)

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Follow