Category Archives: Events

Dyalog ’24 Videos: Week 4 – Migrating APL Code, and New GUI Frameworks

Posted on by
Reply

The presentations released this week are about APL applications on the move. Some applications are moving from one APL implementation to another, for example from APL2 or APL+Win to Dyalog APL. Others are moving from one platform to another, for example from a mainframe or the Microsoft Windows desktop to Linux in docker containers in the cloud. Sometimes both moves are being considered – the need to switch platforms often triggers a review of which APL system to use on the new platform.

Migrating to Dyalog APL

OLE or ActiveX is an old technology which is being replaced by web services using tools like Jarvis. However, quite a few existing applications use it to make APL business logic available from tools like Microsoft Excel. Dyalog allows you to turn any namespace into an OLE Server with an API that is specific to your application – this requires the design of an object-oriented API. APL+Win comes with a simpler (evil tongues might say less secure) but in some ways easier-to-use framework in which a single OLE Server called WSEngine allows you to load any workspace and execute code within it. Jürgen Wiedemann from Dittrich und Partner Consulting (DPC) shows how the Dyalog framework can be used to implement something very similar to the APl+Win tool. Not only does this simplify some migrations, it also provides a simpler OLE Server framework for users of Dyalog who don’t need or want an application-specific API.

We are in the process of migrating the METSIM® application from APL+Win to Dyalog. As part of this project, we are developing general-purpose tools for moving APL source code from other APL systems to Git, performing automated translation and/or emulation of language features not directly supported by Dyalog. Karl Holt and I present the current state of these tools, including the current prototype of a function named ∆WI, which emulates ⎕WI from APL+Win or MicroAPL APLX, and will help move GUI applications to Dyalog with minimal application code changes. All these tools will be made available free as public repositories on GitHub.

Markos Mitsos of ERGO insurance has taken a completely different approach to his migration from APL+Win to Dyalog. Rather than use tools like those described in the previous two presentations, which aim to minimise the changes required to application code, Markos has carefully studied the Dyalog APL language and tools, and decided to refactor his old code to take full advantage of the new platform. This obviously takes much longer than using semi-automated migration tools. In his presentation on workspace transformations, Markos explains why he feels the extra work was worth it.

New GUI Frameworks

First, a bit of fun: RayLib is a simple and easy-to-use library for video game programming (although it can be used for more serious purposes). Brian Ellingsgaard is the creator of raylib-bqn, an interface from BQN, which is an array language with many similarities to Dyalog APL. He spent this summer with us as an intern, creating a similar interface from Dyalog – the result is raylib-apl. In his presentation, he gives examples of 2D and 3D game programming using RayLib…including the mandatory 3D duck rendering!

The original goal of our Everywhere WC project (EWC) was to allow existing applications using the Microsoft Windows GUI based on the ⎕WC system function to be moved to other platforms, including Linux in the cloud. EWC provides cover-functions that emulate the ⎕WC family of functions using a JavaScript (React) application to recreate equivalent GUIs. This allows applications to be migrated to Linux and macOS (or Microsoft Windows) desktops using the HTMLRenderer, or to any platform with a web browser to render the user interfaces – with minimal changes. However, the implementation also supports changing the look and feel using CSS styling, Flex layout for responsive UI, and several new controls based on JavaScript libraries like the ApexCharts graphics package. In this presentation, I also suggest that it has potential for completely new applications.

In the 1980s, Pete Donnelly and Geoff Streeter oversaw the design and development of ⎕SM, a system function for creating text-based (“80×25”) user interfaces. When Graphical User Interfaces came with Microsoft Windows, Pete worked with John Daintree to design ⎕WC, a system function for creating GUIs under Windows. In both cases, these features were quickly mastered by APL users, and subsequently used to build many of the most successful applications written in Dyalog. Now that applications increasingly need to be cross-platform and/or remotely controlled, Pete has been Noodling*, searching for intuitive ways for APL users to define cross-platform, HTML-based UI, based on popular widget libraries. Given his track record, it is worth paying attention to Pete’s Noodles!

* John Daintree refers to experiments as “Noodles”

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’24 webpage.

Email, RSS Follow

Dyalog ’24 Videos: Week 3 – The APL Forge, Education, and Data Science

Posted on by
Reply

The recordings released this week focus on activities related to attracting new users to APL.

For many years, the APL Problem Solving Competition was a cornerstone of our efforts to promote APL to a new generation of potential users. This year, we introduced the APL Forge, where prizes are awarded for original work done in APL, rather than solutions to “classical” computer science problems. Our Managing Director (CEO) Stine Kromberg introduced the new format and presented the Grand Prize to this year’s winner, Holder Hoover. Holden, who is an aviation geek and still in High School, then explained how he built his own system for receiving and processing the ADS-B data that is transmitted by nearly all aircraft so that he can track aircraft within range of receivers operated by himself and a friend in the Azores. In three days (apparently between two exams), he managed to decode data using APL that he had spent three months failing to handle in JavaScript.

We were very pleased to learn that Sergey Ichtchenko and his two collaborators, Dani Adham and Hex, have been working on a MOOC (Massive Open Online Course) for the University of Helsinki, and we invited Sergey to Dyalog ’24 to tell us more about this project. When completed, the APL MOOC will be an online APL course for university students, worth 5 ECTS credits (or 3% of a Bachelor’s Degree) to Finnish and other European students. You can review the work in progress, and send comments or words of encouragement to the team, at aplmooc.fi.

Data Science is a term that – along with artificial intelligence – is used to describe a very wide range of data processing techniques, from cross-tabulation and standard statistical techniques over regression, to large and small “language models”. The other three talks released this week look at the use of APL as a tool for Data Science from different perspectives.

In his presentation Dyalog for Data Science, Jesús Galán López compares APL to popular Python libraries, solving some well-known introductory problems in what used to be a statistics curriculum but is now called Data Science. As he concludes: “If you are an APLer and you have been using APL to extract useful information from data: Congratulations, you are a Data Scientist! And if you are a Data Scientist – Try APL!”.

Stephen Mansour is the creator of TamStat, an application written in Dyalog that he uses to teach statistics to University students. A core component of the design of TamStat is the use of defined operators to create a domain-specific notation that is not only flexible and powerful but makes the fundamentals of statistics easier to grasp – hence the name “Taming Statistics”. His presentation on Taming Regression using APL illustrates how, by designing user functions and operators to be similar to APL primitives, you can produce a notation that is a user-friendly Tool of Thought, helping you to understand statistics in ways that are close to impossible in other languages. In addition to being available as a free application, the source code of TamStat is free and open source, and available for use in your own applications.

Not all data science applies to rectangular data that can be easily grouped and cross-tabulated. In 2015 I did a presentation in the Talks at Google series. Towards the end of my presentation, I was asked about Iverson’s original intention of having APL natively apply to arrays and trees…”What happened to the latter?”. Arthur Whitney was in the room, and I asked out loud “What happened to the trees, Arthur?”, to which he responded “I don’t know” and can be seen shaking his head. I then waffle on a bit about nested arrays and how they lead to poor performance. Today, thanks to Aaron Hsu, I have learned that regular flat APL arrays allow for extremely efficient representations of trees (his thesis on the co-dfns compiler shows that this can be used to achieve orders-of-magnitude improvements in performance in compilers). In his presentation Climbing Trees and Catching Bugs, our Senior Intern (spending his second consecutive summer with us!) Asher Harvey-Smith introduces a tutorial that he is working on, with the goal of making Aaron’s techniques easily approachable to newcomers. He also talks about some of the other fun he had this summer.

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’24 webpage.

Email, RSS Follow

Dyalog ’24 Videos: Week 2 – New Functionality

Posted on by
Reply

The recordings released this week focus on active development projects at Dyalog Ltd.

The first three presentations describe features that are expected to be in the next version of Dyalog. In Setting and Getting Variable Values Mk II, Adám Brudzewsky describes a set of new system functions for setting and getting variables’ values using arrays. Until now, this required the use of the primitive function execute (⍎), in ugly and inefficient expressions. Also, execute is increasingly seen as a potential security risk, as careless use allows code injection attacks – so these new Dyalog v20.0 features will not only increase the readability and performance of application code, they will also make applications more secure.

John Daintree returns with WC Plugins. This is not a talk about tools for unclogging drains, but a general mechanism that will allow both the Dyalog development team and users of Dyalog to create self-describing libraries in C, which the ⎕WC (and ⎕NEW) system functions will be able to create instances of, in the same way that they can currently be used to create GUI objects and OLE or OCX components. This will allow us to open-source parts of the Dyalog interpreter, such as our HTMLRenderer interface to the Chromium Embedded Framework, the Conga interface to TCP and UDP, and an upcoming cryptographic library. It will also make it easier for users to contribute to the Dyalog ecosystem.

Peter Mikkelsen is one of the youngest members of the Dyalog development team. In New Function for Shell Calls, he describes the work that he has done to create a modern version of one of the oldest system functions in Dyalog APL – the ⎕SH function for making shell calls (known as ⎕CMD under Microsoft Windows). Among other things, this new function can be interrupted and can run the shell command on a separate operating system thread. It allows you to separate and redirect output streams, and specify what encodings are used. It even allows you to stream output to and from the external code asynchronously, using tokens and callbacks. It is slightly surprising that it cannot make toast!

The other two recordings this week are about features that are already available, but more people need to know about!

Adám Brudzewsky shows how text files containing APL code or data are all that you need to define and run an application in Initialising and Starting from Text Files. You can also extend your APL development environment with a few text files in the right place!

Finally, in New Tatin Packages, Brian Becker reintroduces the Tatin package manager, and describes three new Tatin packages that Dyalog Ltd have recently made available – a package for loading and using .NET packages hosted in the NuGet system (dyalog-NuGet), a package for managing APL processes in the same way on all platforms supported by Dyalog (dyalog-APLProcess), and a somewhat experimental package for tapping into the OpenAI platform and generate images, have conversations with large language models (LLMs), and access other “Artificial Intelligence” features (dyalog-OpenAI).

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’24 webpage.

Email, RSS Follow

Dyalog ’24 Videos: Week 1 – Welcome to the Dyalog ’24 Presentations!

Posted on by
Reply

We’ve finished processing the first recordings of the presentations given at Dyalog’24 in Glasgow. The first four are now available to watch, and you can look forward to five or six new recordings each week until the end of November.

As usual, the Dyalog user meeting opened with presentations by the CEO and CTO, who provided statements of direction for our technical and marketing activities. As mentioned in my blog post introducing the videos after last year’s user meeting, we have a new CEO since 1 January – Stine Kromberg is now at the helm, and opened the meeting. I once again followed up the CEO’s introduction with a whirlwind tour of what I consider to be the most important stops on our technical road map – also introducing many of the talks that will follow later.

The two other talks we are releasing this week are from the other end of the meeting, appearing first due to technical issues that have delayed a couple of other recordings. First up is Aarush Bhat, who is making his first presentation at a Dyalog user meeting. Aarush is based in Bangalore and has been working for Dyalog for just over a year now. So far, most of his efforts have been concentrated on building a new framework for very thoroughly testing primitive elements of the Dyalog APL language; in this presentation, he explains what he has been able to do so far.

At the other end of the spectrum, we have lost count of how many presentations our Chief Architect John Daintree has made! In the third of his three presentations this year, he talks about limits in the Dyalog APL interpreter. These limits include things like the maximum rank of an array, the number of different data types we can support, the maximum number of constants and names in an expression, and so on. Many of the structures that impose these limits have remained unchanged since Dyalog version 1.0, but some of the limits are now coming under pressure by migrants from APL systems that have different limits, and some of the new features that we are adding to Dyalog, like array notation – and new primitives.

Any change to such structures has potential impacts on performance. Work will need to be done to continue to allow the sharing of arrays between different versions of Dyalog through workspaces, component files, and TCP sockets. If we do make changes, we’d like to get them all over and done with in one go, and ideally not change them again for another 40 years.

I hope you enjoy watching the first set of recordings from Dyalog ’24!

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’24 webpage.

Email, RSS Follow

Dyalog ’23 Videos: Week 8 – Celebrate Solstice with the Last of Our Dyalog ’23 Videos

Posted on by

Whether you are celebrating a winter holiday, looking forward to the days getting longer from tomorrow, or enjoying summer south of the equator, we hope you have time to enjoy the final collection of light-hearted presentations from Dyalog ’23.

Andy Shiers has overall responsibility for making sure that Dyalog gets correctly built and tested before it reaches the users – and that it is supported once it has been released. In his “Fireside Chat”, Andy covers a wide range of topics that he regularly encounters but feels might have not been sufficiently emphasised in other talks or in our documentation – and that you might need to know about. Andy is back by popular demand from user meeting delegates, after a gap of a few years.

The development team at SimCorp Italiana have been regular contributors to Dyalog user meetings, with insightful and amusing anecdotes about the relationship between humans who work with technology. This year, in “Once Upon A File”, the stories are mostly about the trials and tribulations of importing data.

Ray Cannon has been working on synthesising music on various computing devices. From a humble start where he tapped notes out himself using “A Pointy Stick”, he learned how to generate chords, add harmonics, attack, decay, and reverberate – and store the result in .WAV files. The result is a rendering of J.S. Bach’s Tocatta and Fugue in D Minor, BMV565 – complete with animations – on multiple organ pipes, all done in APL.

We hope that you enjoyed the presentations from Dyalog ’23, that you have a Happy New Year ahead, and that we will see many of you at Dyalog ’24 in Glasgow (15-19 September 2024).

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Email, RSS Follow

Dyalog ’23 Videos: Week 7 – Performance and Scaling

Posted on by

Although run-time performance is rarely the most important reason for selecting APL, good performance often becomes important during the lifetime of an application (especially if it is successful and, therefore, has to deal with growing data volumes and numbers of users). Array-oriented programming naturally encourages Subject Matter Experts to use dense and pointer-free structures, which allow APL-based solutions to do things like balancing thousands of portfolios in a fraction of the time that more traditional solutions need.

The “set functions” – membership (∊), index of (⍳), intersection (∩), union (∪), and without (~) – are already some of the most highly-tuned primitives in the history of APL because they are critical to the performance of very many APL applications. New parallel instruction sets keep appearing in modern processors, and the balance between processor and memory performance is in a constant state of flux. Also, computer scientists continually improve algorithms for searching. Karta Kooner’s talk on the performance of Set Functions describes the approach that he is taking as we embark on another round of optimisations.

Sometimes, the best way to improve application performance or reliability is to split the application into multiple processes that can run independently and be scaled up by adding more processes as required. Apache Kafka is a widely-used tool for connecting such processes and reliably forwarding streams of messages between them. Stefan Kruger presents the benefits of Kafka and his initial experiments on what a Dyalog interface to Kafka might look like.

Application performance increasingly depends on how much memory you use, and how efficiently you move data around. If you want to help ensure that your APL algorithms have the best possible mechanical sympathy with Dyalog APL, Richard Smith’s “Introduction to the Workspace” will help you understand how the interpreter manages the memory that holds your arrays in the workspace.

An APL compiler promises to help APL users take advantage of highly parallel hardware like General-Purpose Graphics Processing Units (GPGPUs). If the compiler is self-hosted, it also makes it practical to quickly port APL to virtual environments, or base an APL implementation on other programming languages such as Python or JavaScript. Aaron Hsu presents an update on recent enhancements to the Co-dfns compiler, and plans for the near future.

Although I have been an APL developer for more than four decades, it is only recently that I understood how APL can be used to efficiently and elegantly handle tree structures using simple arrays. Brandon Wilson has been studying techniques developed by Aaron Hsu that make it possible to parallelise compilation of APL by the Co-dfns compiler. YAML parsers are notoriously difficult to write accurately, and Brandon hopes to find an effective description of YAML through APL that can help the community better understand its edge cases.

Next week the final videos from Dyalog ’23 will be published (along with my final blog post on the subject) – a great way to end the year!

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Email, RSS Follow