Dyalog ’24 Videos: Week 3 – The APL Forge, Education, and Data Science

The recordings released this week focus on activities related to attracting new users to APL.

For many years, the APL Problem Solving Competition was a cornerstone of our efforts to promote APL to a new generation of potential users. This year, we introduced the APL Forge, where prizes are awarded for original work done in APL, rather than solutions to “classical” computer science problems. Our Managing Director (CEO) Stine Kromberg introduced the new format and presented the Grand Prize to this year’s winner, Holder Hoover. Holden, who is an aviation geek and still in High School, then explained how he built his own system for receiving and processing the ADS-B data that is transmitted by nearly all aircraft so that he can track aircraft within range of receivers operated by himself and a friend in the Azores. In three days (apparently between two exams), he managed to decode data using APL that he had spent three months failing to handle in JavaScript.

We were very pleased to learn that Sergey Ichtchenko and his two collaborators, Dani Adham and Hex, have been working on a MOOC (Massive Open Online Course) for the University of Helsinki, and we invited Sergey to Dyalog ’24 to tell us more about this project. When completed, the APL MOOC will be an online APL course for university students, worth 5 ECTS credits (or 3% of a Bachelor’s Degree) to Finnish and other European students. You can review the work in progress, and send comments or words of encouragement to the team, at aplmooc.fi.

Data Science is a term that – along with artificial intelligence – is used to describe a very wide range of data processing techniques, from cross-tabulation and standard statistical techniques over regression, to large and small “language models”. The other three talks released this week look at the use of APL as a tool for Data Science from different perspectives.

In his presentation Dyalog for Data Science, Jesús Galán López compares APL to popular Python libraries, solving some well-known introductory problems in what used to be a statistics curriculum but is now called Data Science. As he concludes: “If you are an APLer and you have been using APL to extract useful information from data: Congratulations, you are a Data Scientist! And if you are a Data Scientist – Try APL!”.

Stephen Mansour is the creator of TamStat, an application written in Dyalog that he uses to teach statistics to University students. A core component of the design of TamStat is the use of defined operators to create a domain-specific notation that is not only flexible and powerful but makes the fundamentals of statistics easier to grasp – hence the name “Taming Statistics”. His presentation on Taming Regression using APL illustrates how, by designing user functions and operators to be similar to APL primitives, you can produce a notation that is a user-friendly Tool of Thought, helping you to understand statistics in ways that are close to impossible in other languages. In addition to being available as a free application, the source code of TamStat is free and open source, and available for use in your own applications.

Not all data science applies to rectangular data that can be easily grouped and cross-tabulated. In 2015 I did a presentation in the Talks at Google series. Towards the end of my presentation, I was asked about Iverson’s original intention of having APL natively apply to arrays and trees…”What happened to the latter?”. Arthur Whitney was in the room, and I asked out loud “What happened to the trees, Arthur?”, to which he responded “I don’t know” and can be seen shaking his head. I then waffle on a bit about nested arrays and how they lead to poor performance. Today, thanks to Aaron Hsu, I have learned that regular flat APL arrays allow for extremely efficient representations of trees (his thesis on the co-dfns compiler shows that this can be used to achieve orders-of-magnitude improvements in performance in compilers). In his presentation Climbing Trees and Catching Bugs, our Senior Intern (spending his second consecutive summer with us!) Asher Harvey-Smith introduces a tutorial that he is working on, with the goal of making Aaron’s techniques easily approachable to newcomers. He also talks about some of the other fun he had this summer.

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’24 webpage.