Numeric Case Conversion

Posted on April 1, 2024 by Adám Brudzewsky

Dyalog version 18.0, released in June 2020, introduced the Case Convert system function ⎕C. It was a replacement for the long-lived (since version 15.0, from June 2016) I-beam 819⌶, which was then deprecated. (By the way, did you know that the digits 819 were chosen to be reminiscent of the letters BIg as in big — uppercase — letters?) It is expected that 819⌶ will be disabled in the next major version of Dyalog APL.

⎕C already has several advantages over 819⌶, for example the ability to case-fold rather than lowercase. (Did you know that Cherokee syllabary case-folds to uppercase?) With today’s release of Dyalog version 19.4.1, we’re happy to announce a further extension of ⎕C, covering scaled format (also known as scientific or E notation) and complex numbers.

By default, APL uses the letter E to separate mantissa and exponent in very large and very small numbers. Similarly, the letter J is used to separate real and imaginary parts of complex numbers:

      2 20 * 64 ¯24
1.844674407E19 5.960464478E¯32
      ¯1 ¯5*0.5
0J1 0J2.236067977

For input, however, e and j are accepted in addition to E and J:

      1E4 1J4 ≡ 1e4 1j4
1

You can now conveniently mitigate this asymmetry using ⎕C:

      ⎕C 2 20 * 64 ¯24
1.844674407e19 5.960464478e¯32
      ⎕C ¯1 ¯5 * 0.5
0j1 0j2.236067977

We hope that this added functionality will be exploited to its fullest by all our users. Please contact us if you experience any stability issues with the new feature.

The APL Quest Series

Posted on March 15, 2024 by Adám Brudzewsky

It seemed like a normal Friday until mid-afternoon. But on 4 February 2022, I embarked on a journey that, at the time, seemed to stretch impossibly far into the future — a future that wasn’t entirely known yet. In the APL Orchard chat room on Stack Exchange, a dozen APLers, some experts and some newbies, held the very first APL Quest chat event.

In this first session, we explored the oldest APL Problem Solving Competition phase 1 problem – question number 1 from 2013 – presented our solutions, and discussed them for about half an hour. The following week, I recorded a video where I went through some of these solutions, with the code posted on GitHub. This pattern continued each week; chat events on Fridays, and a follow-up video released (usually) the week after, each session looking at a different phase 1 problem.

APL Quest chat in progress

Inspiration

Originally, the idea came from Richard Savenije, inspired by LeetCode. Both he and Stefan Kruger, who later became my colleague, were frustrated with LeetCode’s assumptions about programming languages – assumptions that didn’t really hold for APL. Aside from that, their test framework didn’t permit APL submissions anyway.

So, Richard suggested that we should make our own problems website, and Stefan pointed out that my colleague Rich Park had already set up a website that offered automatic checking of solutions to simple problems. With the help of a summer intern, Rich had populated the site with all APL Problem Solving Competition phase 1 problems from 2013 until 2021 (the 2022 round was scheduled to launch two months later). Richard suggested using this site for weekly puzzles, one every Friday afternoon, and we found a time that suited the people present.

Next, we had to decide on a format. Should it be a Zoom meeting or a chat event? Earlier, there had been a couple of chat events series in the APL Orchard; the APL Cultivation series ran weekly from October 2017 until May 2018 and semi-weekly from 28 November 2019 until August 2020. Stefan Kruger was in the process of converting these to a since-completed book, APL Cultivations. After some discussions, we decided to have the sessions in chat, and I came up with the idea of recording a short screen cast after each event.

I got the arithmetic wrong, and claimed we’d have 100 weeks worth of problems, as the 2022 problems would be available by the time we had explored the other problems, giving us almost two years of material. Actually, the 2023 problems were ready when we got to them! But either way, the end seemed very far away…

Technical Details

And yet, here we are, after 110 weeks, 110 live chat events, and 110 published videos – a total of over 22 hours of video contents! We never missed a week, even on various holidays, though Rich Park did have to substitute for me a few times when I was prevented from hosting, and sometimes I had to push off recording and/or publishing a video until two or three weeks after the chat event. Sometimes, I’d host the chat event while travelling in a car or a train, and sometimes I’d record videos in hotel rooms or in other people’s homes. By always using a plain white (or off-white) background, I was able to get a consistent look, irrespective of where I was, though I sometimes had to move furniture to sit in front of a bare wall, and once I had to drape my bedspread over the hotel room’s television…

Recording an APL Quest session

Speaking of looks, I’ve received praise for the technical quality of my videos; for their smooth integration of presentation and live coding and for their nice design. So, for those who are interested in how I did it…the introductory screen and the problem statement are simply PowerPoint slides with Fade transitions. After the problem description slide, I faded to a blank slide, and from there, I switched application to RIDE (on Microsoft Windows, you have to install it separately), which was running in full-screen mode with the Language Bar and Status Bar hidden. When I started, the newest released RIDE was version 4.3, but I was running pre-releases of v4.4 and Dyalog v18.2 that added the Nord theme which I had fallen in love with in 2021. By matching my slide colours to the RIDE theme, I achieved a seamless transition without having to do any video editing.

I’m somewhat of a typeface enthusiast. Previously, I had searched for what I considered good sans-serif fonts, and for this project, I choose the humanist Go font. For APL code, I went with my own SAX2 font which I had created by extracting letterforms from the old SAX (SHARP APL for UNIX) manual, and then extended to cover the characters necessary today. However, it bothered me that the SAX font looked so thin next to Go, due to being digitised from the golf ball of a IBM Selectric without accounting for the visual weight normally added by the typewriter’s ink ribbon.

I had to hack font selection into RIDE, which I was able to do because RIDE is built using normal web technology, in particular CSS. After some experimentation, I found the way to do it. After I switched to RIDE v4.5, I was able to set the fontface in an official manner, but even this wasn’t enough. I had relied on PowerPoint’s reasonable auto-bolding of the SAX font, which otherwise didn’t include bold, but RIDE v4.5 wouldn’t let me style the APL font further, so I had to hack RIDE again!

“Hacking” RIDE v4.5

This time, I didn’t need to modify source files, but rather found that RIDE was “vulnerable” (though not in a dangerous way) to CSS injection through its font input field. Making the font bold didn’t look right, as that would only smear the letters horizontally, but adding a “text stroke” had the desired effect. If you want your RIDE to resemble what you can see in the videos, set the APL font to SAX2')}.monaco-editor *{-webkit-text-stroke:.67px currentcolor}. With the visuals set, I recorded using OBS Studio and, on the rare occasion that I needed to edit something, I used DaVinci Resolve.

Concluding the Series

When we finished that last session, it felt rather anti-climactic, but I can look back on a very enjoyable time. And of course, the efforts that all participants put into this are not forgotten; we’ve got an amazing chat and video series that future APLers can enjoy. Thank you to everyone who contributed; chat participants, video commenters, colleagues (especially Brian Becker who authored all the problems), and most of all my wife, who often had to encourage me to record the next video(s) and also gave me the time and space to do so, even if it meant single-handedly keeping our children quiet. If you’re up for a marathon, you can watch the entire 22-hour series on YouTube, and APL Quest on the APL Wiki includes links to all the problems, chat sessions, code, and videos.

Welcome Mike Mingard

Posted on January 10, 2024 by Dyalog

Mike recently joined Dyalog Ltd, bringing his experience in web and digital design.

Mike grew up in West London, and now lives by the sea in East Sussex. His background is in Recording Engineering, and he worked for a number of years as a front-of-house sound engineer and theatre stage manager. Having learnt the basics of HTML while at University, he started to develop websites as a hobby; it wasn’t long before he realised the hobby was the more rewarding pursuit.

Four years of web and digital design followed, based in London. Clients included many international companies and organisations such as Merial, The World Health Organization (WHO), the United States Department of Agriculture (USDA) and The Pirbright Institute.

Mike then worked for ten years at a West Sussex-based web agency with a diverse mix of clients, from small local companies to brands such as BMW and Suzuki.

In 2013 Mike joined Optima Systems and immediately moved all design and branding work that was previously outsourced to be in-house. Optima System’s internal needs were a primary focus, supplemented with working on websites for many local companies and brands. One of the companies Mike worked with was Dyalog Ltd, so when he needed a new challenge he knew exactly who to speak to!

Mike is now our in-house resource for all marketing and graphical design efforts (social media images, banners, video thumbnails, and so on).

Welcome Abs Suri

Posted on January 3, 2024 by Dyalog

After graduating from the University of Portsmouth with a bachelor’s degree in software engineering, Abs started investigating IT roles. He wanted something that provided a challenge, working for a team that was passionate about what they do, and in which he could contribute to that effort through the software and IT abilities that he has accumulated; with Dyalog Ltd he found it! Abs was hurled into the world of APL, which was very different to the Python that he was used to. Although his role doesn’t involve much APL, he appreciates how compact APL solutions can be when wielded correctly. Armed with a fresh perspective and a helpful attitude, he joins our IT department hoping to maintain and improve the current IT infrastructure and provide technical solutions to anyone that needs it.

When Abs isn’t working, he can be found gaming, a hobby that started his fascination with computers and technology. He also loves to learn languages and experience different cultures through travelling, and is currently working on his spoken Japanese. However, his true passion lies within music, and he enjoys playing guitar, bass, and drums (whenever he can get his hands on them).

Dyalog ’23 Videos: Week 8 – Celebrate Solstice with the Last of Our Dyalog ’23 Videos

Posted on December 22, 2023 by Morten Kromberg

Whether you are celebrating a winter holiday, looking forward to the days getting longer from tomorrow, or enjoying summer south of the equator, we hope you have time to enjoy the final collection of light-hearted presentations from Dyalog ’23.

Andy Shiers has overall responsibility for making sure that Dyalog gets correctly built and tested before it reaches the users – and that it is supported once it has been released. In his “Fireside Chat”, Andy covers a wide range of topics that he regularly encounters but feels might have not been sufficiently emphasised in other talks or in our documentation – and that you might need to know about. Andy is back by popular demand from user meeting delegates, after a gap of a few years.

The development team at SimCorp Italiana have been regular contributors to Dyalog user meetings, with insightful and amusing anecdotes about the relationship between humans who work with technology. This year, in “Once Upon A File”, the stories are mostly about the trials and tribulations of importing data.

Ray Cannon has been working on synthesising music on various computing devices. From a humble start where he tapped notes out himself using “A Pointy Stick”, he learned how to generate chords, add harmonics, attack, decay, and reverberate – and store the result in .WAV files. The result is a rendering of J.S. Bach’s Tocatta and Fugue in D Minor, BMV565 – complete with animations – on multiple organ pipes, all done in APL.

We hope that you enjoyed the presentations from Dyalog ’23, that you have a Happy New Year ahead, and that we will see many of you at Dyalog ’24 in Glasgow (15-19 September 2024).

——————————————

This week’s videos:

Materials for all presentations can be downloaded from the Dyalog ’23 webpage.

Dyalog ’23 Videos: Week 7 – Performance and Scaling

Posted on December 15, 2023 by Morten Kromberg

Although run-time performance is rarely the most important reason for selecting APL, good performance often becomes important during the lifetime of an application (especially if it is successful and, therefore, has to deal with growing data volumes and numbers of users). Array-oriented programming naturally encourages Subject Matter Experts to use dense and pointer-free structures, which allow APL-based solutions to do things like balancing thousands of portfolios in a fraction of the time that more traditional solutions need.

The “set functions” – membership (∊), index of (⍳), intersection (∩), union (∪), and without (~) – are already some of the most highly-tuned primitives in the history of APL because they are critical to the performance of very many APL applications. New parallel instruction sets keep appearing in modern processors, and the balance between processor and memory performance is in a constant state of flux. Also, computer scientists continually improve algorithms for searching. Karta Kooner’s talk on the performance of Set Functions describes the approach that he is taking as we embark on another round of optimisations.

Sometimes, the best way to improve application performance or reliability is to split the application into multiple processes that can run independently and be scaled up by adding more processes as required. Apache Kafka is a widely-used tool for connecting such processes and reliably forwarding streams of messages between them. Stefan Kruger presents the benefits of Kafka and his initial experiments on what a Dyalog interface to Kafka might look like.

Application performance increasingly depends on how much memory you use, and how efficiently you move data around. If you want to help ensure that your APL algorithms have the best possible mechanical sympathy with Dyalog APL, Richard Smith’s “Introduction to the Workspace” will help you understand how the interpreter manages the memory that holds your arrays in the workspace.

An APL compiler promises to help APL users take advantage of highly parallel hardware like General-Purpose Graphics Processing Units (GPGPUs). If the compiler is self-hosted, it also makes it practical to quickly port APL to virtual environments, or base an APL implementation on other programming languages such as Python or JavaScript. Aaron Hsu presents an update on recent enhancements to the Co-dfns compiler, and plans for the near future.

Although I have been an APL developer for more than four decades, it is only recently that I understood how APL can be used to efficiently and elegantly handle tree structures using simple arrays. Brandon Wilson has been studying techniques developed by Aaron Hsu that make it possible to parallelise compilation of APL by the Co-dfns compiler. YAML parsers are notoriously difficult to write accurately, and Brandon hopes to find an effective description of YAML through APL that can help the community better understand its edge cases.

Next week the final videos from Dyalog ’23 will be published (along with my final blog post on the subject) – a great way to end the year!