APL Seeds ’23: Wednesday 22 March

Posted on by

Pie chart of APL Seeds '23 registrant prior knowledgeLast Wednesday we hosted APL Seeds ’23, the third of our free online user meetings aimed at new and prospective users of APL. In attendance was a mix of new users and the APL-curious, as well as more experienced APLers answering questions and imparting wisdom in the chat and during the post-event meetup.

Rich Park began the meeting with an introduction to the basic syntax of the language and a sample solution to a problem from last year’s APL Problem Solving Competition – from data about the schedule of an online event, and the times people joined the event, which sessions did each user attend? He demonstrated how the thought process of a solution could map neatly onto an executable APL expression. In particular, he showed how an elegant solution tends towards using primitives directly with array argument values, as opposed to abstracting the solution for one piece and explicitly looping over the collection.

Adám Brudzewsky then explored how to remember unfamiliar ideas. He presented some mnemonic devices linking the glyphs that make up the core APL language with both the concepts that they represent and their locations on the APL keyboard. Whether they are two related concepts represented by the same symbol, or if the dyadic form is the same as monadic with a default left argument, or if two unrelated functions use the same glyph, these tricks are really useful for memorising the fundamental building blocks of APL.

Screenshot of Adám Brudzewsky's presentation

A still screenshot doesn’t do justice to the wonderful animations in Adám’s presentation.

We then heard about learning APL from the perspective of a non-programmer. Richard Savenije is a project director for infrastructure projects in a construction company. After a stint of programming in C at IBM, Richard decided he did not want to pursue programming any further. When he left, he took an APL manual with him, as he had been intrigued by the glyphs. It sat untouched for 28 years, until he recently decided to start learning to solve problems with APL. He told us about the new perspectives he has gained while using APL, and parallels with the type of problem solving he does in his day-to-day life.

APL media often shows the symbols, interesting approaches to algorithms, and other ways in which APL is fascinatingly unique in the world of programming languages. However, this often leaves people wondering whether they could use this in the more pedestrian tasks of automation, data processing, and application development. The final two talks of the day addressed the more practical use of APL.
You might have heard Stefan Kruger’s name before – he wrote the excellent Learning APL book and gave a presentation at APL Seeds ’22. He acknowledged that the domain of APL is largely seen as “super tight” numerical algorithms and array transformations. This year he explained the convenience of HttpCommand and ⎕JSON for ingesting data from external sources, and explored the creation of user commands to provide a convenient interface for performing repetitive tasks. Stefan used these to rewrite a substantial collection of (mostly Python) utility scripts as APL functions and user commands, ultimately shrinking the code by a factor of 10!

Screenshot of Stefan Kruger's presentation

Stefan shows us how to construct user commands to conveniently perform repetitive tasks.

In the final presentation of the day, Josh David answered the question of “what does it look like when my APL Seeds come to fruition?” He began with a demonstration of some analysis of water potability data that turned out to look dangerous – or fake. Throughout his demonstration, we got a feeling for how to rapidly inspect and transform data to gain insights. He included plenty of very useful tips for new APLers, such as how to stop the stream of text into the session when you execute an expression that produces long output (Action > Interrupt in the Session menu…there are more tips like this on our website). Finally, he showed some full-stack graphical applications – his SQL-Browser and The Carlisle Group’s FlipDB – with examples of database connectivity, querying and analysis.

After the talks, there was an informal online meetup. This included lively discussion about many topics, including language design and terminology. We also discussed questions about teaching tacit programming, and heard perspectives from some APL beginners – if someone is learning the concept of functions for the first time, then composition of functions is probably not appropriate at that point.

We hope we were able to spark interest, give some clarity to people’s image of APL, and inspire you to start or continue learning and using APL. Relevant materials have been uploaded to the APL Seeds ’23 webpage, including links to recordings of the presentations on dyalog.tv.

Email, RSS Follow

Structural vs. Mathematical “Under”

Posted on by

The APL Farm has recently hosted discussions about the proposed under operator, asking “why Dyalog hasn’t implemented it a long time ago”. I claimed that an important reason was that Roger Hui wasn’t a big fan of the proposed extension known as “structural under”. Stefan Kruger then asked me why that was so. Since Roger isn’t able to respond himself, I’ll do my best to explain my own understanding of the issues.

For anyone not familiar with the concept of under, or dual as it was known when first implemented in SHARP APL, it is an operator that provides a way to express a common mathematical pattern, where a computation or proof is simplified by first translating the input domain, applying a function, and then performing the inverse transformation. For example, if we use the symbol ⍢ to denote under, have the translation function as the right operand, and the function that applies to the translated domain as the left, then we could use +⍢⍟ to demonstrate that multiplication is equivalent to addition under logarithm, which is why slide rules work:

      1 2 3 4 (× ≡ +⍢⍟) 0.1 1.1 17 100
1

Another example of simplifying multiplication using dual is the use of Fast Fourier Transforms to multiply very large numbers, as explained in the notes about xtimes in the dfns workspace.

It is easy to find examples of dualities outside the mathematical domain, such as midnight snacking: (open refrigerator door, remove food, close door) or Roger’s favourite example of medical surgery: apply anaesthetic to greatly simplify the operation, perform surgery and (very important) back out the anaesthetic.

I believe that Roger was of the opinion that under/dual should be restricted to functions that have well-defined inverses using the “mathematical” definition used in SHARP APL and the J language. We can model this with a defined operator, let us call it Um:

   Um←{⍺←{⍵ ⋄ ⍺⍺} ⋄ ⍵⍵⍣¯1⊢(⍵⍵ ⍺)⍺⍺(⍵⍵ ⍵)}

The extension known as structural under allows the transformation function to be a structural function, even though such functions do not have inverses in the mathematical sense. In this case, the left operand is applied to the subset of items selected from the right argument by the structural function. This is easy to model using a feature of some APLs known as selective specification, in which you can have a function to the left of an assignment whereby the assignment only modifies items selected from an array using so-called “structural” functions – those functions that select or reorder items of an array without doing math, such as ↑ ↓ ⍉ ⌽ ∊.

Let us call this Us:

   Us←{⍺←{⍵ ⋄ ⍺⍺} ⋄ w←⍵ ⋄ ((⍵⍵)w)←(⍵⍵ ⍺)⍺⍺(⍵⍵ ⍵) ⋄ w}

This is very useful, for example, we could negate all items of an array except the first two by writing:

      - Us (2∘↓) ⍳5
1 2 ¯3 ¯4 ¯5

The big question is whether Us and Um can be combined into the same operator, or two separate operators are needed. One problem is that ↓ already has a defined inverse in Dyalog APL, which pads with zeros, so Um returns something different:

      - Um (2∘↓) ⍳5
0 0 ¯3 ¯4 ¯5

If we want to combine the two into one, then we need a rule that allows us to decide which definition to use. Marshall Lochbaum‘s BQN language has a table of structural primitive functions; it performs a structural under when possible (according to an extensively described rule), and otherwise attempts a mathematical under.

Personally, this sort of conditional logic in the definition of a key primitive makes me very uneasy, and I think Roger was struggling with this as well. Unfortunately he passed away before we were able to finish argdiscussing the design. Another reason for feeling that the two operators are different is that, while it often makes sense for the transformation to apply to both arguments in the mathematical case, it is extremely rare for structural under. You cannot do:

       1 + Us (2∘↓) ⍳5

because when you apply (2∘↓) to the left argument, you get an empty vector. Instead, you need to turn the left operand into a monadic function to add one to the three trailing elements:

       1∘+ Us (2∘↓) ⍳5
1 2 4 5 6

This increases my personal unease about combining the two into a single operator. Even though this would work “just fine” most of the time, it would still make it difficult to explain exactly what is going on.

It is my opinion that APL needs to remain a rationalised mathematical notation if it is to remain useful and relevant in the long term. Deciding whether to implement a strict mathematical operator, a combined operator, or two separate operators (or do nothing for a while longer), is one of the key design decisions that we will be faced with in the near future.

Email, RSS Follow

Welcome Peter Mikkelsen

Posted on by

Peter joined Dyalog Ltd in mid-July 2022, shortly after finishing his bachelor’s degree in Computer Science from Aalborg University in Denmark. He didn’t originally plan to look for a job so soon, but when he saw that Dyalog Ltd had an opening involving three of his favorite things (C programming, interpreters, and APL) it was too good an opportunity to let pass and he was offered the job after a couple of online calls and interviews. Peter and his girlfriend had planned to move to Esbjeg in the south-west of Denmark for a while, so he thought it was great that Dyalog Ltd made it possible to work from home. He joined just in time to be able to participate in Dyalog ’22, which meant that he met lots of customers, which he considered a great experience.

His interest in APL started while he was at university – at one point he thought to himself: “This language looks so strange and different that it might be worth learning.” After spending some time playing around with it and participating in the 2021 APL Problem Solving Competition, he started writing his own little APL interpreter, which he says must have been the point of no return for him. Even in the first few months at Dyalog Ltd, he has learned a great deal about the language and its quirks, and a lot about implementation techniques.

Peter hopes to be able to help develop the language in the years to come, and to make sure dfns get the love they deserve, as he thinks that functional programming is definitely the way to go!

Even though he is part of the “young generation” of APLers, writing long trains and doing code golfing in APL has yet to catch his interest.

When not working, Peter is likely to be found taking a long walk, or trying out something computer science related that has caught his interest. Often, the code he writes in his free time is for the Plan 9 operating system, which in his mind is so elegant and simple that it is almost a shame it has so few users.

Email, RSS Follow

Welcome Silas Poulson

Posted on by

As is often the case these days, Silas found APL through online programming language discussions – both in various forums and through videos (Aaron Hsu’s Co-dfns and John Scholes’ Game of Life demonstatration in particular) – whilst attempting to understand the more esoteric ideas and their theoretical underpinnings provided by Scheme and other functional languages. After discovering APL, Silas attempted to use it for his Data Analysis coursework, sticking primarily to a small set of primatives. However, hitting WSFULLs with the datasets required meant that APL remained just a fun system to play with, which led to an understanding of where (⍸) in terms of other primatives (⍳ and ⍴).

Silas saw one of Dyalog Ltd’s adverts for a C programmer; this was a language he had learnt during his exploration of the Raspberry Pi and its associated add-on boards, so he applied. He joined Dyalog Ltd in June 2022 after graduating from Royal Holloway in England with a bachelor’s degree in Computer Science, and eased his way into exploring the internal workings of the interpreter, where he now spends most of his time.

When not programmaing, Silas can be found enjoying a good book and the occasional summer hike with friends and family.

Email, RSS Follow

Dyalog ’22 Day 5: Looking Forward

Posted on by

Links to recordings from this day of the user meeting are at the bottom of this post.

To begin the final day of the user meeting, Aaron Hsu presented some strategies and ways of thinking about processing data that come from text. While his specific focus is parsing APL code for the co-dfns compiler, he believes that a parallel data-flow approach will be useful across domains, for example, when processing JSON data, which is common in modern web communication.

Aaron talks about parsing data from text

Aaron talks about parsing data from text

Adám Brudzewsky, despite not being in the room, captivated us with a supremely convincing argument for why three primitives could be considered to be filling in “core language” gaps: the “Select” function (⍺⊇⍵) for elegant indexing; the “Depth” operator (F⍥k) as analogous to rank (F⍤k) for nested arrays; and the “Behind” (F⍛G) operator complementing other function compositions “Beside” (F∘G), “Atop” (F⍤G) and “Over” (F⍥G) to make tacit programming an even better extension of APL expressivity.

Adám proposes a new "behind" function composition operator

Adám proposes a new “behind” function composition operator

It was exciting to see a return for APL to its birthplace in academia. Jesús Galán López of the Technical Univerity of Delft told us his plan to use APL in metallurgical models and analysis of experimental data. He hopes to use Dyalog APL as a central connecting piece in education, science, engineering and research. This is one of the first projects to eventually be supported by the APL fund mentioned by Stine on Monday.

Jesús with a typical data analysis example in materials science

Jesús with a typical data analysis example in materials science

A living legend in APL education, Raymond P. Polivka presented his latest book aimed at high school students with no programming knowledge whatsoever. The clearly structured modules and exercises will be an invaluable resource for teachers and recruiters alike, and they have been developed with both a lifetime of experience teaching APL as well as very recent classes with real high school students. He is asking for feedback on a draft available at aplclass.com/book.

Ray Polivka introduces his new book aimed at high school students

Ray Polivka introduces his new book aimed at high school students

For the last talk of the user meeting, Rich Park took us on a whirlwind tour of the media and training materials being developed at the moment. He also commented on some of the work that he believes is needed to tie these resources together and make them more discoverable, and what might be done to improve the experience for new users trying to find things out for themselves.

Rich Park talks about media, training and outreach at Dyalog

Rich Park talks about media, training and outreach at Dyalog

In the afternoon we ended the user meeting with some workshops. We got to know about performance considerations in the context of the Dyalog interpreter with Aaron Hsu and Karter Kooner. There was a chance to get hands-on experience using one of the asynchronous, concurrent programming methods in Dyalog: Futures and Isolates with Morten Kromberg and John Daintree. Josh David and Brian Becker taught us about sharing APL tools by creating and publishing packages using Tatin.

It has truly been an incredible experience returning to in-person user meetings. The sense of community felt in Olhão this year was far beyond what most people predicted, not just in light of separation over the last two years, but also in looking forward to an exciting future for APL and Dyalog.

Today’s presentations (links to recordings will be added as they become available):

Email, RSS Follow

Dyalog ’22 Day 4: Celebrations

Posted on by

Links to recordings from this day of the user meeting are at the bottom of this post.

The third day of presentations kicked off with Brian Becker running us through the gauntlet of setting up and deploying web services. Initial setup with Jarvis and Docker containers seems like an absolute breeze. However, the later stages configuring cloud services can be more fiddly.

Brian Becker talks about creating and deploying web services.

Brian Becker talks about creating and deploying web services.

Stephen Mansour of Misericordia University then gave us the hot tips for producing hot tubs. His new system TAMPA (Taming Mathematical Programming in APL) can be used to optimise some decision (e.g. how many hot tubs to produce) for some objective function (e.g. maximise profit) according to some constraints (e.g. resources available). The use of APL in TAMPA makes a near 1-to-1 translation of linear programming expressions into executable code.

Stephen Mansour explains the TAMPA mathematical programming framework.

Stephen Mansour explains the TAMPA mathematical programming framework.

We then got to hear some strong but convincing opinions about graphical user interfaces from Norbert Jurkiewicz, who told us about how The Carlisle Group has been incrementally integrating the HTMLRenderer and “the triad” of HTML, CSS and JavaScript into their systems. He championed the benefit of hiring external developers, in addition to the portability of using web-technologies, for graphical front ends.

Norbert Jurkiewicz gives his views on using web stack for front end development.

Norbert Jurkiewicz gives his views on using the web stack for front end development.

Neither Gitte Christensen nor Brian Becker are shy about saying that some of their favourite parts of every user meeting are the competition presentations. Luckily for us, both this year’s student winner and professional winner came to share their experiences about participating.

Professional winner Michael Higginson had actually been a kdb+ and q programmer for many years before recently deciding to expand his horizons with APL. He gave a fantastic breakdown of his thought process in solving both an easier problem which he found built his confidence, and then the notorious problem 6 on interpreting human-readable dates and times.

Michael Higginson takes us through his array programming journey.

Michael Higginson takes us through his array programming journey.

The audience could definitely empathise with all of the reasons given by student winner Tzu-Ching Lee as to why he likes APL: the glyphs; the concise syntax; operators; and algorithms as primitives. Alongside his excellent walkthroughs of two of his solutions, he had brilliant ideas for extending the problem description for the Base85 encoding/decoding problem once he noticed additional symmetry which could be expressed in his solutions.

Student winner Tzu-Ching Lee presents his winning solutions to the APL Problem Solving Competition.

Student winner Tzu-Ching Lee presents his winning solutions to the APL Problem Solving Competition.

In the afternoon, we took a coach about an hour away to the Quinta dos Vales vineyards and winery. We were treated to a tour of the winery, learning about the fermentation process; what goes into deciding whether to make single grape or blended wines; and the use of wooden barrels to imbibe additional flavour. Afterwards, we were split into teams and challenged to make our best and favourite blends of wines from three grapes. According to the judges, a majority Cabernet Sauvignon, with about a third Aragonês and just ten percent Touriga Nacional makes for the most delicious blend of tannins and spices. Later that evening, we enjoyed a delicious Portuguese churrasco – or barbeque.

Delegates enjoy the afternoon sun at Quinta dos Vales winery in the Algarve, Portugal.

Delegates enjoy the afternoon sun at Quinta dos Vales winery in the Algarve, Portugal.

Congratulations to the winners of the APL Problem Solving Competition, and congratulations also to the winners of the wine blending competition!

Today’s presentations (links to recordings will be added as they become available):

Email, RSS Follow