Richard Smith asks and answers: Is it Christmas Yet?

Depending on which day you decide to watch this recording, you may get a different answer from the one that Richard did as he answered his own question (“Is It Christmas Yet?”) in the first minute of his presentation. Of course, the true purpose of the talk was to show off a potential new system function for converting between a variety of time encodings. Not just the obvious ones like the 7-element ⎕TS format timestamp and the Dyalog Date Number, which is the number of days since the 31st of December 1899, but also a variety of Julian Dates, ⎕FRDCI style timestamps, UNIX time, Excel datetimes, Stata, R and SPSS dates and more – a total of more than 20 different time formats. Richard also shows how version 18.0 will allow you to determine the time in different time zones, and ends with formatting the current time in Helsinki – in Welsh.

Roberto and students from Liceo Scientifico GB Grassi Saronno

Inspired by Tetsuya Miyamoto, the inventor of the KenKen and other puzzles, Roberto Minervini avoids lecturing and prefers to present students with puzzles that they will be motivated to solve, learning new skills including mathematics and APL in the process. Pietro, Gabriele, Alessandro had their first exposure to APL in Roberto’s class at the at the Liceo Scientifico GB Grassi Saronno near Milan in Italy. Together with Roberto, they have created “MathMaze”, a platform for hosting real-time puzzle tournaments.

In this talk, they explain the unique scoring algorithm and the difficulty of creating puzzles that don’t make it clear whether you should solve them by thinking about them, by making drawings, or using the computer. A really good puzzle will requires a combination of techniques. As Alessandro explains, APL makes me think about the real mathematics behind a puzzle before I start writing the code.

Summary of this week’s videos:

• D08: Is it Christmas Yet? (Richard Smith)
• U11: Why Teach APL? The Art of Teaching Without Teaching (Roberto Minervini with Pietro Pio Palumbo, Gabriele Meroni and Alessandro Iaselli, Liceo Scientifico GB Grassi Saronno)

(Video releases will resume in January 2020)

Follow

When Aaron Hsu was at Dyalog ’19 in Elsinore, he was preparing the defence of his PhD Thesis on A Data Parallel Compiler Hosted on the GPU. In his talk “Lessons for the Masses from the Trenches of Co-dfns” he looks back on some of the key lessons learned while working on the PhD and the Co-dfns compiler.

Aaron Hsu presents some of his insights from his work on the Co-dfns project

As usual, Aaron delivered a talk designed to make every one of us question the fundamental assumptions that we make about programming. Selected sound bites include:

Uncle Andy’s back with another fireside chat

To bring you back down to earth, (Uncle) Andy Shiers’ fifth Fireside Talk is about little things that Andy thinks are important to anyone managing or using a Dyalog APL installation that he suspects you have forgotten about, or may have missed when reading the documentation. Some of them are things that he overheard developers talking about and suspects are not documented at all! Most of them are things that he needed himself, or used to handle a support call. Serial numbers play an important role this year; the changes we have made so that we can support the use of unregistered versions of APL for testing and demonstration purposes make it important to understand the impact of serial numbers and how to manage them.

Join us again in week 9 to hear Richard Smith explain how to compute whether it will soon be Christmas and Roberto Minervini (and students) tell us about the Art of Teaching without Teaching.

Summary of this week’s videos:

• U03: Lessons for the Masses from the Trenches of Co-dfns (Aaron Hsu, Indiana University)
• D13: Uncle Andy’s Fireside Chat v5 (Andy Shiers)

Follow

Week 7 features talks by two recent additions to the Dyalog team. Richard Park joined Dyalog a year ago, and his primary focus is the production of new teaching materials. Nathan Rogers is the newest member of our US consulting team and is based in Denver, Colorado.

Jupyter notebooks have recently become a very popular mechanism for publishing scientific and technical content. In addition to nicely formatted text and graphics, notebooks can contain executable expressions in a growing collection of programming languages – including Dyalog APL. In his talk at Dyalog ’19, Richard shows that it has become really easy to get started with notebooks containing executable APL code. Thanks to recent work that he has done, you can even get started without installing anything on your own machine!

Richard Park shows a new way to access Jupyter documents

Nathan Rogers presents APL2XL from the other side of the ocean

Excel workbooks are nothing new; the first version of Microsoft Excel appeared in 1987, only 4 years after the release of Dyalog version 1.0. For decades, users have used OLE Automation to interact with Excel and create workbooks. However, this is not a suitable technology for use on servers (even Windows-based servers). Nathan could not attend Dyalog ’19 in person due to a theatre production in Denver where he was a member of a team dancing tango, so he had to present his APL2XL project via a remote connection. The goal of this open source project is to create Excel workbooks (.xlsx files) under Windows, Linux and macOS, without any external requirements other than Microsoft .NET compression libraries.

Summary of this week’s videos:

• D16: Creating your First Dyalog Jupyter Notebook (Richard Park)
• D12: Exporting Simple Microsoft Excel Tables from APL (Nathan Rogers)

Follow

Claus Madsen of FinE Analytics

Scrinium is a portfolio management system that can handle a comprehensive range of financial assets, collect them into “portfolios”, and compute returns (SR, ANN, TWR), risk (Std, VaR, Duration, Convexit, Delta, Gamma, Sharpe) and relative risk (Alpha, Beta, Jensens Alpha, Tracking Error). It deals with benchmarks, and so on and so forth. However, Claus Madsen’s presentation at Dyalog ’19 doesn’t delve deeply into financial maths. Instead, his talk is mostly about the architecture of Scrinium. To allow himself to focus on all of the above-mentioned computations, Claus has organised all of his computations into classes, which he exposes as Microsoft.NET assemblies. This allows him to leave the production of user interfaces and other wrapping to “IT people”.

The other two presentations featured this week are about the statistical package TamStat. TamStat has a new graphical user interface based on the HTMLRenderer, which means that the interface is identical under Microsoft Windows, macOS and Linux.

Richard Park talks TamStat

Stephen Mansour was unable to attend the Dyalog user meeting this year as he was busy teaching statistics using TamStat at Scranton University in Pennsylvania, US. In two talks at Dyalog ’19, Richard Park and Michael Baas provide two different perspectives on the new TamStat UI. First, Richard talks about the fundamental design of the underlying statistical language and how the UI guides inexperienced users and helps construct executable TamStat statements. Michael follows up with a talk on the implementation of the “Wizards” and other features of the new UI that he has worked on during 2019 in collaboration with Dr. Mansour.

Summary of this week’s videos:

• U09: A Transaction-based Portfolio Management System in Dyalog APL using .NET (Claus Madsen, FinE Analytics)
• U04: TamStat 2019 (Richard Park)
• U05: TamStat 2019 – Technical Update (Michael Baas)

Follow

Francesco Garue of SimCorp Italiana

In week 5 of the Dyalog ’19 videos, Francesco Garue tells how, after APL Italiana was acquired by SimCorp and became SimCorp Italiana, the two development teams got to know each other by sharing some code. At the time, APL Italiana’s SOFIA product contained a “Rebalancer”, a central component of Asset and Liability Management systems, capable of adjusting the contents of portfolios of financial instruments to bring them into alignment with investment rules or limits. The rebalancer had functionality which could be useful to add to SimCorp Dimension. Francesco tells the story of how a team of Danes, Ukrainians and Italians worked to extract and refactor the components so that it could be shared by the two products.

Geoff Streeter tells the story of shared code files

The two Dyalog talks this week are both about improving performance. Geoff Streeter explains the ideas behind a new way to save workspaces in a format known as a shared code file. Shared code files can be memory mapped, making the contents immediately available to the application, even though only a small part of the file is actually loaded. Functions in a shared code file are only loaded from file if/when they are used. For applications that have a large body of code, only a fraction of which is used in any particular user session, this can mean a huge reduction in application start-up time. If several processes run the same application on a machine, then process starts after the first one run faster still, because the most frequently-used code is already loaded into memory. As code is shared between the processes, there is a reduced memory footprint in addition to the increased performance.

Marshall Lochbaum

One of the most widely used mechanisms for iteration in APL is the reduction operator, which injects its operand function between elements of an array. For example, (+/1 2 3 4) is the “plus reduction” of the first four positive integer, computed as 1+2+3+4. In his talk on “Implementing Reduction”, Marshall Lochbaum explains how, for many of the most commonly used reductions using functions like + - × ⌈ and ⌊ on integers and doubles, and a much larger set of functions on Boolean arrays, he has worked to make the reductions run as fast as possible. To Marshall, that ideally means “as fast as the arguments can be loaded from memory”.

Summary of this week’s videos:

• U08: Getting to Know Each Other by…Sharing Code (Francesco Garue, SimCorp Italiana)
• D07: Shared Code Files (Geoff Streeter)
• D09: Implementing Reduction (Marshall Lochbaum)

Follow

Welcome to week 4 of the Dyalog ’19 recordings! This week, Tommy Johannesen of Jersie Data tells us about an interesting application that he has written in Dyalog APL. In the Copenhagen area, many school children are “fed with APL” in the sense that Tommy’s APL system connects the parents of ten thousand hungry children to about 60 vendors of school lunches. The service currently handles about 100,000 users with a system that runs under IIS and uses Dyalog as an ASP.NET implementation language. Each year 10,000 children leave after 10 years at school and another 10,000 enter grade 0; over the 30 years that Tommy has been in business, he has served more than a quarter of a million users.

Tommy Johannesen of Jersie Data ApS

Peak time is Sunday evening, when parents and children log on to order meals for the coming week. Tommy tells the story of how he has been struggling with a memory leak that caused some user requests to fail when loads grew large; obviously not a good thing! Since upgrading to Dyalog version 17.0 he has been chasing this problem together with John Daintree – it was not completely solved at the time Tommy was at Dyalog ’19, but I am happy to be able to report that we believe it finally bit the dust in October!

The other two recordings in week 4 are centred around the topic of source code management. First, Adám Brudzewsky and I introduce the tool called “Link” which is included with version 17.1. Link creates a link (hence the name) between each function (or other “code object”) in an active workspace and a corresponding text file. The namespace structure of the workspace corresponds to the directory structure that the workspace is linked to. Changes made to either side of such a link – to code in the workspace or to files outside it – are immediately reflected on the other side.

Paul Mansour demonstrates Git integration with AcreTools

The intention of a tool like Link is to enable the use of source code management systems like SVN or Git to manage source in the form of text files. These source code management systems allow the implementation of many different “workflows”, defined by parameters like how branches are used, how frequently merges are done, and how conflicts are handled. Git allows a lot of freedom, which many small teams (and that includes one-man-bands 😊) don’t need. Immediately following the talk on Link, Paul Mansour took the stage at Dyalog ’19 to talk about tools that he has developed to implement a simple but effective Git workflow, suitable for many APL projects.

Summary of this week’s videos:

• U01: Feeding Children with APL (Tommy Johannesen, Jersie Data ApS)
• D05: Link (Morten Kromberg and Adám Brudzewsky)
• U02: A Git Workflow for Dyalog APL (Paul Mansour, The Carlisle Group)

Follow