Most of the recordings that we are releasing in week 5 concern “best practices” for the use of APL in large or complex systems. Many of these are mentioned in the invited keynote Dyalog APL in the Biggest Data Centres at the Heart of the Investment Management Industry by Oliver Lanz from the product development team at SimCorp, one of the biggest users of Dyalog.
In his talk, Oliver gives an overview of the architecture of Simcorp Dimension, which is the backbone of more than 50% of the top investment management firms worldwide; it manages over €35 trillion combined. Oliver explains the position that Dyalog has within their architecture, and how shared code files, the .NET bridge, multi-threading, and Simcorp’s homemade type checking system, all contribute to making the system secure and efficient. He introduces many of the topics discussed in the other talks released this week, and explains why it is important to Simcorp that Dyalog Ltd and the APL community develop and use such tools.
One of the points raised by Oliver Lanz in his talk is that developers are increasingly required to use automated tools to measure code quality, and document this to auditors. As applications are exposed to the outside world over the internet, an important goal of automated analysis is to detect and prevent vulnerabilities that would allow attackers to access or modify data without permission. In his talk Static Analysis of APL in APL, Brandon Wilson describes the work that he is doing with Aaron Hsu to teach the co-dfns parser about traditional functions so that it can parse existing APL application code and enable static analysis of APL code. Aaron follows up his compiler and parser status in Co-dfns Roadmap and Updates.
As APL applications become part of elaborate architectures where APL code provides and/or makes use of other services, another requirement that is emerging is the provision of telemetry to support the monitoring. This can be used to track performance over time and – when things go wrong – troubleshoot system failures that involve calls between multiple components, potentially implemented using different technologies. In their presentation on Telemetry and Protobuf, Gilgamesh Athoraya presents how OpenTelemetry can be used from Dyalog and demonstrates a sample application that emits telemetry data. Telemetry data can, optionally, be emitted using Protobuf (Protocol Buffers), a mechanism for serialising structured data. Sandra Persson joined him to present an APL implementation of Protobuf and a plugin that generates APL code from schema files.
Kai Jaeger is the most prolific tool builder in the Dyalog community. He is the author of many useful tools, both for use at runtime and during development. He has contributed the vast majority of the packages available on the Tatin package manager (which is also his work). In Developing in Dyalog with Modern Tools, Kai takes us through “a day in the life of an APL developer”, demonstrating the most important tools that he uses on a daily basis, to make development both easier and safer. Amongst other things, Kai demonstrates Git, GitHub, Cider, Tatin, Snippets, CommTools, FiRe, APLGit2, Testing (with code cverage), and CodeBrowser for code review.
I must admit that Brandon Wilson’s Data Parallel Proof Verification was not something I thought that I would fully appreciate, as my understanding of proofs and how to use them is extremely limited. However, I was still intrigued by the insight (obvious once Brandon has pointed it out) that proofs are tree structures, with a top-level proof being constructed from several smaller proofs. Brandon has been experimenting with using co-dfns/Hsu-style tree wrangling to validate proofs in parallel, and the results are very promising. As mathematicians move in the direction of software systems to make proofs more reliable, it matters a lot whether your new proof can be verified in a split second or it takes hours or even days. Although there are few current APLers who need a proof verification system, this could help make APL more visible to communities who are currently unaware of it.
——————————————
This week’s videos:
- U01: Dyalog APL in the Biggest Data Centres at the Heart of the Investment Management Industry (Oliver Lanz) – PUBLICATION POSTPONED
- D06: Static Analysis of APL in APL (Brandon Wilson)
- D13: Co-dfns Roadmap and Updates (Aaron Hsu)
- U02: Telemetry and Protobuf (Gilgamesh Athoraya and Sandra Persson)
- U11: Developing in Dyalog with Modern Tools (Kai Jaeger)
- D14: Data Parallel Proof Verification in APL (Brandon Wilson)
Materials for all presentations can be downloaded from the Dyalog ’24 webpage.
Follow