If you have not already been there, I highly recommend visiting the APL Orchard (apl.chat). Adám Brudzewsky originally started this chat room on Stack Exchange to teach and answer questions, including many successful introductory sessions to people who wanted to learn APL. Since then, it has become an extremely active discussion forum, with a very wide range of interesting conversations, from regularly helping more newcomers get started with APL to theoretical discussions about the design of future array languages that will improve APL and perhaps compete against it some day.
One of Adám’s infamous invitations to a personalised APL introduction
Several of the most active participants have written their own APL implementations and some are working on array languages that are quite different from APL. It is a lively crowd, generating lots of thought-provoking discussions that feel a lot like the big arguments that I remember witnessing between Iverson, Benkard, Bernecky, Falkoff, Trenchard More and others at APL conferences back in the days when the APL language was still being born.
The participants include a mix of current, past and future employees of Dyalog Ltd, and it is a source of much useful inspiration for future work on our product and healthy challenges to conventional wisdom, so I try to spend some time there every day.
APL Orchard message statistics. The bottom contains the all-time total number of participants and messages.
To the New APLers
The Orchard is sometimes a challenging environment for a Dyalog CTO. Many of the bright young minds with computer science backgrounds that arrive here are quick to latch on to the fact that many things about APL go against the grain of what they were taught at university, and that a few of the design decisions made during the 55 years since the first APL implementation were questionable and could do with rationalisation. There is a constant barrage of complaints that we are not working hard to “fix” the things that they consider to be wrong about APL. This is, of course, how things should be when youthful enthusiasm meets us “fossils”; I fully understand many of the points made and actually agree with quite a few of them.
It is sometimes hard work having to repeatedly defend why Dyalog does not do things like just fix index origin at 0 and give the customers a few years to refactor their code. The implication is often that we are incompetent, and occasionally there is the insinuation that we are driven by questionable commercial motives like trying to get rich by locking the customers in and then doing a minimum of work.
Evolution, not Revolution
In my opinion, a large part of the value of our product rests on the fact that our customers can do things like read new legislation that affects the way their code needs to compute something, write code to deal with it before lunch, and then expect the code to keep working without changes until the law changes again, which might be in four years (depending on which country you live in), or in a few decades, after the current programmers have retired. Meanwhile, the code might need to move from the mainframe via UNIX and DOS to Microsoft Windows or macOS and then on to Linux in the cloud, and whatever comes after that.
APLers actually do enjoy refactoring code *if* it is in order to meet new challenges in the markets that they serve with products based on APL or compute better results. Our duty is to make this easy for them. On the other hand, there is no business value in refactoring code because we decided to change the way the language works, due to theoretical considerations – and it is our duty to protect them from that.
Breaking changes to Dyalog APL are quite simply not an option, at least not if users have to take immediate action. When we accidentally make a breaking change despite our best efforts not to, there is much wailing and gnashing of teeth, and we have to drop everything to fix it. That doesn’t mean we can’t make significant changes to how things work, but we must take an evolutionary approach, where the old way continues to work until everyone who needs it has moved on, possibly controlled by (sometimes undocumented) switches, like the one that forces APL to continue to give a DOMAIN ERROR on ¯1*0.5 to protect legacy code that relies on trapping the error in certain financial calculations.
Over the past decade, we have been able to move almost all of our user base from a 32-bit product with a fixed “alphabet” of 256 characters to using 64-bit APL with full Unicode support, in many cases without requiring any changes to application code. The stragglers are mostly major clients with support contracts that justify the additional cost of continuing to support the “legacy” version of the product until they are eventually able to move on. Once the Raspberry Pi stabilises as a 64-bit platform, we may finally be able to completely retire 32-bit Dyalog APL. I estimate that the “Classic” (non-Unicode) version probably needs to exist for another 5-10 years.
The next big challenges for us are to ensure that code can be moved to macOS and Linux (and “the cloud”) with a minimum of changes, and that both new and existing users have ways to integrate both new and legacy APL code nicely with modern source code management systems and continuous integration pipelines.
Existing Customers First
Our first priority at Dyalog is to support our existing clients and make sure that they remain competitive in their respective marketplaces.
Some contributors to the APL Orchard have suggested that this prevents us from being able to attract new users, which they believe can only happen if we quickly fix some of the “warts” in the language, or add some even more powerful language constructs.
First of all: not a single person who I consider to be a real prospect for writing substantial new application code in APL has brought up a single one of these issues as a reason why they might not want to adopt APL.
Secondly: if we do not first ensure our own financial stability, we will not have the resources to perform evolution, let alone revolution. For example, our investment in the creation of the APL Orchard, the APL Wiki, and the creation of new training materials for potential new users of APL, all depend on this.
As they say in the pre-flight briefing: put on your own oxygen mask before helping others.
Third: it is my very strong conviction that making breaking changes would violate the trust that exists between Dyalog and its customers, and I would personally consider it to be unethical. This may sound like a radical position, but it is one that I would expect to resonate with the younger generation. My personal motivation for working on software stems from witnessing the value that is created when people use software that I have helped create. Once someone starts using “my” product (and they continue to indicate that it is important to them by paying for support), I have an obligation to protect them from harm, if it is in my power.
Conclusion
Old timers: If you are looking for some interesting discussions about current and future array languages, to help some newcomers, and maybe learn some new tricks yourself, I suggest that you head over to the APL Orchard and dive in, it is a lot of fun! You might want to check out apl.wiki/APL_Orchard, which has some helpful hints and tips, before you start.
New folks: I hope that this explanation of some of the philosophy that drives our work at Dyalog helps to explain some of the decisions that we make and that you find to be puzzling. I fully recognise that it is hard to imagine that a software company might have customer relationships that started before you were born 😊.
There are a couple of other topics that I should possibly also explain my views on, such as the dilemma of whether APL is a Notation or a Programming Language, and why Dyalog APL is not open source, but this post is very long already. Depending on the response I get to this post, I may return to that in the future.
Follow
I entirely agree with the your emphasis on keeping things working for existing customers who have built their systems, and are running their businesses, on your language and environment. I thank you that code I wrote on Dyalog 7 still works in today’s version, coming back to APL after a long hiatus in “boring land”.
I concur. (My first contact with APL happened I think in 1973 on an IBM time-sharing system.)