Monday at Dyalog ‘15

There is so much going on at Dyalog ’15. The general theme of the day was set with Morten Kromberg’s Technical Road Map. Dyalog is constantly refining and expanding APL’s capabilities and ease of use, quickly adding functionality to tie into new technologies, and adapting to emerging trends. Here are some highlights from the day:

ride-blogRIDE: A New Way to Interact with the Interpreter

Nick Nickolov gave a humorous and informative presentation of his big project – a brand new interface to Dyalog APL, separating the front-end from the actual interpreter. Built on established modern technologies, RIDE (Remote IDE) is inherently cross-platform, ensuring a quality front-end to Dyalog for all the systems that the future may bring. Nick inspired several old-IDE-using members of the audience to try out this new and upcoming interface.

nicegraphCore Performance

While 14.1 did not bring many new language features like 14.0 did, there were many less visible changes. As the ratio between blue and red areas in the above chart shows, significant work has been done on performance: Jay Foad and Roger Hui enlightened us all with a peek behind the curtain on the challenges and pathways to improving Dyalog’s performance.

IMG_5034Cross-platform User Interfaces

Morten Kromberg introduced us to MiServer, Dyalog’s new tool for creating UI that is rendered in a browser, and is therefore inherently platform-independent. MiServer is already very usable – in fact a number of brand new applications are under development. However, MiServer continues to be under active development – new features are on the way.

IMG_5040Cross-platform File Functions

Richard surprised everyone (including Morten) with a working prototype of version 15.0 that included a whole suite of long awaited system functions that will make it easier to interact with the native file system in the same way on all platforms without code changes. Functions include (but were not limited to) listing of files, creation of folders, and automatic parsing of path-filename-extension strings.

How to Save an Italian School with APL

Blog post from presentation by Roberto Minervini with 4 of his students; Arianna Greco, Samuele Luigi Di Gioia, Kariman Admed and Silvana Maria Belén Colman Cabrera – all from Liceo Classico S.M. Legnani, a Public Secondary School.

By Vibeke Ulmann #Dyalog15

Getting the message of the power of Dyalog APL to young people, is one we have worked with for many years, as we need to recruit the next generation of APL programmers into our community.

Today was exceptional for the Dyalog User Meeting here in Sicily, as we had the pleasure of welcoming 19 young students  – all around the age of 14 – who have been taught Dyalog APL for the purpose of using it for Maths.

Roberto Minervini has a background as an APL programmer, having worked 5 years in APL Italiana. Last year he left the Company to follow his dream of becoming a teacher.

He persuaded the school to let him teach a Dyalog APL class, and a team of 25 young people enrolled in extra classes after School in order to learn the language.

The presentation they gave today was enourmously fun with video clips with music, dancing and general ‘chaos’, which –  from my point of view –  seems to be the best way to have fun whilst learning a new (programming) language.

Roberto and his Students

From Roberto’s point of view, he told us that we are faced with some serious issues when it comes to teaching our young people. There is increased competition between Schools to produce good results (i.e. higher number of students with high marks), there is a rigid curriculum which has to be followed, and it does not allow pupils to choose subjects they are interested in, or think would be fun. ‘Bad results’ is punished by ‘Bad marks’, which further de-motivates students. Bascially, today’s school and teaching is using methods that were developed for the age of Industrialiam. Whilst politicians have tried to introduce the concept of La Buona Scuola (The Good School) in Italy,  there is confusion between what the goals and intentions of the Poitics behind the concept is, and how the teachers experience their everyday work.

Having used APL for many years, Roberto knows that APL is a ‘Tool of Thought’, and he decided that it would be better to try an engage the students in solving mathematic problems than teach them straight forward ‘maths by the book’.

Taking their offset in  Kakuro – a Maths game which is the equivalent of a Cross Word puzzle, and an article written by Dan Baronet, as well as using other ‘guru’s from the world of teaching, writing and illustration/drawing-  the class set to work.

According to Arianna, “we have no computer classes at school, so this was a completely new experience for all of us. We met once a week at lunch to discuss our progress and we quickly realised that with APL you can codify logic. I have personally found the language of the future!”

Kariman said, ” We started the project without any money at all – and although it was a bit strange to learn the APL squiggles – once you grasped them, it was fun.”

Roberto stated that in today’s society, failure is considered to be bad – and is hence punished with bad grades. The result is that pupils give up, and do not learn enough mathematics. On the other hand, we also know that in Technology and Science, failure is considered a good thing, because it forces you to learn new things and find new ways. So by implementing this approach instead, we were effectively “building the road as we walked on it.”.

According to Roberto, “We also took our offset in the book  “A Matematician’s Lament” – by Paul Lockhard – which supports intuitive learning of math. Maths is really an ‘art’ – but today is taught in a boring way, with fixed exersizes  and no real problems to solve.”

Samuel said that, “How to solve a problem involves deduction versus Induction.”

The process the class used for learning to solve problems using Dyalog APL was this:

If you cannot solve a problem – then there is an easier problem you can solve, so go find it!

However, Roberto warned, when teaching APL do NOT start with symbols. Learning APL is a Journey and once you have started this journey and apply it to learning Mathematics, maths almost becomes Utopia.  The trick is to start with Key Words and key symbols, then practise and continue to practise, until it become ‘natural’.

Asked about what is the coolest thing about APL – the answer was a unanimous: It can do things by ilself!

Asked about what the biggest challenge had been, the answer was ” To learn the language and try to use it at home. You learn APL by using it, not by describing it!”

The students had also found that problems presented in the Programming contest 2015 had been helpful.

I can only encourage our blog readers to check back on the Dyalog website when the Videos from Dyalog’15 are posted and enjoy the presentation from Roberto and his pupils.

The Importance of Communication, Transparency and Practice

Blog post from Presentation by Gianfranco Alongi from Ericsson AB

By Vibeke Ulmann #Dyalog15

Those of you who have been following my blog posts from Dyalog User meetings the past couple of years, will know that I am a always alert when we take a departure from talking about ‘coding’ or ‘technology solutions’, and ‘target achievement in production environments’. And this is usually what Gianfranco is challenging us with in his presentations. Today was no exception.

In his pursuit of mastery for teams of programmers Gianfranco had been looking into the softer issues this year. Performing mob-programming where all team members share the same computer while solving problems issues with communication becomes very important. Gianfranco pointed us to Non-Violent communcation (also known as Giraff language) as a way to improve communication within a team and not least applied to one self as a mean to self insight.

Moving to Transparancy-  on a personal level this is about being more open as a person not hiding your emotions and your true self at Work.

Gianfranco used the Johari Window as illustration – inserted below:

The Johari Model

By allowing yourself to be more open about your personal life, feelings etc. team members can forge a higher degree of trust among each other – almost to the degree that they can feel like a ‘family’. The tool is an awareness tool to promote more openness between co-workers.

On a corporate level transparency is about making everyone aware of the company’s strategy and direction, enough to let it guide you in all of the decision you have to make on the company’s behalf every day.

Lastly: Practise makes perfect. How do we get better at the personal level. Gianfranco mentioned the now widely established fact that contrary to all business school knowledge money does not incentivise knowledge workers. On the contrary, trying to reward or using stick/carrot methods to achieve better performance with this Group of people slow them down. What does motivate “us” is Purpose, Mastery and Autonomy.

So to achive mastery in a team the most important things to be aware of is to make people feel part of something bigger  – and preferably beautiful –  and to allow them the freedom to nurse their mastery by practising alongside their work.

The most discussion afterwards was caused by a fact mentioned by Gianfranco – that intelligence tests and aptitude tests fail to reveal who will later become experts!
Only the amount of pratice determines whether you become really good at coding.

Premium Capping Schemes: An important steering Mechanism in German Health Insurance

Blog post from presentation by Dr. Markos Mitsos – Deutsche Krankenversicherung AG DKV – ERGO, Actuarial Department @ Dyalog ’15.

by Vibeke Ulmann

DKV has many million health insurance contracts. Each tariff of each contract has to be checked and possibly adjusted annually in terms of insurance premium based on a number of criteria – without imposing undue ‘financial’ hardship on the Insured individual. Dr. Mitsos is in the process of migrating the simulation modelling into Dyalog APL.

The German Healthcare system is unlike any other in the world. If you earn more than € 45,000 per annum you can choose between the Government Healthcare scheme or draw up a private health insurance. If you are in the state health system you have the option of adding private health insurance on top.

There is legislation in place to ensure that Health insurance companies do not calculate premiums arbitrarily. When entering into a contract the tariff for your health insurance is calculated such that it stays the same over your lifetime. This means that young people pay more than the insurance needs to cover their healthcare expenses whereas older persons are drawing on the reserves that has been saved for them.

The two illustrations below show examples of contract lifetimes

Illustration of correlation between typical annual premiums and claims 07-09-2015

Illustration of  liability high end premium versus annual net premium  and age 07-09-2015

With millions of contracts DSV has Billions of Euro in reserve which they have to manage to the best of their ability. 90% of the proceeds from fund management plus unspent security fees of 5% on each contract has to be paid back to the insurance holders within 3 years and how this is done is the only area where the insurance company has a little freedom to decide how to best distribute the surplus (the socalled “war chest”) to the insurance holders.

If the premium recalculation  shows that some contracts need to have a huge premium increase, DKV, can use part of the  ‘war chest’ to help fund (read cap off) this increase after agreement with an independent Trustee. (Unabhäniger Treuhänder)

The actuary department of DKV – undertakes simulated calculations on the costs of the capping schemes. The results go to the  Executive Board of ERGO and they go to the Chief Actuary in DKV, who is personally liable for the adjustments being in line with the taxation rules, the legislation, and that it is reasonable to impose a premium adjustment accordingly to each individual contract. The final sign off is done by the independent Trustee and then they are implemented into production.

The calculations are typically done on chuncks of around 1 million contracts, and although the maths is ‘simple’ in Actuary terms, the number crunching is massive – thus rendering APL a fast and precise modelling tool for ERGO.

In the ‘old’ days a simulation modelling calculation used to take days. However, according to Dr. Mitsos the simulation calculation done in APL takes mere minutes – and ERGO is  looking into further speeding it up. The actual figures are presented in Excel, and it’s the posting of the number to Excel which actually takes up most of the time.

My main take away from this brilliant presentation –  and subsequent conversation over lunch  – is  the absolutely fantastic number of complex assumptions which have to go into simulating the annual checks and recalculations for premium increases. This becomes especially impressive when you take into consideration that the German Government does NOT allow the Insurance companies to make any forward projections on how they assume/expect a contract will develop in terms of claims over time. Everything has to be calculated ‘as the situation is’ in the year in question.
The way the system is constructed is such that in order to make money as an insurance company you will have to service your customers well.

Parallel Programming with Futures/Isolates

DSC05195Morten Kromberg went right to it with his well-prepared workshop on the prototype features that were introduced in version 14.0 of Dyalog APL. The features are still partly modelled in APL to allow the first wave of users to pass judgement on the design. Throughout the workshop, we were asked for feedback, to ensure that the new primitives will be as well implemented as possible.

Everyone is encouraged to apply the new tools in practice so they can be adjusted and possibly get expanded functionality based on feedback from actual use-cases.

There is no denying the the obvious performance gains that Morten demonstrated: He was able to let the computer waste 9 idle seconds in only slightly more than 3 seconds!

Joking aside, one practical example that showed how useful parallel processing can be, was collating character counts from online news sites. Because of latency in loading these pages over the internet, starting many parallel threads that each requested a page greatly reduced the wait time. DSC05197 One workshop participant volunteered a piece of code that needed to be parallelized got up and pair programmed this with immediately achieving significant performance gains.

The idea of Futures is allowing the continuation of other work before an asynchronously launched calculation is finished. All selection and structural functions can be used on an array containing futures without blocking to wait for the result, as long as no actual data values are required. Meanwhile, other processes (or computers!) could be finishing the evaluation in the background.

It was captivating to observe output being printed and check the status while the calculation work progressed, or accessing parts of an array while other parts still reported as in progress. Morten even had a graphical representation of how load is distributed across processor cores.

Maybe the most exciting thing about all this is that we will soon welcome several new APL squiggles; Isolate (¤) and Parallel (). And at the end of the day, Morten even pulled a pet (pun intended) card out of his sleeve: A Schrödinger’s cat operator! The symbol is of course a question mark in a box (), signifying how the evaluation of the cat’s life (or in our case – the result) does not even begin until actually queried. This is a novel way to specify specific lazy evaluation.

All in all, an exciting look at what possibilities for parallelism the future will bring.

Building a Web Application with MiServer

IMG_4875
One of Sunday’s tracks was Brian Becker’s engaging workshop on MiServer 3.0 – the newest version of Dyalog’s APL-based web application framework that allows you to develop a cross platform application for stand-alone use or delivery via the web.

The motto of MiServer is that everyone who can develop an APL application should also be able to make it available via the web, allowing users to access the application via their favourite browser. In other words, MiServer is browser and platform agnostic.

The main take-away from the morning’s workshop is that if you are used to developing your GUI using ⎕WC, developing the GUI in MiServer 3.0 creates some slightly different rules. First we were showed where to download MiServer, and then Brian guided us though building out very first one-page website. The interactive nature of the workshop captivated us all. Before we knew of it, it was time for lunch.

IMG_4869After lunch – instead of having a sleep inducing talk, Brian challenged us to build a website for a Big Brian’s Burger Bistro offering ordering facility for various products with side orders, and a dashboard to monitor the ordering, product category, speed of order processing, and – most importantly from a management point of view – keep an eye on revenue and which products generate most revenue, with auto update when new orders where processed and paid.

It may have been down to the eating habits of the delegates in terms of their love for burgers, fries etc., but the enthusiasm with which we attacked the challenge was probably more due to how Brian managed to whip up and present MiServer 3.0’s capabilities during the morning. Actually, we got so engrossed in the afternoon that we almost missed our coffee break!

A group of participants was tasked with creating a dashboard, and after some discussion decided on two tables, which would pull data from the business logic, do a bit of calculation, and auto update when new orders got processed.

Hugely entertaining, and we actually managed – courtesy of Rick Proctors swift keyboard handling and deep understanding of APL – to get the code for the dashboard done, working and showcased before we ended the workshop.

We are all looking forward to the MiServer UI Controls presentation on Tuesday morning where we will get a chance to see some the many fancy widgets that MiServer 3.0 offers.