After many months of planning and preparation it was finally time to head from the UK to Sicily for Dyalog ’15. Sun, sea and an excellent conference facility were awaiting us at the Atahotel Naxos Beach in Giardini Naxos and this year, because it would be very hot, we had decided to forgo the traditional Viking Challenge team activity for a more relaxed excursion by coach to see the craters at the top of Mount Etna – Europe’s largest active volcano.

Dyalog ’15 kicked off on Sunday morning and by Tuesday everything was going to plan and we were stubbornly ignoring the forecast for storms and rain on Wednesday – after all, the forecasters always over-exaggerate don’t they?
In this case it seems not as we woke up Wednesday morning to torrential rain and had to wade from our rooms (sometime through water up to our knees) to the conference centre.

There we were greeted by more floods and rain only this time it was inside the main conference room and perilously close to the desk with all our AV equipment on it. Time for an emergency change of room but where do you fit 80+ people at short notice? A room was found, the projectors and microphones were set up ready for the start of the day – and we were only 15 minutes behind schedule! OK, so maybe it was a tight squeeze but by then everyone was bonding over flood stories and determined not to beaten by a “bit of weather”.

Just when we thought it couldn’t get worse, we then found out that the beautiful seaside terrace restaurant, where the Dyalog ’15 banquet was due to take place that evening, was also flooded so Banquet Plan C (Banquet Plan B was also under water) was implemented and the whole event moved indoors – not such a stunning location but at least we would all be dry.

During the morning the rain eased off, the floods subsided and our planned excursion to Etna during the afternoon was looking more and more likely. We finally got the go ahead and 69 brave souls set off after lunch to battle with the elements, optimistically hoping the weather would improve as the afternoon went on. However, once we reached 1900m we were greeted by yet more torrential rain and a pretty bad thunderstorm. Were we deterred? Oh no, we just headed off into the storm to stomp around the Silvestri Craters for a bit – although one delegate did, rather hysterically, point out that it probably wasn’t very sensible to be waving an umbrella around in a thunderstorm at that height. Eventually a very wet, cold and bedraggled bunch got back on the coaches but we all survived the adventure, made it back to the hotel and most agreed we had had a lot of fun!

And the moral of the story? It was beautifully summed up by Gitte Christensen in her closing speech: “If you don’t have a Viking Challenge at a Dyalog User Meeting then one will be provided for you!”

Follow

MiServer UI Controls

Dan Baronet and Adám Brudzewsky continued yesterday’s presentation by Morten Kromberg on Cross-platform User Interfaces by demonstrating a lot of varied interface elements that are available when creating front-ends in MiServer. Especially encouraging were the special controls designed by Dyalog. These make it possible to have arrays render as complete super-elements (e.g. a table). No need to worry about the many underlying HTML elements making up the super-structure (e.g. table rows and their table cells).

The Dyalog Project Project

Morten Kromberg outlined his plans for a brand new tool to ease development and getting started with APL: An online repository for APL code – software packages that may be retrieved and installed on a computer. This is a significant step in ensuring the future, as a newcomer to programming (if ignoring all the other advantages to APL) often looks at amount of easily available pre-existing code when choosing a language.

Data Binding – The Matrix / Waste Time with John Daintree

Light-hearted as always, John Daintree introduced us all to Dyalog’s already implemented capabilities in making user interfaces automatically update when values the datatable are changed. Obviously this is best described in a video, which will be available soon on video.dyalog.com. Meanwhile, there is an excellent demo on APL Wiki. After John’s presentation, a user demonstrated how he with minimal time effort set up a modern-looking dashboard for presenting numerous varied data.

External Workspaces

Morten Kromberg introduced a new facility by which you can save your code and access it as a memory mapped file which can be shared by the users. This is an advantage when you have to watch how much of the workspace is consumed by the code-base. Stig Nielsen from SimCorp told us how they have applied these external workspaces with much success. Not only can many users access the same workspace, but even users on a Citrix farm can access the file simultaneously. Using this facility, enables SimCorp to start using futures and isolates that would otherwise exhaust the memory available on their costumers’ server farms.

Follow

Blog post from presentation by Stephen M Mansour PhD, The Carlisle Group as well as the University of Scranton,  and Jerry Brennan PhD, A former teacher of Statistics in Hawaii, who has now moved being a fulltime Psycologist.

By Vibeke Ulmann #Dyalog15

Last year I managed to squeeze in a mention of TamStat in a blog post. This year we will do a bit more on this new application which has been developed by Stephen Mansour. The name TamStat stems from ‘Taming Statistics’ and Stephen willingly admitted that he stole the paradigm from Ken Iversson who once talked about ‘Taming Maths’.

Stephen teaches Statistics at the University of Scranton – outside as working at the Carlisle Group. The basis for the development of TamStat was that he wanted to create a one or two semester stats course at the University. In order to create what he considers to be an easy to use, APL based statistics programme, he has included a seamless interface to R.

TamStats functionality can be divided into four different groups:
Summary statistics
Probability Distributions – based on theoretical models,
Relations and
Logic

The underlying implementation is a set of operators representing the different distributions with operands specifying the conditions and the Relations, or Logical conditions which apply

As an example Stephen then conducted a comparison of a Movie Theatre with 130 seats, with an average of 100 attendees  and a 20 seat deviation. He ran the comparison in Excel, Minitab, R and TamStat to show the audience how easy it was to create a close to normal English language expression in TamStat to get the statistical result.

Since last year Stephen has been adding a lot of new features and not least documentation. There is now a 60 page user guide both in print and as a PDF download, a Reference Card  in fold out hardcopy, based on the same structure as the Dyalog APL reference card. The reference card is also available as a download. There is an Excel import functions, and operational R interface. It is now possible to work with Bayesian statistics – a request from last year.

TamStat is now available in 3 versions (almost 4) namely

• A Dyalog workspace for APL users
• A downloadable standalone App for students and non-APL users (with ASCII input).No set-up required just get it and start.
• A Web application  – similar to TryAPL – using MiServer for casual users
• A Computer programme for professional users (currently under development in Carlisle Group)

Jerry Brennan then took over demonstrating his JMB.APLcloud.com, which is an educational site teaching APL. Implemented in APL of course and using MiServer as the vehicle to bring the content to the web.

Jerry is currently running on MiServer 2.1 and he is in the process of moving to MiServer 3.0.

As Stephen, Jerry has made available documentation, and examples, in the form of games and other ‘fun’ examples to attract the attention of a young audience. He has included drag and drop facilities, to make it even easier for people who have no prior knowledge of Dyalog APL to get going.

Jerry has included TamStat on his website including all the examples from Stephen’s User Guide allowing the users to interactively experiment with the statistics.

Outside of this, he is working with ‘Hacker Highschool’ who work with security awareness for teens and have 1.4 Million subscribers. Hacker Highschool takes an extremely ethical approach, so he is not concerned about making the application available to students.

Stephen Mansour would like to run a workshop on his “Statistical Language” at the Dyalog User meeting next year if there is sufficient interest, so asked delegates and blog readers to let Dyalog know if we should include this in the next User Meeting.

Follow

Blog post from Aaron Hsu’s (Indiana University, USA) presentation Wednesday 9th September.

By Vibeke Ulmann #Dyalog15

For the past 2-3 years Aaron and Dyalog have been collaborating on creating a new compiler. This is an Open Source project and the compiler – which is currently in ‘Version 0.5’, is available as a zip download from Github. Aaron started by showing us how to download, unzip and set-up the compiler. It is available for both Windows and Linux platforms.

This was a live coding session (working on the Window platform) where Aaron demonstrate some of the feature/functionality expected to go into Version 1 of the compiler. He then proceeded to do a number of blackscholes benchmark, working various Co-dfns implementations, in order to demonstrate the order of magnitude of speedup’s for chosen expressions. The performance speedup’s achieved were  60% -93% or, as Aaron prefers to see it, 3 – 10 times speed-,up , so significant improvements can be had by using the compiler. However, there is still work to be done in terms of implementing more expressions, and Aaron encouraged everyone to download the compiler, play with it, and most importantly – try to break it – as he needs feedback in order to start bug fixing.

The compiler is targeted solely at dfns and you will need access to a c-compiler on your machine. Also only 0-origin code is accepted for now.

There is a lot of additional information available on the Github wiki, so please visit Github, get hold of the Compiler and start playing!!

Follow

We have been spoilt with a large number of User presentations today, and although I have focused on writing more indepth about How to Save an Italian School with APL in a separate blog, I find it important to also give our blog readers a glimpse of  what we also saw and heard today.

By Vibeke Ulmann #Dyalog15

We kicked off this morning with a User presentation called ‘Mining the Depths of Excel – a Case Study in Objects and Arrays – by Richard Procter from APL Borealis in  Canada.

Richard also works for BCA Research in Canada, a Company who provides analysis of the global economy and publishes regular reports on their findings. BCA get their data from Bloomberg, Reuters and other sources f.ex. Excel spreadsheets,  to the tune of downloading 10ths of thousands of csv’s every night. And  – importantly – they keep track of everything in Excel. This further means that BCA is using a number of add-in’s to Excel.

Currently BCA is using ⎕WC to create graphs but they’re in the process of moving to using SharpPlot.  But even so, users want to make their own graphs and they can do that in Excel. As a consequence many spreadsheets have to be made available in an automated fashion and Microsoft is not really supporting the com model like they did in the past.

Richard proceeded to show us how using the openXML specification of spreadsheets in combination with the Syncfusion XLsIO library allows us to work more efficiently with APL and Excel files.

Before lunch, Paul Grosvenor, Managing Director of Optima Systems in the UK gave us a presentation called – APL Really Can make us Healthy(er)!

Those who have attended the Dyalog user meetings the last 2-3 years will have heard several presentations on COSMOS – the solution developed by Optima to allow people to explore data with no preconceptions, make data ‘understandable’, show points worth researching, and which takes a top down approach.

Paul Grosvenor on stage

The original inspiration for the design of COSMOS was 1) Neural Network and 2) Netmap – a solution which was used in the UK by the police  to track phonecalls between criminals.

The COSMOS solution is fully developed in APL – has a flash FrontEnd on MiServer with an HTML5 dashboard facility.

COSMOS works with BIG DATA as well as Complex data. As an example Paul told us that one client in the US is working with 8 TeraBytes of datasets, 250 Million Patient visits each with 2-3,000 cost items per visit, and 45 Million unique patients.

The latest interest from Pharmaceutical companies springs from the fact that  a drug development and trial cycle is long  – typical traditional clinical trials takes 10-12 years, and then drugs need to get FDA approval, which can add a further 2-3 years. The entire process often runs into 1 Billion USD or more. Patient lives, on the other hand, are short – and new drugs are needed faster. Currently, it is estimated that around 1,000 new drugs are awaiting clinical trials approval and COSMOS can help bringing down the waiting by assiting in identifying prospects for the trials which is currently a difficult and time consuming task.

COSMOS will be used in a small gen sequencing trial which starts next month – and the aim is to reduce the clinical trial time to 2-3 years.

If you would like to know more about COSMOS, why not read the article featured in the Summer issue of our Dyalogue newsletter here: http://www.truepr.co.uk/news/dl/41/040.asp?

After lunch, it was Ray Cannon’s turn to take to the stage with a presentation named Ants, AJAX and APL. Ray  is an independent APL developer and the origination of his presentation was a game programming competition, where you had to create a visualisation of a match between two species of ants.

Recently Ray has replaced the use of ⎕WC  with MiServer 3.0 which has vastly speeded up the user interface. The game is programmed in APL on a Linux platform on a Raspberry PI, but Linux has no frontend capability so using a TCP/IP link between the client (a web browser) and the Server (APL), he demonstrated the game in the new webfront-end. The basis for the game is that you work with 2 ant colonies, and the ant colony which brings back most food to the ant-hill wins the game. Using the new parallel features of Dyalog allowed us to see the game as 2 halves – running in parallel.

Moving swiftly on…….

Databinding is something we have been talking a lot about this year, and after a technical introduction by John Daintree, Alexey Miroshnikov founder and MD of InfoStroy Ltd, Russia took to the stage with a presentation named Data-bound Dashboard.

Databinding is a way of sharing data between GUI elements and the logical layer of an application. InfoStroy is the developer of GAMA – Russia’s leading Global Asset Management Solution for Pension Funds, Banks, Investment Companies and other types of clients who deals with billions worth of (any currency) Investments every day.

It has always been difficult to find a way for a ‘normal’ dashboard to present data from GAMA in a meaningful way, because most dashboards cannot access the data unless they reside in a SQL database and GAMA uses a custom build column store. With the new data-binding solutions offered by Dyalog APL in combinations with WPF the problem has been solved.

Alexey told us that the development of the dashboard took 1 man week, and InfoStroy do not have to design any ‘bespoke’ dashboards anymore – as GAMA can now create visualisation of all necessary data a client wants to view.  Voila!  And very elegant it was too!

Follow