Last September I had the pleasure to attend to the European Conference on Machine Learning (ECML), which has been held simultaneously with the Principles and Practice of Knowledge Discovery in Databases Conference (PKDD) for the last 13 years.
This year the conference was held in Skopje, where they received us with an excellent and priceless Balkan hospitality. The capital of Macedonia is a lively and nice city, surrounded by hills and blessed by a blue sky.
Although being a European conference, many speakers and assistants come from all around the world. I had the chance to meet colleagues from France, UK, Russia, US and to share the acquired knowledge and spare time with them.
ECML is a large conference (400-500 assistants), with 5 tracks taking place simultaneously; the fact that some of them only have 30-40 people in the audience decreases slightly the level of some of the talks. Therefore, having the opportunity to interact with people who have attended different tracks is really useful in order to share impressions on the most interesting talks.
John Quackenbush, working at Harvard T.H. Chan School of Public Health and contributor to the Human Genome Project, presented his work on the role that gene expression plays in diseases.
If you really want to be moved by this great, engaging speaker and outstanding physicist please take a look to this highly inspiring 1-hour talk, where you will travel from the wonderful world of a single protein to the intricate universe of complex systems and its application to the understanding of how a group of genes can trigger and influence disease.
Cordelia Schmid, an expert in computer vision working at INRIA, France, presented “Automatic Understanding of the Visual World”, a plenary talk where she showed the improvements her group has reached in moving objects segmentation by using a convolutional recurrent memory module to catch the evolution of objects over time. She visually impressed the audience by showing the SURREAL dataset (synthetic hUmans forR REAL tasks ) they’ve created to improve video image segmentation.
In question time, she claimed the biggest challenge neural networks face in the future is the problem of catastrophic forgetting or tendency of an artificial neural network to forget previously learned information upon learning new information.
Frank Hutter presented another outstanding plenary talk entitled “Towards end-to-end Learning & Optimization”, where he summarized his work, remarking the importance of understanding hyper-parameters influence. He first introduced his work on AutoWeka, to continue with auto-sklearn and bayesian optimization extensions for autoML. You can start trying it right now because, as he emphasized, “everything is forked in github”. He finished the talk presenting auto-Nets: “Towards Automatically-Tuned Neural Networks” (here you can take a look to the highly detailed slides of the plenary talk) and how to speed-up hyper-parameter optimization for several network architectures and optimizers. You can also fork it on github! In fact, there was a full-day tutorial and workshop which gathered a large audience devoted to these automatic machine learning methods; more specifically to meta-learning, algorithm selection, and algorithm configuration. If you are interested take a look to the slides of the tutorial and the proceedings of the workshop.
Alex Graves, from DeepMind, presented a technical plenary talk on recurrent neural networks. In particular, on the challenges arising when dealing with scaling RNN’s in space and time, it is to say, as the amount of memory grows. He introduced his paper on SAM’s (sparse access memory) to deal with such problems, and an adaptive computation time algorithm and its application to the Wikipedia dataset. In particular, he showed how this algorithm could distinguish structure from noise when other metrics such as prediction loss were not as successful. He finished talking about synthetic gradients and other algorithms such as curriculum learning to accelerate learning.
The Best Student Paper Award was granted to “Arbitrated Ensemble for Time Series Forecasting”, an adaptive ensemble method to deal with the different dynamics of the time series data, by decomposing the prediction in a base predictor and several meta-predictors. If you’re interested, there’s a R package called tsensembler ready for use.
There was a presentation which aroused great interest in the audience for the novelty and simplicity of the idea. FCNN, Fourier Convolutional Neural Networks, conducted the entire training of a CNN within the Fourier domain, resulting in advantages in speed without loss of effectiveness. It was implemented using a Nvidia K40c GPU of 12GB RAM + Keras.
But without a doubt, one of my favorite presentations of the entire conference was part of the session Learning and Optimization by Alon Zweig and was entitled “Group Online Adaptive Learning (GOAL)” Here, the author presented a method where multiple learners, each of them in their own changing environment, share information in order to accelerate the global learning. This method is valid for plenty of applications: object recognition, finance and recommender systems, among them. The authors noted that sharing all data does not scale; therefore their model is restricted to share experts’ data, while experts are added or retired following the leading history, and other agents can use experts who have previously remained frozen. The algorithm is named SOAL, and its performance was shown in a visual navigation task where a robot learns to navigate based on 6 outdoor video scenes. SOAL proofed to remarkably improve learning when knowledge from other robots in related scenes was available.
In the field of networks, we could see an interesting talk on “Activity-Driven Influence Maximization in Social Networks” by Rohit Kumar, devoted to find a top set of influential locations. For that purpose, instead of selecting the users with the largest set of friends or the set of locations visited by the most users, they identify influence propagation patterns in a scalable way.
Other interesting Tutorials and Workshops
Interactive Adaptive Learning: You can find an interesting introductory paper to AI (“Challenges of Reliable, Realistic and Comparable Active Learning Evaluation”) and the rest of the papers presented in the workshop, where we could find methods to optimize the whole learning process, from interaction with human supervisors to adaptive or transfer learning techniques.
Scaling-Up Reinforcement Learning: This workshop brought together researchers working on different methods to scale-up reinforcement learning.
MIDAS “MIning DAta for financial applicationS”: The workshop proceedings show some of the challenges of the application of machine learning to financial data.
Learning with Imbalanced Domains: Theory and Applications: The workshop proceedings deal with the problem of imbalanced data, which happens very often in data science.
Looking forward you enjoyed the videos and learning material as I did, see you all in Dublin, 2018!!