Big Learning 2011
Big Learning: Algorithms, Systems, and Tools for Learning at Scale met on 16–17 December 2011 in the Montebajo theater at the NIPS workshop site in Sierra Nevada, Spain. The edition brought algorithms, parallel hardware and distributed software into the same discussion. It asked how learning changed when data or computation reached a scale that a traditional sequential approach could not comfortably handle.
The programme occupied two days, with morning and afternoon sessions on each. Invited talks and contributed talks shared the timetable with poster spotlights, poster discussions and practical tutorials. The accepted-paper list and invited-talk page give the full titles and names for their respective parts of the meeting.
A meeting across computational layers
The 2011 workshop description at NeurIPS connected cheap storage, data networks and structured models with problems that had outgrown sequential systems. It also identified a wide span of applications, from bioinformatics and astronomy to recommendation, computer vision and web search. This breadth mattered to the workshop’s central question. Scale could arise because there were many observations, because a model involved complicated dependencies, or because the calculation needed to run repeatedly.
The meeting’s aims therefore included more than new algorithms. Case studies, demonstrations, benchmarks and implementation lessons were part of its remit. The workshop also sought discussion between system builders, learning researchers and end users. That combination made room for questions about the form of the data, the available computing platform and the engineering effort needed to use a method. The introduction to big learning develops those distinctions in a general setting.
The rhythm of the two days
The first morning began with opening remarks and an invited talk on GPU metaprogramming. Poster spotlights and a poster session followed before a contributed presentation on a common GPU array and an invited talk on a reconfigurable dataflow processor for vision. The afternoon combined application subjects with tools, contributed talks and further poster discussions. A Vowpal Wabbit tutorial occupied a separate early-afternoon slot. The programme’s repeated returns to posters gave the day a different shape from a continuous sequence of long talks.
The second morning began with an invited subject at the intersection of statistical applications and data systems. Spotlights and posters were followed by a contributed talk on Hamiltonian Monte Carlo, an invited subject on real-time sketches and a contributed optimisation presentation. The afternoon included a GraphLab 2.0 tutorial, distributed learning, cluster computation and matrix factorisation. Poster sessions appeared between several presentations, and closing remarks ended the programme.
Spotlights and contributed talks served distinct roles in that arrangement. A short spotlight introduced a poster subject to the room, while a longer contributed slot allowed a paper to occupy its own place in the timetable. Poster sessions then provided time around the displayed work. The author-guidelines page explains the extended-abstract format that accompanied the workshops’ talk and poster presentations. The meeting also included a best-talk award decided by a vote.
Hardware beside statistical method
The first day’s paper titles included GPU arrays, Theano, probabilistic inference and hashing. The second day’s titles included Monte Carlo methods, stochastic optimisation, matrix factorisation and distributed learning tools. Read together, the lists show how the programme placed computational representation beside statistical technique. An array format belongs to the software layer; an inference method belongs to the statistical layer. Both can become central when a learning calculation has to run across parallel resources.
Evaluation also appeared in the programme through titles concerning cross-validation and the bootstrap. These subjects shift attention from fitting a model to assessing it. Wikipedia’s explanation of bootstrapping describes resampling as a way to estimate an estimator’s distribution. Repeated statistical work raises its own computational questions at scale. The paper titles identify that area of interest without establishing any particular paper’s findings or comparing its method with another.
A contemporary account of the range
In an AMPLab trip report, Matei Zaharia described a workshop in Spain with about ten invited talks and ten paper presentations. The account ranged from FPGAs and GPU programming to commodity cloud clusters, and discussed GraphLab, Spark and Scala. It also mentioned work on stochastic optimisation and the bootstrap. This account is useful for the breadth of subjects it reported: specialised hardware, programming tools and statistical ideas all formed part of the same meeting.
The location can be understood within the conference’s wider schedule. Wikipedia’s conference history places NIPS 2011 in Granada, Spain; the Big Learning workshop met at the Sierra Nevada workshop site. These are different levels of the event’s geography. The room and workshop dates identify this edition, while the conference history places it in the annual meeting.
Connections with the later editions
The 2012 edition retained the broad relationship among algorithms, systems, applications and tools, but compressed its programme into one day at Lake Tahoe. The 2013 edition then made database management an explicit partner to machine learning. Looking across the editions makes the change of emphasis easier to see: the first programme offered a wide survey of hardware, methods and frameworks, while the later programme concentrated attention on data systems and parallel learning.
The 2011 material also connects naturally with the topic pages. Hardware for learning explains accelerator and multicore vocabulary; graph-parallel computation considers dependency structure; and stochastic optimisation introduces repeated parameter updates. These explanations give readers ways to compare subjects across the programme without treating a talk title as a complete account of the research behind it.