Melbourne Satellite
A SIGIR-AP 2024 satellite event will be held at RMIT city campus this year, thanks to the generous support of the ADMS Centre. This is a unique opportunity for the Australian IR community to gather, share research, and connect with the larger Asia-Pacific IR community in real time.
Event Details
The satellite event will run in parallel with the main conference in Tokyo from December 10–12. Our format includes:
Keynote: Peter Bailey (Canva) will give a keynote talk about “Failing Faster by Climbing the Ladder of Experimentation.”
In-person presentations: Accepted papers will be presented locally by attending authors.
Hybrid live stream: We’ll join the main conference broadcast, allowing seamless participation from Melbourne.
Social mixer: We’ll also have a social event in Melbourne at the end of the first day, supported by The University of Queensland.
Whether you’re presenting or attending, this event offers a fantastic chance for networking and collaboration among Australian IR researchers.
Keynote
Talk Title: Failing Faster by Climbing the Ladder of Experimentation
Speaker: Peter Bailey
Abstract: Improving complex search systems requires more than just achieving high ELO scores in Chatbot Arena or increasing DCG scores in your favourite test collections. In reality, around 80% of ideas do not enhance success metrics for mature stage products. While A/B experimentation is the standard for validating customer-facing products, these are also relatively expensive and slow. At Canva, we have invested in a ladder of experimentation, aiming to identify likely failures more rapidly and efficiently. This presentation outlines our ladder and explores two recently developed steps: an offline evaluation method for private search re-finding tasks that does not involve user queries or results, and our transition from Team Draft Interleaving to Competitive Pairs Interleaving. Additionally, we will discuss some challenges related to interleaving evaluation for whole page experiences and with generative content involved.
Bio: Peter is an expert in the field of information retrieval, and currently leads machine learning for Discovery at Canva. He has worked in a range of organisations, from academia to industrial research labs, from startups and consulting, to more than a decade at Microsoft. He is a co-author of numerous scientific papers and co-inventor of a number of patents. He is the co-creator of several widely used datasets, including WT10g, CERC, UQV100, and CRD3. While at Microsoft, he lead the Contextual Relevance team when it introduced personalised search ranking to the Bing web search engine in 2010. Elements of that work were recognised in the ACM SIGIR 2022 Test of Time paper award for "Modeling the impact of short- and long-term behavior on search personalization". In addition to his applied scientist and manager responsibilities, he was a member of Microsoft's Responsible AI AETHER Committee's Fairness and Inclusiveness working group from 2018 through 2021. He is a Senior Member of the Association for Computing Machinery, and contributes regularly to program committees for conferences in the information retrieval community.
Register Now
To participate, you’ll need to register as an online attendee for the main event:
https://www.sigir-ap.org/sigir-ap-2024/registration/
Students: We’ve arranged for a discounted early bird rate available until November 15. Please immediately contact Oleg Zendel oleg.zendel@rmit.edu.au (Melbourne Satellite Chair) to get the special link for the discounted early bird rate registration.
All others: Register as soon as possible to secure the early bird rate, as prices will increase after November 15.
Confirm Your Spot for the Satellite Event
Please also complete a short registration for the satellite event itself. There’s no additional fee, and catering will be provided. We need participant numbers as soon as possible to finalize catering arrangements.
https://forms.office.com/r/CuGaFmQedD
Contact
Melbourne Satellite Chair
Oleg Zendel oleg.zendel@rmit.edu.au