ICML 2021 workshop · 3rd edition

Reinforcement Learning for Real Life

July 23, 2021Virtual workshop at ICML 2021
73 papers6 panelsVirtual
Overview

About the workshop

Reinforcement learning (RL) is a general learning, predicting, and decision making paradigm and applies broadly in many disciplines, including science, engineering and humanities. RL has seen prominent successes in many problems, such as Atari games, AlphaGo, robotics, recommender systems, and AutoML. However, applying RL in the real world remains challenging, and a natural question is:

Why isn’t RL used even more often and how can we improve this?

Original website ↗ICML 2021 workshop page (videos) ↗Slack workspace ↗Machine Learning Journal special issue (2nd) ↗
Program

Schedule

  • 9:00-11:00Poster Session
  • 11:00-12:00RL Foundation PanelVideo ↗
  • 12:00-13:00RL Explainability & Interpretability PanelVideo ↗
  • 13:00-14:00RL + Robotics PanelVideo ↗
  • 14:00-18:00Break
  • 18:00-19:00RL + Recommender Systems PanelVideo ↗
  • 19:00-20:00SpotlightVideo ↗
  • 20:00-21:00RL Research-to-RealLife Gap PanelVideo ↗
  • 21:00-22:00Break
  • 22:00-23:00RL + OR PanelVideo ↗
  • 23:00-1:00Poster Session
Discussion

Panels

We will request invited panelists/moderators to share their expertise w.r.t. the real life aspects of RL by pre-recording videos. The moderators will host live panel discussions, and there will be polls for audience to submit questions.

RL Foundation Panel

Co-Chairs: Csaba Szepesvari, Lihong Li and Yuxi Li
Video ↗ Audience questions ↗
Thomas DietterichOregon State Universityvideo ↗ more ↗
Warren PowellPrinceton / Optimal Dynamicsvideo ↗ more ↗ more ↗

RL Research-to-RealLife Gap Panel

Co-Chairs/Moderators: Matthew E. Taylor and Kathryn Hume
Video ↗ Audience questions ↗
Hasham BurhaniRBC Capital Markets
Craig BuhrMathWorks
Yang YuPolixir.ai / Nanjing University
Kathryn HumeCo-ChairBorealis AI

RL + Recommender Systems Panel

Co-Chairs/Moderators: Minmin Chen and Lihong Li
Video ↗ Audience questions ↗
Ed ChiGoogle

RL + Robotics Panel

Chair/Moderator: Rupam Mahmood
Video ↗ Audience questions ↗
Angela SchoelligUniversity of Torontovideo ↗
Rose YuUC San Diegovideo ↗

RL Explainability & Interpretability Panel

Co-Chairs/Moderators: Omer Gottesman and Niranjani Prasad
Video ↗ Audience questions ↗
Alan FernOregon State Universityvideo ↗
Zachary C. LiptonCarnegie Mellon Universityvideo ↗

RL + Operations Research Panel

Co-Chairs: Zhiwei (Tony) Qin and Zongqing Lu (Moderator)
Video ↗ Audience questions ↗
Jim DaiCornell / CUHK-Shenzhenvideo ↗
Fei FangCarnegie Mellon Universityvideo ↗
Shie MannorTechnion & NVIDIA Researchvideo ↗
Yuandong TianFacebook AI Researchvideo ↗
Contributed

Accepted papers

The poster specs for ICML 2021 main conference: in .png format with a max size of 5120 x 2880, with an optional thumbnail image of size 320 x 256.

  1. DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning
    Xianyuan Zhan (JD Intelligent Cities Research); Haoran Xu (Xidian University); Yue Zhang (JD Intelligent Cities Research); Xiangyu Zhu (JD Intelligent Cities Research); Honglei Yin (JD Intelligent Cities Research)
    Paper
  2. Neural Rate Control for Video Encoding using Imitation Learning
    Hongzi Mao (MIT CSAIL); Chenjie Gu (Deepmind); Miaosen Wang (Google); Angie Chen (Google); Nevena Lazic (DeepMind); Nir Levine (Deepmind); Derek Pang (Google); Rene Claus (Google); Marisabel Hechtman‎ (Google); Ching-Han Chiang (Google Inc.); Cheng Chen (Google Inc.); Jingning Han (Google Inc.)
    Paper
  3. Reinforcement Learning for (Mixed) Integer Programming: Smart Feasibility Pump
    Mengxin Wang (University of California, Berkeley); Meng Qi (University of California, Berkeley); Zuo-Jun Shen (University of California, Berkeley)
    Paper
  4. Continuous Doubly Constrained Batch Reinforcement Learning
    Rasool Fakoor (Amazon); Jonas Mueller (AWS); Kavosh Asadi (Brown University); Pratik Chaudhari (University of Pennsylvania); Alex J Smola (Amazon)
    Paper
  5. Contingency-Aware Influence Maximization: A Reinforcement Learning Approach
    Haipeng Chen (Harvard University); Wei Qiu (Nanyang Technological University); Han-Ching Ou (Harvard University); Bo An (Nanyang Technological University); Milind Tambe (Harvard University)
    Paper
  6. On the Difficulty of Generalizing Reinforcement Learning Framework for Combinatorial Optimization
    Mostafa Pashazadeh (University of Victoria); Kui Wu (University of Victoria)
    Paper
  7. Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model
    Haruka Kiyohara (Tokyo Institute of Technology); Yuta Saito (Hanjuku-kaso, Co., Ltd.); Tatsuya Matsuhiro (Yahoo Japan Corporation); Yusuke Narita (Yale University); Nobuyuki Shimizu (Yahoo Japan Corporation); Yasuo Yamamoto (Yahoo! Japan)
    Paper
  8. OffWorld Gym: Open-Access Physical Robotics Environment for Real-World Reinforcement Learning Benchmark and Research
    Ashish Kumar (OffWorld Inc); Toby Buckley (Offworld Inc.); John Lanier (OffWorld, Inc.); Qiaozhi Wang (OffWorld, Inc.); Alicia Kavelaars (OffWorld Inc.); Ilya Kuzovkin (Offworld Inc.)
    Paper
  9. Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations
    Yuping Luo (Princeton University); Tengyu Ma (Stanford)
  10. Automatic Risk Adaptation in Distributional Reinforcement Learning
    Frederik Schubert (Leibniz University Hannover); Theresa Eimer (Leibniz University Hannover); Bodo Rosenhahn (Leibniz University Hannover); Marius Lindauer (Leibniz University Hannover)
    PaperAppendix
  11. Coordinate-wise Control Variates for Deep Policy Gradients
    Yuanyi Zhong (University of Illinois at Urbana-Champaign); Yuan Zhou (UIUC); Jian Peng (University of Illinois at Urbana-Champaign)
    Paper
  12. Disentangled Attention as Intrinsic Regularization for Bimanual Multi-Object Manipulation
    Minghao Zhang (Tsinghua University); Pingcheng Jian (Tsinghua University); Yi Wu (Tsinghua University); Huazhe Xu (UC Berkeley); Xiaolong Wang (UCSD)
    Paper
  13. Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
    Ruihan Yang (UC San Diego); Minghao Zhang (Tsinghua University); Nicklas A Hansen (UC San Diego); Huazhe Xu (UC Berkeley); Xiaolong Wang (UCSD)
    Paper
  14. Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systems
    Daniele Gammelli (Technical University of Denmark (DTU)); Kaidi Yang (Stanford University); James Harrison (Stanford University); Filipe Rodrigues (Technical University of Denmark (DTU)); Francisco Pereira (DTU); Marco Pavone (Stanford University)
    PaperVideo
  15. Reward-Free Attacks in Multi-Agent Reinforcement Learning
    Ted Fujimoto (Pacific Northwest National Laboratory); Timothy Doster (Pacific Northwest National Laboratory); Adam Attarian (PNNL); Jill M Brandenberger (PNNL); Nathan Hodas (Pacific Northwest National Lab)
    Paper
  16. Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research
    Juan Jose Garau Luis (MIT); Edward Crawley (MIT); Bruce Cameron (MIT)
    Paper
  17. Corruption Robust Offline Reinforcement Learning
    Xuezhou Zhang (UW-Madison); Yiding Chen (University of Wisconsin-Madison); Xiaojin Zhu (University of Wisconsin-Madison); Wen Sun (Cornell University)
  18. Deep Reinforcement Learning for 3D Furniture Layout in Indoor Graphic Scenes
    XINHAN DI (Deepearthgo); Pengqian Yu (Sea AI Lab)
    Paper
  19. Learning to Represent State with Perceptual Schemata
    Wilka Carvalho (University of Michigan); Murray P Shanahan (DeepMind Technologies Ltd)
    Paper
  20. Continual Meta Policy Search for Sequential Multi-Task Learning
    Glen Berseth (University of California Berkeley); Zhiwei Zhang (Uc Berkeley)
    Paper
  21. Reinforcement Learning as One Big Sequence Modeling Problem
    Michael Janner (UC Berkeley); Qiyang Li (University of California, Berkeley); Sergey Levine (UC Berkeley)
    Paper
  22. Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap
    Gokul Swamy (Carnegie Mellon University); Sanjiban Choudhury (Aurora Innovation); J. Andrew Bagnell (Aurora Innovation); Steven Wu (Carnegie Mellon University)
    PaperVideo
  23. Learning Space Partitions for Path Planning
    Kevin Yang (UC Berkeley); Tianjun Zhang (UC Berkeley); Chris Cummins (Facebook AI Research); Brandon Cui (Facebook AI Research); Benoit Steiner (Facebook AI Research); linnan wang (brown); Joey Gonzalez (Berkeley); Dan Klein (University of California, Berkeley); Yuandong Tian (Facebook)
    PaperAppendix
  24. ReLMM: Practical RL for Learning Mobile Manipulation Skills Using Only Onboard Sensors
    Charles J Sun (UC Berkeley); Jedrzej Orbik (UC Berkeley); Coline Devin (UC Berkeley); Abhishek Gupta (UC Berkeley); Glen Berseth (University of California Berkeley); Sergey Levine (UC Berkeley)
    Paper
  25. Representation Learning for Out-of-distribution Generalization in Downstream Tasks
    Frederik Träuble (MPI for Intelligent Systems); Andrea Dittadi (Technical University of Denmark); Manuel Wüthrich (MPI for Intelligent Systems); Felix Widmaier (MPI for Intelligent Systems, Tübingen); Peter Gehler (Amazon); Ole Winther (DTU and KU); Francesco Locatello (Amazon); Olivier Bachem (Google Brain); Bernhard Schölkopf (MPI for Intelligent Systems, Tübingen); Stefan Bauer (Max Planck institute)
    Paper
  26. Symbolic Relational Deep Reinforcement Learning based on Graph Neural Networks
    Jaromír Janisch (Czech Technical University in Prague); Tomas Pevny (Czech Technical University in Prague); Viliam Lisy (Czech Technical University)
    Paper
  27. Hierarchical Multiple-Instance Data Classification with Costly Features
    Jaromír Janisch (Czech Technical University in Prague); Tomas Pevny (Czech Technical University in Prague); Viliam Lisy (Czech Technical University)
    Paper
  28. Multi-agent Deep Covering Option Discovery
    Jiayu Chen (Purdue University); Marina W Haliem (Purdue University); Tian Lan (The George Washington University); Vaneet Aggarwal (Purdue University)
    Paper
  29. Efficient Exploration by HyperDQN in Deep Reinforcement Learning
    Ziniu Li (The Chinese University of Hong Kong, Shenzhen); Yingru Li (The Chinese University of Hong Kong, Shenzhen); Hao Liang (The Chinese University of Hongkong, Shenzhen); Tong Zhang (The Hong Kong University of Science and Technology)
    Paper
  30. Revisiting Design Choices in Offline Model Based Reinforcement Learning
    Cong Lu (University of Oxford); Philip J Ball (University of Oxford); Jack Parker-Holder (University of Oxford); Michael A. Osborne (University of Oxford); Stephen Roberts (Oxford)
    Paper
  31. De novo drug design using reinforcement learning with graph-based deep generative models
    Sara Romeo Atance (Chalmers University of Technology); Juan Viguera Diez (Chalmers University of Technology); Ola Engkvist (AstraZeneca AB); Simon Olsson (Chalmers University of Technology); Rocío Mercado (AstraZeneca)
    PaperAppendix
  32. Optimization of high precision manufacturing by Monte Carlo Tree Search
    Dorina Weichert (Fraunhofer IAIS); Felix Horchler (Bonn University); Alexander Kister (Fraunhofer IAIS); Marcus Trost (Fraunhofer IOF); Johannes Hartung (Fraunhofer IOF); Stefan Risse (Fraunhofer IOF)
    Paper
  33. Designing Online Advertisements via Bandit and Reinforcement Learning
    Richard Liu (Yale University); Yusuke Narita (Yale University); Kohei Yata (Yale University); Shota Yasui (Cyberagent)
    PaperAppendix
  34. Semantic Tracklets: An Object-Centric Representation for Visual Multi-Agent Reinforcement Learning
    Iou-Jen Liu (University of Illinois at Urbana-Champaign); Zhongzheng Ren (UIUC); Raymond A Yeh (UIUC); Alexander Schwing (UIUC)
    Paper
  35. Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings
    Shengpu Tang (University of Michigan); Jenna Wiens (University of Michigan)
    Paper
  36. Offline Reinforcement Learning as Anti-Exploration
    Shideh Rezaeifar (Geneva University); Robert Dadashi (Google); Nino Vieillard (Google Research); Léonard Hussenot (Google Research, Brain Team); Olivier Bachem (Google Brain); Olivier Pietquin (Google Research - Brain Team); Matthieu Geist (Google Brain)
    Paper
  37. What Can I Do Here? Learning New Skills by Imagining Visual Affordances
    Khazatsky Alexander (UC Berkeley); Ashvin V Nair (UC Berkeley)
    PaperVideo
  38. IV-RL: Leveraging Target Uncertainty Estimation for Sample Efficiency in Deep Reinforcement Learning
    Vincent Mai (Mila, Université de Montréal); Kaustubh Mani (Mila, Université de Montréal); Liam Paull (Université de Montréal)
    Paper
  39. Learning a Markov Model for Evaluating Soccer Decision Making
    Maaike Van Roy (KULeuven); Pieter Robberechts (KU Leuven); Wen-Chi Yang (KU Leuven); Luc De Raedt (KU Leuven); Jesse Davis (KU Leuven)
    Paper
  40. Topological Experience Replay for Fast Q-Learning
    Zhang-Wei Hong (Massachusetts Institute of Technology); Tao Chen (MIT); Yen-Chen Lin (MIT); Joni Pajarinen (Aalto University); Pulkit Agrawal (MIT)
    PaperVideo
  41. AppBuddy: Learning to Accomplish Tasks in Mobile Apps via Reinforcement Learning
    Maayan Shvo (University of Toronto and Vector Institute); Zhiming Hu (Samsung AI Center, Toronto); Rodrigo A Toro Icarte (University of Toronto and Vector Institute); Iqbal Mohomed (Samsung Research America); Allan D Jepson (Samsung Toronto AIC); Sheila A. McIlraith (University of Toronto and Vector Institute)
    Paper
  42. Reward Shaping for User Satisfaction in a REINFORCE Recommender
    Konstantina Christakopoulou (Google); Can Xu (Google); Sai Zhang (Google); Sriraj Badam (Google); Trevor Potter (Google); Daniel Li (Google); Hao Wan (Google); Xinyang Yi (Google); Ya Le (Google); Chris Berg (Google); Eric Bencomo Dixon (Google); Ed H. Chi (Google); Minmin Chen (Google)
    Paper
  43. Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Tasks
    Yijie Guo (University of Michigan); Qiucheng Wu (University of Michigan); Honglak Lee (LG AI Research / University of Michigan)
    PaperAppendix
  44. Mind the Gap: Safely Bridging Offline and Online Reinforcement Learning
    Wanqiao Xu (University of Michigan); Kan Xu (University of Pennsylvania); Hamsa Bastani (Wharton); Osbert Bastani (University of Pennsylvania)
  45. Deploying a Machine Learning System for COVID-19 Testing in Greece
    Hamsa Bastani (Wharton); Kimon Drakopoulos (USC, Data Sciences and Operations); Vishal Gupta ()
    Paper
  46. The Reflective Explorer: Online Meta-Exploration from Offline Data in Visual Tasks with Sparse Rewards
    Rafael Rafailov (Stanford University); Varun Kumar (Intel AI Lab); Tianhe Yu (Stanford University); Avi Singh (UC Berkeley); Mariano Phielipp (Intel AI Lab); Chelsea Finn (Stanford)
  47. Improving Human Decision-Making with Machine Learning
    Hamsa Bastani (Wharton); Osbert Bastani (University of Pennsylvania); Wichinpong Sinchaisri (Berkeley Haas)
    Paper
  48. Avoiding Overfitting to the Importance Weights in Offline Policy Optimization
    Yao Liu (Stanford University); Emma Brunskill (Stanford University)
    Paper
  49. Towards Reinforcement Learning for Pivot-based Neural Machine Translation with Non-autoregressive Transformer
    Evgeniia Tokarchuk (University of Amsterdam); Jan Rosendahl (RWTH Aachen University); Weiyue Wang (RWTH Aachen University); Pavel Petrushkov (eBay); Tomer Lancewicki (eBay Research); Shahram Khadivi (eBay, Inc.); Hermann Ney ( RWTH Aachen University)
    Paper
  50. Data-Pooling Reinforcement Learning for Personalized Healthcare Intervention
    Xinyun Chen (Chinese University of Hong Kong, Shenzhen); Pengyi Shi (Purdue University); Xiuwen Wang (Chinese University of Hong Kong, Shenzhen)
    PaperVideo
  51. Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage
    Jonathan Chang (Cornell University); Masatoshi Uehara (Cornell University); Dhruv Sreenivas (Cornell University); Rahul Kidambi (Amazon Search & AI); Wen Sun (Cornell University)
  52. MobILE: Model-Based Imitation Learning From Observation Alone
    Rahul Kidambi (Amazon Search & AI); Jonathan Chang (Cornell University); Wen Sun (Cornell University)
  53. Objective Robustness in Deep Reinforcement Learning
    Jack Koch (Unaffiliated); Lauro Langosco di Langosco (ETH); Jacob Pfau (University of California San Francisco); James Le (Unaffiliated); Lee D Sharkey (ETHZ)
    Paper
  54. Is Bang-Bang Control All You Need?
    Tim N Seyde (MIT); Igor Gilitschenski (Massachusetts Institute of Technology); Wilko Schwarting (Massachusetts Institute of Technology); Bartolomeo Stellato (Princeton University); Martin Riedmiller (DeepMind); Markus Wulfmeier (DeepMind); Daniela Rus (MIT CSAIL)
    Paper
  55. Off-Policy Evaluation with General Logging Policies
    Kyohei Okumura (Northwestern University); Yusuke Narita (Yale University); Kohei Yata (Yale University); Akihiro Shimizu (Mercari)
    Paper
  56. Safe Deep Reinforcement Learning for Multi-Agent Systems with Continuous Action Spaces
    Ziyad Sheebaelhamd (ETH Zurich); Konstantinos Zisis (ETH Zurich); Athina Nisioti (ETH Zurich); Dimitris Gkouletsos (ETH Zurich); Dario Pavllo (ETH Zurich); Jonas Kohler (ETHZ)
    Paper
  57. Reinforcement Learning with Logical Action-Aware Features for Polymer Discovery
    Sarathkrishna Swaminathan (IBM Research); Dmitry Zubarev (IBM Research-Almaden); Subhajit Chaudhury (IBM Research AI); Asim Munawar (IBM Research)
    Paper
  58. Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning
    Haoran Xu (Xidian University); Xianyuan Zhan (JD Intelligent Cities Research); Xiangyu Zhu (JD Intelligent Cities Research)
    Paper
  59. ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control
    Xingshuai Huang (McGill University); di wu (McGill); Benoit Boulet (Benoit Boulet, McGill University)
    Paper
  60. Robust Risk-Sensitive Reinforcement Learning Agents for Trading Markets
    Yue Gao (University of Alberta); Pablo Hernandez-Leal (Borealis AI); Kry Yik Chau Lui ()
    Paper
  61. Reinforcement Learning for Power System Control: Using Adversarial Training to Improve Robustness
    Alexander Pan (Caltech); Yongkyun Lee (Caltech); Yuanyuan Shi (Caltech); Huan Zhang (UCLA)
    Paper
  62. Understanding the Generalization Gap in Visual Reinforcement Learning
    Anurag Ajay (MIT); Ge Yang (University of Chicago); Ofir Nachum (Google); Pulkit Agrawal (MIT)
    Paper
  63. Optimizing Dynamic Treatment Regimes via Volatile Contextual Gaussian Process Bandits
    Ahmet Alparslan Celik (Bilkent University); Cem Tekin (Bilkent University)
    Paper
  64. Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention
    Abhishek Gupta (UC Berkeley); Justin Yu (Berkeley); Zihao Zhao (UC Berkeley); Vikash Kumar (Univ. of Washington); Aaron Rovinsky (UC Berkeley); Kelvin Xu (University of California, Berkeley); Thomas Devlin (UC Berkeley); Sergey Levine (University of California, Berkeley)
    Paper
  65. Attend2Pack: Bin Packing through Deep Reinforcement Learning with Attention
    Jingwei Zhang (University of Freiburg); Bin Zi (Dorabot Inc.); Xiaoyu Ge (Australian National University)
    Paper
  66. Designing Interpretable Approximations to Deep Reinforcement Learning
    Nathan J Dahlin (University of Southern California); Krishna C Kalagarla (University of Southern California); Nikhil Naik (University of Southern California); Rahul Jain (University of Southern California); Pierluigi Nuzzo (University of Southern California)
    Paper
  67. Decision Transformer: Reinforcement Learning via Sequence Modeling
    Lili Chen (UC Berkeley); Kevin Lu (UC Berkeley); Aravind Rajeswaran (University of Washington); Kimin Lee (UC Berkeley); Aditya Grover (Facebook AI Research); Michael Laskin (UC Berkeley); Pieter Abbeel (UC Berkeley); Aravind Srinivas (UC Berkeley); Igor Mordatch (Google)
    Paper
  68. Multi-Task Offline Reinforcement Learning with Conservative Data Sharing
    Tianhe Yu (Stanford University); Aviral Kumar (UC Berkeley); Yevgen Chebotar (Google); Karol Hausman (Google Brain); Sergey Levine (UC Berkeley); Chelsea Finn (Stanford)
    Paper
  69. Value-Based Deep Reinforcement Learning Requires Explicit Regularization
    Aviral Kumar (UC Berkeley); Rishabh Agarwal (Google Research, Brain Team); Aaron Courville (University of Montreal); Tengyu Ma (Stanford); George Tucker (Google Brain); Sergey Levine (UC Berkeley)
    Paper
  70. A Policy Efficient Reduction Approach to Convex Constrained Deep Reinforcement Learning
    Tianchi Cai (Ant Group); Wenpeng Zhang (Ant Group); Lihong Gu (Ant Group); Xiaodong Zeng (Ant Services Group); Jinjie Gu (Ant Group)
    Paper
  71. Hierarchical Imitation Learning with Contextual Bandits for DynamicTreatment Regimes
    Lu Wang (East China Normal University); Wenchao Yu (UCLA); Wei Cheng (NEC); Bo Zong (NEC); Haifeng Chen (NEC)
    Paper
  72. Reinforcement Learning Agent Training with Goals for Real World Tasks
    Xuan Zhao (Microsoft); Marcos Campos (microsoft)
    Paper
  73. RRL: Resnet as representation for Reinforcement Learning
    Rutav M Shah (Indian Institute of Technology, Kharagpur); Vikash Kumar (Univ. of Washington)
    Paper
Archive

Call for papers

Reinforcement learning (RL) is a general learning, predicting, and decision making paradigm and applies broadly in many disciplines, including science, engineering and humanities. RL has seen prominent successes in many problems, such as Atari games, AlphaGo, robotics, recommender systems, and AutoML. However, applying RL in the real world remains challenging, and a natural question is:

Why isn’t RL used even more often and how can we improve this?

The main goals of the workshop are to: (1) identify key research problems that are critical for the success of real-world applications; (2) report progress on addressing these critical issues; and (3) have practitioners share their success stories of applying RL to real-world problems, and the insights gained from such applications.

We invite paper submissions successfully applying RL algorithms to real-life problems and/or addressing practically relevant RL issues. Our topics of interest are general, including (but not limited to):

Practical RL algorithms, which cover all algorithmic challenges of RL, especially those that directly address challenges faced by real-world applications;

Practical issues: generalization, sample/time/space efficiency, exploration vs. exploitation, reward specification and shaping, scalability, model-based learning (model validation and model error estimation), prior knowledge, safety, accountability, interpretability, reproducibility, hyper-parameter tuning;

Applications: advertizing, autonomous driving, business processes, chemical synthesis, conversational AI, drawing, drug design, education, energy, finance, healthcare, industrial control, music, recommender systems, robotics, transportation, or other problems in science, engineering and humanities.

We invite unpublished submissions up to 8 pages excluding references and appendix, in PDF format using the ICML 2021 template and style guidelines. Here is a customized style file for the workshop. (In the .tex file, use "\usepackage{icml2021}" for the submission, and use "\usepackage[accepted]{icml2021}" for the final version if accepted.) We welcome position papers and are open to papers currently under review at other venues. We will follow a double-blind review. We also welcome recently published work, which may keep its original format (for submission and for final version). For the submissions of ICML rejections, please attach the reviews. All accepted papers will be presented as posters, and will be made available on the workshop website. Accepted papers are non-archival, i.e., there will be no proceedings for this workshop.

The submission website is: https://cmt3.research.microsoft.com/RL4RealLife2021/

Important dates
Deadline
June 12, 2021
People

Organizers

Co-chairs
Lihong LiAmazon
Zongqing LuPeking University
A. Rupam MahmoodUniversity of Alberta
Niranjani PrasadMicrosoft Research
Csaba SzepesvariDeepMind & University of Alberta
Matthew E. TaylorUniversity of Alberta
Reviewers

Program committee

Abhishek Naik, University of Alberta
Alberto Maria Metelli, Politecnico di Milano
Aleksandra Faust, Google Brain
Alex Lewandowski, University of Alberta
Amarildo Likmeta, Universita di Bologna, Politecnico di Milano
Benjamin Eisner, Carnegie Mellon University
Bo Chang, Google
Bo Liu, Auburn University
Chih-wei Hsu, Google Research
Craig Sherstan, University of Alberta
Daochen Zha, Texas A&M University
David Janz, University of Cambridge
Hager Radi, University of Alberta
Haipeng Chen, Harvard University
Hanjun Dai, Google Brain
Hengshuai Yao, Huawei
Henry Charlesworth, University of Warwick
Hongming Zhang, University of Alberta
Hugo Caselles-Dupré, Flowers Team (ENSTA ParisTech & INRIA) & Softbank Robotics Europe
Ioannis Boukas, University of Liège
Jincheng Mei, Google
John Martin, University of Alberta
Juan Jose Garau Luis, MIT
Julian Skirzynski, University of California, San Diego
Junfeng Wen, University of Alberta
Justin Basilico, Netflix
Kamyar Azizzadenesheli, Purdue University
Luchen Li, Imperial College London
Manan Tomar, University of Alberta
Myounggyu Won, University of Memphis
Natasha Jaques, UC Berkeley
Nazim Kemal Ure, Istanbul Technical University
Ofir Nachum, Google
Pablo Samuel Castro, Google
Rasool Fakoor, Amazon
Rohin Shah, UC Berkeley
Ruitong Huang, Borealis AI
Ruofan Kong, Microsoft
Sarah Dean, UC Berkeley
Scott Rome, Comcast
Shangtong Zhang, University of Oxford
Shengpu Tang, University of Michigan
Shuai Li, Shanghai Jiao Tong University
Srijita Das, University of Alberta
Srivatsan Krishnan, Harvard University
Stav Belogolovsky, Technion
Subhojyoti Mukherjee, University of Massachusetts Amherst
Tengyang Xie, University of Illinois at Urbana-Champaign
Tengyu Xu, The Ohio State University
Vianney Perchet, ENS Paris-Saclay & Criteo AI Lab
Vidit Saxena, KTH Royal Institute of Technology, Stockholm
Wann-Jiun Ma, Duke University
Xinshi Chen, Georgia Institute of Technology
Xuan Zhao, Microsoft
Ya Le, Google
Yao Liu, Stanford University
Yi Wan, University of Alberta
Yongshuai Liu, University of California, Davis
Yuqing Hou, Intel Labs China
Yuyan Wang, Google Brain
Zhang-Hua Fu, The Chinese University of Hong Kong, Shenzhen
Zhimin Hou, National University of Singapore
Zhipeng Wang, Apple