Bo (Beau) Liu


Ph.D. AAAI SM, IEEE SM

[Brief Bio] [Google Scholar] [DBLP]





(Machine Learning Conference Ratings) (CS Conference Ratings 1) (CS Conference Ratings 2) (CS Conference Ratings 3)

Publications

Sort by

2024

  • From Past to Future: Rethinking Eligibility Traces.
    Dhawal Gupta, Scott Jordan, Shreyas Chaudhari, B. Liu, Philip Thomas, Bruno C. da Silva
    Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI)

    • 2023 (3)

  • A Critical Review of Inductive Logic Programming Techniques for Explainable AI.
    Zheng Zhang, Levent Yilmaz, B. Liu
    IEEE Transactions on Neural Networks and Learning Systems (IEEE-TNNLS)

  • Offline Reinforcement Learning for Price-Based Demand Response Program Design.
    Ce Xu, B. Liu, Yue Zhao
    Proc. the 57th Annual Conference on Information Sciences and Systems (CISS)

  • Contrastive topic-enhanced network for video captioning.
    Yawen Zeng, Yiru Wang, Dongliang Liao, Gongfu Li, Jin Xu, Xiangmin Xu, B. Liu, Hong Man
    Expert Systems with Applications (ESA)

    • 2022 (6)

  • TOPS: Transition-based volatility-reduced policy search.
    Liangliang Xu, Daoming Lyu, Yangchen Pan, Aiwen Jiang, B. Liu
    International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
    Best and Visionary Paper Award [link]

  • Self-supervised multi-scale pyramid fusion networks for realistic bokeh effect rendering.
    Zhifeng Wang, Aiwen Jiang, Chunjie Zhang, Hanxi Lia, B. Liu
    Journal of Visual Communication and Image Representation

  • TDM: Trustworthy Decision-Making via Interpretability Enhancement.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, Wen Dong, Levent Yilmaz, B. Liu
    IEEE Transactions on Emerging Topics in Computational Intelligence (IEEE-TETCI), 2022 [code]
    This paper builds up a trustworthy decision-making framework with novel trust evaluation and explainability enhancement methods.

  • Tutorial: Risk-averse Reinforcement Learning: Algorithms And Meta-algorithms.
    B. Liu, Bo An, Yangyang Xu.
    Conference on Uncertainty in Artificial Intelligence (UAI), Eindhoven, Netherlands, 2022

  • Tutorial: Efficient Neural-Symbolic Reasoning via Reinforcement Learning.
    Daoming Lyu, B. Liu, Jianshu Chen, Akshat Kumar, Jiajing Ling.
    32nd International Conference on Automated Planning and Scheduling (ICAPS), 2022

  • Tutorial: Risk-aware Single-agent & Multi-agent Reinforcement Learning: Algorithms and Meta-algorithms.
    B. Liu, Bo An, Yangyang Xu.
    International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2022

    • 2021 (5)

  • Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning.
    Zhang, S., B. Liu, Whiteson, S.
    35th AAAI Conference on Artificial Intelligence (AAAI), 2021 [code]
    The first meta-framework that can "ROBUSTIFY" your vanilla RL algorithm! Throw in any off-the-shelf policy search algorithm, and it will return you a risk-aware one!

  • Explainable Neuro-Symbolic Hierarchical Reinforcement Learning.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, B. Liu, Wen Dong, Levent Yilmaz
    Neuro-Symbolic Artificial Intelligence: The State of the Art (book chapter), 2021 [code]

  • Ensemble single image deraining network via progressive structural boosting constraints.
    Long Peng, Aiwen Jiang, Haoran Wei, B. Liu, Mingwen Wang
    Signal Processing: Image Communication, Elsevier, 2021

  • A Lightweight Multi-scale Aggregated Model for Detecting Aerial Images Captured by UAVs.
    Zhaokun Li, Xueliang Liu, Ye Zhao, B. Liu, Zhen Huang, Richang Hong.
    Journal of Visual Communication and Image Representation,2021

  • Crowd understanding and analysis.
    Qi Wang, B. Liu, Jianzhe Lin.
    IET Image Processing (IET-IP), 2021

    • 2020 (3)

  • Model Credibility Revisited: Concepts and Considerations for Appropriate Trust.
    Levent Yilmaz, B. Liu.
    Journal of Simulation (JoS), 2020

  • Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation.
    Zhang, S., B. Liu, Yao, H., Whiteson, S.
    International Conference on Machine Learning (ICML), 2020 [code]
    This paper, together with ACE (Imani et al. 2018), offers an off-policy counterpart of the classical policy gradient theorem with function approximation (Sutton et al. 2000). Specifically, ACE offers the off-policy policy gradient theorem, and we offer the off-policy compatibility with function approximation.

  • Gradientdice: Rethinking generalized offline estimation of stationary values.
    Zhang, S., B. Liu, Whiteson, S.
    International Conference on Machine Learning (ICML), 2020 [code]
    The state-of-the-art achievement on behavior-agnostic off-policy density ratio estimation!

    • 2019 (7)

  • A Human-Centered Data-Driven Planner-Actor-Critic Architecture via Logic Programming.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    35th International Conference on Logic Programming (ICLP), Las Cruces, NM, 2019. [ppt]

  • Systems and methods for neural clinical paraphrase generation.
    Sadid Hasan. S., B. Liu, O. Farri Farri, Junyi Liu, & Aaditya Prakash.
    U.S. Patent Application No. 16/072,128, 2019

  • Optimal Control of Complex Systems through Variational Inference with a Discrete Event Decision Process.
    F. Yang, B. Liu, W. Dong
    Autonomous Agents and Multi-agent Systems (AAMAS), Montreal, Canada, 2019

  • Deep Residual Refining based Pseudo Multi-frame Network for Effective Single Image Super Resolution.
    K. Mei, A. Jiang, J. Li, B. Liu, M. Wang
    IET Image Processing (IET-IP), 2019

  • SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, 2019.
    • This paper gains the state-of-the-art result on Montezuma's Revenge with interpretability at the task level. This is one of the first work towards human-interpretable data-driven decision-making! [ppt] [code] [poster]

  • QUOTA: The Quantile Option Architecture for Reinforcement Learning.
    S. Zhang, B. Mavrin, L. Kong, B. Liu, H. Yao
    33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, 2019.

  • Hierarchical Feature Selection for Random Projection.
    Wang, Q.; Wan, J.; Nie, F.; B. Liu; Young, C.; Li, X
    IEEE Transactions on Neural Networks and Learning Systems (IEEE TNN), 2019

    • 2018 (5)

  • A Block Coordinate Ascent Algorithm for Mean-Variance Optimization.
    B. Liu*, T. Xie* (* equal contribution), Y. Xu, M. Ghavamzadeh, Y. Chow, D. Lyu, D. Yoon
    32nd Conference on Neural Information Processing Systems (NIPS), Montreal, CA, 2018
    • The first risk-sensitive policy search algorithm with single time-scale and sample complexity analysis. It is also the first time introducing coordinate descent/ascent formulation into Reinforcement Learning.
    * reads: Co-primary authors with equal contributions. The authorship is in either alphabetic or reverse alphabetic order. [ppt] [code]

  • A Novel Restoration Algorithm for Noisy Complex Illumination.
    S. Li, Z. Liu, T. Gao, F. Kong, Z. Jiao, A, Yang, B. Liu
    IET Computer Vision (IET-CV), 2018

  • Stable and Efficient Policy Evaluation.
    D. Lyu, B. Liu, M. Geist, W. Dong, S. Biaz, and Q. Wang
    IEEE Transactions on Neural Networks and Learning Systems (IEEE TNN), 2019

  • Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity.
    B. Liu, I. Gemp, M. Ghamvamzadeh, J. Liu, S. Mahadevan, and M. Petrik
    Journal of Artificial Intelligence Research (JAIR), 2018. (Journal version of our 2014 arxiv paper with extended results.) [code]

  • PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making.
    F. Yang, D. Lyu, B. Liu, S. Gustafson
    27th International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, 2018. Press Coverage [ppt] [poster] [code]

    • 2017 (2)

  • Deep Multimodal Reinforcement Network with Contextually Guided Recurrent Attention for Image Question Answering.
    A. Jiang , B. Liu, & M. Wang.
    Journal of Computer Science and Technology, 32(4), 738-748, 2017

  • Neural Clinical Paraphrase Generation with Attention.
    Hasan, S. A., B. Liu, Liu, J. et.al.
    Proceedings of the Clinical Natural Language Processing Workshop (ClinicalNLP), 2017

    • 2016 (3)

  • Dantzig Selector with an Approximately Optimal Denoising Matrix and its Application to Reinforcement Learning.
    B. Liu, L Zhang, J Liu.
    32nd Conference on Uncertainty in Artificial Intelligence (UAI), Jersey City, NJ, 2016

  • Proximal Gradient Temporal Difference Learning Algorithms.
    B. Liu, J Liu, M Ghavamzadeh, S Mahadevan, M Petrik.
    25th International Joint Conference on Artificial Intelligence (IJCAI), New York City, 2016
    • [code]

  • Uncorrelated Group LASSO.
    D Kong, J Liu, B. Liu, X Bao.
    30th AAAI Conference on Artificial Intelligence (AAAI), Phoenix, AZ, Feb 12-17, 2016

    • 2015 (1)

  • Finite-Sample Analysis of Proximal Gradient TD Algorithms.
    B. Liu, J Liu, M Ghavamzadeh, S Mahadevan, M Petrik.
    31st Conference on Uncertainty in Artificial Intelligence (UAI), Amsterdam, The Netherlands, July 12-16, 2015, Facebook Best Student Paper Award. [ppt] [video] [code]
    The first paper giving sample complexity analysis of RL algorithms with linear computational cost per step.

    • 2014 (2)

  • Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual Spaces.
    S Mahadevan, B. Liu, P Thomas, W Dabney, S Giguere, N Jacek, I Gemp, J Liu
    arXiv preprint arXiv:1405.6757, 2014
    The first paper setting up a stochastic optimization framwork for TD learning using Legendre-Fenchel duality and proximal operators, and pointing out GTD algorithm is a saddle-point algorithm.

  • Bluetooth aided mobile phone localization: a nonlinear neural circuit approach.
    S Li, Y Lou, B. Liu
    ACM Transactions on Embedded Computing Systems (ACM TECS), 2014

    • 2013 (4)

  • Selective Positive-Negative Feedback Produces the Winner-Take-All Competition in Recurrent Neural Networks.
    S Li, B. Liu, Y Li
    IEEE Transactions on Neural Networks and Learning Systems (IEEE TNN) 24, 301-309, 2013

  • Neural network based mobile phone localization using Bluetooth connectivity.
    S Li, B. Liu, B Chen, Y Lou
    Neural Computing & Applications, 2013

  • Decentralized control of collaborative redundant manipulators with partial command coverage via locally connected recurrent neural networks.
    S Li, H Cui, Y Li, B. Liu, Y Lou
    Neural Computing & Applications, 1-10, 2013

  • A Nonlinear Model to Generate the Winner-take-all Competition.
    S Li, Y Wang, J Yu, B. Liu
    Communications in Nonlinear Science and Numerical Simulation, 2013

    • 2012 (6)

  • Regularized Off-Policy TD-Learning.
    B. Liu,     S Mahadevan, J Liu.
    26th Annual Conference on Neural Information Processing Systems (NIPS), Lake Tahoe, Nevada, 2012, December 3-6, Spotlight Presentation (5% acceptance). [ppt] [video]
    • The first paper introducing saddle-point formulation into TD learning and Reinforcement Learning.

  • Sparse Q-learning with Mirror Descent.
    S Mahadevan, B. Liu.
    28th Conference on Uncertainty in Artificial Intelligence (UAI), August 15-17, 2012, Catalina Island, CA. [ppt]

  • Sparse Manifold Alignment.
    B. Liu, C Wang, H Vu, S Mahadevan.
    Univ. of Massachusetts Technical Report UM-CS-2012-030.

  • Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks.
    S Li, S Chen, B. Liu, Y Li, Y Liang
    Neurocomputing, 2012, One of the most-cited Neurocomputing paper since 2012 according to Scopus

  • Intelligent control of a sensor-actuator system via kernelized least-squares policy iteration.
    B. Liu, Sanfeng Chen, Shuai Li, Yongsheng Liang.
    Sensors,2012.

  • Neural Network-Based Mobile Phone Localization Using Bluetooth Connectivity.
    Shuai Li, B. Liu, Baogang Chen, and Yuesheng Lou.
    Neural Computing and Applications,2012.

    • 2011 (1)

  • Compressive Reinforcement Learning with Oblique Random Projections.
    B. Liu    , S Mahadevan.
    Univ. of Massachusetts Technical Report UM-CS-2011-024.

    • 2010 (4)

  • Basis Construction from Power Series Expansions of Value Functions.
    S Mahadevan, B. Liu.
    24th Annual Conference on Neural Information Processing Systems (NIPS), Vancouver, B.C., Canada, 2010, December 6-8. [ppt]

  • Two-time-scale online actor-critic paradigm driven by POMDP.
    B. Liu, H He, DW Repperger
    International Conference on Networking, Sensing and Control (ICNSC), 2010.

  • A Hierarchical Learning Architecture with Multiple-Goal Representations Based on Adaptive Dynamic Programming.
    H He, B. Liu,
    International Conference on Networking, Sensing and Control (ICNSC), 2010.

  • Adaptive Dual Network Design for a Class of SIMO Systems with Nonlinear Time-variant Uncertainties.
    B. Liu    , HB He, S Chen
    Acta Automatica Sinica 36 (4), 564-572, 2010


  • TDM: Trustworthy Decision-Making via Interpretability Enhancement.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, Wen Dong, Levent Yilmaz, B. Liu
    IEEE Transactions on Emerging Topics in Computational Intelligence (IEEE-TETCI), 2021
    This paper builds up a trustworthy decision-making framework with novel trust evaluation and explainability enhancement methods.

  • Explainable Neuro-Symbolic Hierarchical Reinforcement Learning.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, B. Liu, Wen Dong, Levent Yilmaz
    Neuro-Symbolic Artificial Intelligence: The State of the Art (book chapter), 2021 [code]

  • Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning.
    Zhang, S., B. Liu, Whiteson, S.
    35th AAAI Conference on Artificial Intelligence (AAAI), 2021

  • Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation.
    Zhang, S., B. Liu, Yao, H., Whiteson, S.
    International Conference on Machine Learning (ICML), 2020
    The first off-policy convergent actor-critic w/o stringent conditions! All assumptions are mild!

  • Gradientdice: Rethinking generalized offline estimation of stationary values.
    Zhang, S., B. Liu, Whiteson, S.
    International Conference on Machine Learning (ICML), 2020
    The state-of-the-art achievement on behavior-agnostic off-policy density ratio estimation!

  • A Human-Centered Data-Driven Planner-Actor-Critic Architecture via Logic Programming.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    35th International Conference on Logic Programming (ICLP), Las Cruces, NM, 2019. [ppt]

  • Optimal Control of Complex Systems through Variational Inference with a Discrete Event Decision Process.
    F. Yang, B. Liu, W. Dong
    Autonomous Agents and Multi-agent Systems (AAMAS), Montreal, Canada, 2019

  • SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, 2019.
    • This paper gains the most state-of-the-art result on Montezuma's Revenge with interpretability at the task level. This is one of the first work towards human-interpretable data-driven decision-making! [ppt] [code] [poster]

  • QUOTA: The Quantile Option Architecture for Reinforcement Learning.
    S. Zhang, B. Mavrin, L. Kong, B. Liu, H. Yao
    33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, 2019.

  • A Block Coordinate Ascent Algorithm for Mean-Variance Optimization.
    B. Liu*, T. Xie* (* equal contribution), Y. Xu, M. Ghavamzadeh, Y. Chow, D. Lyu, D. Yoon
    32nd Conference on Neural Information Processing Systems (NIPS), Montreal, CA, 2018
    • The first risk-sensitive policy search algorithm with single time-scale and sample complexity analysis. It is also the first time introducing coordinate descent/ascent formulation into Reinforcement Learning.
    * reads: Co-primary authors with equal contributions. The authorship is in either alphabetic or reverse alphabetic order.

  • Stable and Efficient Policy Evaluation.
    D. Lyu, B. Liu, M. Geist, W. Dong, S. Biaz, and Q. Wang
    IEEE Transactions on Neural Networks and Learning Systems (IEEE TNN), 2019

  • PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making.
    F. Yang, D. Lyu, B. Liu, S. Gustafson
    27th International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, 2018. Press Coverage [ppt] [poster]

  • Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity.
    B. Liu, I. Gemp, M. Ghamvamzadeh, J. Liu, S. Mahadevan, and M. Petrik
    Journal of Artificial Intelligence Research (JAIR), 2018. (Journal version of our 2014 arxiv paper with extended results.)
    • [code]
  • Proximal Gradient Temporal Difference Learning Algorithms.
    B. Liu, J Liu, M Ghavamzadeh, S Mahadevan, M Petrik.
    25th International Joint Conference on Artificial Intelligence (IJCAI), New York City, 2016
    • [code]
  • Dantzig Selector with an Approximately Optimal Denoising Matrix and its Application to Reinforcement Learning.
    B. Liu, L Zhang, J Liu.
    32nd Conference on Uncertainty in Artificial Intelligence (UAI), Jersey City, NJ, 2016

  • Finite-Sample Analysis of Proximal Gradient TD Algorithms.
    B. Liu, J Liu, M Ghavamzadeh, S Mahadevan, M Petrik.
    31st Conference on Uncertainty in Artificial Intelligence (UAI), Amsterdam, The Netherlands, July 12-16, 2015, Facebook Best Student Paper Award. [ppt] [video] [code]
    The first paper giving sample complexity analysis of RL algorithms with linear computational cost per step.

  • Proximal Reinforcement Learning: A New Theory of Sequential Decision Making in Primal-Dual Spaces.
    S Mahadevan, B. Liu, P Thomas, W Dabney, S Giguere, N Jacek, I Gemp, J Liu
    arXiv preprint arXiv:1405.6757, 2014
    The first paper setting up a stochastic optimization framwork for TD learning using Legendre-Fenchel duality and proximal operators, and pointing out GTD algorithm is a saddle-point algorithm.
    • [code]
  • Regularized Off-Policy TD-Learning.
    B. Liu,     S Mahadevan, J Liu.
    26th Annual Conference on Neural Information Processing Systems (NIPS), Lake Tahoe, Nevada, 2012, December 3-6, Spotlight Presentation (5% acceptance). [ppt] [video]
    • The first paper introducing saddle-point formulation into TD learning and Reinforcement Learning.

  • Sparse Q-learning with Mirror Descent.
    S Mahadevan, B. Liu.
    28th Conference on Uncertainty in Artificial Intelligence (UAI), August 15-17, 2012, Catalina Island, CA. [ppt]

  • Compressive Reinforcement Learning with Oblique Random Projections.
    B. Liu    , S Mahadevan.
    Univ. of Massachusetts Technical Report UM-CS-2011-024.

  • Basis Construction from Power Series Expansions of Value Functions.
    S Mahadevan, B. Liu.
    24th Annual Conference on Neural Information Processing Systems (NIPS), Vancouver, B.C., Canada, 2010, December 6-8. [ppt]

  • Two-time-scale online actor-critic paradigm driven by POMDP.
    B. Liu, H He, DW Repperger
    International Conference on Networking, Sensing and Control (ICNSC), 2010.

    • Best Paper Award Nominee

    Computer Vision

  • A Lightweight Multi-scale Aggregated Model for Detecting Aerial Images Captured by UAVs.
    Zhaokun Li, Xueliang Liu, Ye Zhao, B. Liu, Zhen Huang, Richang Hong.
    Journal of Visual Communication and Image Representation, 2021

  • A Novel Restoration Algorithm for Noisy Complex Illumination.
    S. Li, Z. Liu, T. Gao, F. Kong, Z. Jiao, A, Yang, B. Liu
    IET Computer Vision (IET-CV), 2018

  • Deep Residual Refining based Pseudo Multi-frame Network for Effective Single Image Super Resolution.
    K. Mei, A. Jiang, J. Li, B. Liu, M. Wang
    IET Image Processing (IET-IP), 2019

  • Deep Multimodal Reinforcement Network with Contextually Guided Recurrent Attention for Image Question Answering.
    A. Jiang, B. Liu, & M. Wang.
    Journal of Computer Science and Technology, 32(4), 738-748, 2017

    • Power Systems

  • Offline Reinforcement Learning for Price-Based Demand Response Program Design.
    C. Xu, B. Liu, & Y. Zhao.
    Proc. the 57th Annual Conference on Information Sciences and Systems (CISS), 2023.

    • Robotics

  • A Lightweight Multi-scale Aggregated Model for Detecting Aerial Images Captured by UAVs.
    Zhaokun Li, Xueliang Liu, Ye Zhao, B. Liu, Zhen Huang, Richang Hong.
    Journal of Visual Communication and Image Representation, 2021

  • Bluetooth aided mobile phone localization: a nonlinear neural circuit approach.
    S Li, Y Lou, B. Liu
    ACM Transactions on Embedded Computing Systems (ACM TECS), 2014

  • Selective Positive-Negative Feedback Produces the Winner-Take-All Competition in Recurrent Neural Networks.
    S Li, B. Liu, Y Li
    IEEE Transactions on Neural Networks and Learning Systems (IEEE TNN) 24, 301-309, 2013

  • Decentralized kinematic control of a class of collaborative redundant manipulators via recurrent neural networks.
    S Li, S Chen, B. Liu, Y Li, Y Liang
    Neurocomputing, 2012, One of the most-cited Neurocomputing paper since 2012 according to Scopus

  • Decentralized control of collaborative redundant manipulators with partial command coverage via locally connected recurrent neural networks.
    S Li, H Cui, Y Li, B. Liu, Y Lou
    Neural Computing & Applications, 1-10, 2012

  • Neural network based mobile phone localization using Bluetooth connectivity.
    S Li, B. Liu, B Chen, Y Lou
    Neural Computing & Applications, 2012

  • Adaptive Dual Network Design for a Class of SIMO Systems with Nonlinear Time-variant Uncertainties.
    B. Liu    , HB He, S Chen
    Acta Automatica Sinica 36 (4), 564-572, 2010

    • Healthcare

  • Neural Clinical Paraphrase Generation with Attention.
    Hasan, S. A., B. Liu, Liu, J. et.al.
    Proceedings of the Clinical Natural Language Processing Workshop (ClinicalNLP), 2017

  • Systems and methods for neural clinical paraphrase generation.
    Sadid Hasan. S., B. Liu, O. Farri Farri, Junyi Liu, & Aaditya Prakash.
    U.S. Patent Application No. 16/072,128, 2019

    • Transparency/Explainability

  • TDM: Trustworthy Decision-Making via Interpretability Enhancement.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, Wen Dong, Levent Yilmaz, B. Liu
    IEEE Transactions on Emerging Topics in Computational Intelligence (IEEE-TETCI), 2021
    This paper builds up a trustworthy decision-making framework with novel trust evaluation and explainability enhancement methods.

  • Explainable Neuro-Symbolic Hierarchical Reinforcement Learning.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, B. Liu, Wen Dong, Levent Yilmaz
    Neuro-Symbolic Artificial Intelligence: The State of the Art (book chapter), 2021 [code]

  • Model Credibility Revisited: Concepts and Considerations for Appropriate Trust.
    Levent Yilmaz, B. Liu.
    Journal of Simulation (JoS), 2020

  • A Human-Centered Data-Driven Planner-Actor-Critic Architecture via Logic Programming.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    35th International Conference on Logic Programming (ICLP), Las Cruces, NM, 2019. [ppt]

  • SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, 2019.
    • This paper gains the most state-of-the-art result on Montezuma's Revenge with interpretability at the task level. This is one of the first work towards human-interpretable data-driven decision-making! [ppt] [code] [poster]

  • PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making.
    F. Yang, D. Lyu, B. Liu, S. Gustafson
    27th International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, 2018. Press Coverage [ppt] [poster]

    • Safety and Risk-Awareness

  • Mean-Variance Policy Iteration for Risk-Averse Reinforcement Learning.
    Zhang, S., B. Liu, Whiteson, S.
    35th AAAI Conference on Artificial Intelligence (AAAI), 2021

  • A Block Coordinate Ascent Algorithm for Mean-Variance Optimization.
    B. Liu*, T. Xie* (* equal contribution), Y. Xu, M. Ghavamzadeh, Y. Chow, D. Lyu, D. Yoon
    32nd Conference on Neural Information Processing Systems (NIPS), Montreal, CA, 2018
    • The first risk-sensitive policy search algorithm with single time-scale and sample complexity analysis. It is also the first time introducing coordinate descent/ascent formulation into Reinforcement Learning.
    * reads: Co-primary authors with equal contributions. The authorship is in either alphabetic or reverse alphabetic order. [ppt] [code]

    Robust and Adaptiveness

  • Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation.
    Zhang, S., B. Liu, Yao, H., Whiteson, S.
    International Conference on Machine Learning (ICML), 2020
    The first off-policy convergent actor-critic w/o stringent conditions! All assumptions are mild!

  • Proximal Gradient Temporal Difference Learning: Stable Reinforcement Learning with Polynomial Sample Complexity.
    B. Liu, I. Gemp, M. Ghamvamzadeh, J. Liu, S. Mahadevan, and M. Petrik
    Journal of Artificial Intelligence Research (JAIR), 2018. (Journal version of our 2014 arxiv paper with extended results.)

  • Stable and Efficient Policy Evaluation.
    D. Lyu, B. Liu, M. Geist, W. Dong, S. Biaz, and Q. Wang
    IEEE Transactions on Neural Networks and Learning Systems (IEEE TNN), 2019

  • Proximal Gradient Temporal Difference Learning Algorithms.
    B. Liu, J Liu, M Ghavamzadeh, S Mahadevan, M Petrik.
    25th International Joint Conference on Artificial Intelligence (IJCAI), New York City, 2016

  • Finite-Sample Analysis of Proximal Gradient TD Algorithms.
    B. Liu, J Liu, M Ghavamzadeh, S Mahadevan, M Petrik.
    31st Conference on Uncertainty in Artificial Intelligence (UAI), Amsterdam, The Netherlands, July 12-16, 2015, Facebook Best Student Paper Award. [ppt] [video] [code]
    The first paper giving sample complexity analysis of RL algorithms with linear computational cost per step.

    • Privacy-Preserving

  • Gradientdice: Rethinking generalized offline estimation of stationary values.
    Zhang, S., B. Liu, Whiteson, S.
    International Conference on Machine Learning (ICML), 2020
    The state-of-the-art achievement on behavior-agnostic off-policy density ratio estimation!

    • Fairness

    in progress.

  • Hierarchical Feature Selection for Random Projection.
    Wang, Q.; Wan, J.; Nie, F.; B. Liu; Young, C.; Li, X
    IEEE Transactions on Neural Networks and Learning Systems (IEEE TNN), 2019

  • Dantzig Selector with an Approximately Optimal Denoising Matrix and its Application to Reinforcement Learning.
    B. Liu, L Zhang, J Liu.
    32nd Conference on Uncertainty in Artificial Intelligence (UAI), Jersey City, NJ, 2016

  • Uncorrelated Group LASSO.
    D Kong, J Liu, B. Liu, X Bao.
    30th AAAI Conference on Artificial Intelligence (AAAI), Phoenix, AZ, Feb 12-17, 2016

  • Sparse Manifold Alignment.
    B. Liu, C Wang, H Vu, S Mahadevan.
    Univ. of Massachusetts Technical Report UM-CS-2012-030.

  • A Nonlinear Model to Generate the Winner-take-all Competition.
    S Li, Y Wang, J Yu, B. Liu
    Communications in Nonlinear Science and Numerical Simulation, 2012


  • TDM: Trustworthy Decision-Making via Interpretability Enhancement.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, Wen Dong, Levent Yilmaz, B. Liu
    IEEE Transactions on Emerging Topics in Computational Intelligence (IEEE-TETCI), 2021
    This paper builds up a trustworthy decision-making framework with novel trust evaluation and explainability enhancement methods.

  • Explainable Neuro-Symbolic Hierarchical Reinforcement Learning.
    Daoming Lyu, Fangkai Yang, Hugh Kwon, B. Liu, Wen Dong, Levent Yilmaz
    Neuro-Symbolic Artificial Intelligence: The State of the Art (book chapter), 2021 [code]

  • A Human-Centered Data-Driven Planner-Actor-Critic Architecture via Logic Programming.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    35th International Conference on Logic Programming (ICLP), Las Cruces, NM, 2019. [ppt]

  • SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning.
    D. Lyu, F. Yang, B. Liu, S. Gustafson
    33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, 2019.
    • This paper gains the most state-of-the-art result on Montezuma's Revenge with interpretability at the task level. This is one of the first work towards human-interpretable data-driven decision-making! [ppt] [code]

  • PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making.
    F. Yang, D. Lyu, B. Liu, S. Gustafson
    27th International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, 2018. Press Coverage [ppt] [poster]