Ningsheng Zhao

Outputs

CLASSIFIER RANK - A NEW CLASSIFICATION ASSESSMENT METHOD

Most of the commonly used confusion matrix-based classification performance metrics, such as f1_score, MCC, and PRC, are sensitive to the class imbalance. To address this problem, we propose a novel classifier evaluation method, called classifier rank which provides the rank of the classifier in the space of all possible classifiers. To rank a classifier, we find the distribution of performance metrics conditional on arbitrary class ratio. However, some metrics like PRC are functions of a large sequence of confusion matrices whose joint distribution is difficult to estimate. Hence, we propose a directed binary tree model to effectively represent this large-scale joint distribution. As a result, we can estimate the classifier rank using graphical inference algorithms, such as Monte-Carlo algorithm.

TOWARDS EXPLAINABLE MACHINE LEARNING OPERATIONS (MLOPS)

A new field of Machine Learning Operations (MLOps) has recently emerged to address the issue of efficient operations and management of machine learning models. In this work, we review commonly used MLOps approaches and explaInability methods and suggest novel methods to address challenges of current model explainability methods. We discuss the need for university curriculum to address the issues facing management of AI models, their diagnostics and explainability.

Explaining Machine Learning Informatively

Description: 1. Explore the informative dependence of model outcomes on each feature, e.g., what elements are associated to certain type of risk; 2. Remedy untenable feature independence or parametric assumptions that cause under-informative explanations; 3. Improve the robustness to data sparsity that easily results in over-informative explanations; 4. Circumvent adversarial attacks on model explanations.

TOWARDS EXPLAINABLE MACHINE LEARNING OPERATIONS (MLOPS)

A new field of Machine Learning Operations (MLOps) has recently emerged to address the issue of efficient operations and management of machine learning models. In this work, we review commonly used MLOps approaches and explaInability methods and suggest novel methods to address challenges of current model explainability methods. We discuss the need for university curriculum to address the issues facing management of AI models, their diagnostics and explainability.