持续学习综述：挑战分类任务中的遗忘问题
Artificial neural networks thrive in solving the classification problem for a particular rigid task, acquiring knowledge through generalized learning behaviour from a distinct training phase. The resulting network resembles a static entity of knowledge, with endeavours to extend this knowledge without targeting the original task resulting in a catastrophic forgetting. Continual learning shifts this paradigm towards networks that can continually accumulate knowledge over different tasks without the need to retrain from scratch. 持续学习将这种范式转变为网络，可以在不同的任务中不断积累知识，而不需要从头再训练。 We focus on task incremental classification, where tasks arrive sequentially and are delineated by clear boundaries.
Our main contributions concern (1) a taxonomy and extensive overview of the state-of-the-art; (2) a novel framework to continually determine the stability-plasticity trade-off of the continual learner; (3) a comprehensive experimental comparison of 11 state-of-the-art continual learning methods and 4 baselines. We empirically scrutinize method strengths and weaknesses on three benchmarks, considering Tiny Imagenet and large-scale unbalanced iNaturalist and a sequence of recognition datasets. We study the influence of model capacity, weight decay and dropout regularization, and the order in which the tasks are presented, and qualitatively compare methods in terms of required memory, computation time and storage.
研究了模型容量、权重衰减、丢失正则化的影响，以及任务出现的顺序
静态模型：无法随着时间的推移调整模型，新数据可用时需重新开始训练模型；存储限制和隐私问题
神经网络：当学习新的概念时，会遭受旧概念的灾难性遗忘
数据可以来自changing input domains变化的输入域，只有a small portion of input data from one or few tasks一个或几个任务一次同时可用。
挑战：随着new tasks or domains增加，先前学习的任务或领域性能不会随着时间的推移而显著下降
<aside> 💡 Plasticity 可塑性：整合新知识的能力 Stability 稳定性：保留先前的知识
</aside>
Task Incremental Learning：考虑一系列任务，一次只接收一个任务的训练数据进行训练，直到收敛