RESEARCH PAPER
Reinforcement-Learning Robust Tracking Control of Discrete-Time Systems with Diagonal Scaling
1
Department of Automatic Control, Guangdong University of Technology, China
A – Conceptualization; B – Methodology; C – Software; D – Validation; E – Formal analysis; F – Investigation; G – Resources; H – Data curation; I – Writing – original draft; J – Writing – review & editing; K – Visualization; L – Supervision; M – Project administration; N – Funding acquisition
Submission date: 2026-03-27
Final revision date: 2026-08-09
Acceptance date: 2026-08-27
Online publication date: 2026-09-28
Corresponding author
KEYWORDS
Reinforcement learningPolicy iterationRobust tracking controlOutput regulationDiagonal scalingData-driven control
TOPICS
ABSTRACT
This paper addresses a robust tracking problem for linear discrete-time systems by proposing a reinforcement learning (RL) control method based on a diagonal-scaling strategy, offering a solution tailored to the demands of enhanced reliability. To overcome a common limitation in policy-iteration-based RL design, namely, the reliance on an initially stabilizing solution, the tracking control problem is reformulated within the robust output regulation framework as a data-driven, solvable form. A convergence-rate condition is incorporated to relax the dependence on an initial stable control policy. The core contribution lies in addressing unknown system dynamics through a diagonal-scaling strategy, as opposed to conventional scalar convergence-rate scaling. The proposed method enables more flexible and precise closed-loop pole placement. The resulting data-driven controller guarantees asymptotic convergence of the tracking error to zero while maintaining robustness against dynamic uncertainties, ensuring computational efficiency and practical ease of implementation.
REFERENCES (42)
1.
Zhou X, Wan Z. Path planning and motion control of robotic arm based on neural network. Eksploatacja i Niezawodność – Maintenance and Reliability 2025; 27(4): 205794. https://doi.org/10.17531/ein/2....
2.
Kocer Ozturk M, Khaniyev T. Optimal maintenance policy for a Markov deteriorating system under reliability limit. Eksploatacja i Niezawodność – Maintenance and Reliability 2024; 26(4): 190865. https://doi.org/10.17531/ein/1....
3.
Josephin Shermila P, Anu Disney D, Reeda Lenus C, Niruban R. Efficiency and reliability: Optimization of energy management in electric vehicles apply monarch butterfly algorithm and fuzzy logic control. Eksploatacja i Niezawodność – Maintenance and Reliability 2025; 27(3): 200691. https://doi.org/10.17531/ein/2....
4.
Chai TY. Operational optimization and feedback control for complex industrial processes. Acta Automatica Sinica 2013; 39(11): 1744-1757. https://doi.org/10.3724/SP.J.1....
5.
Siciliano B, Khatib O (eds). Springer Handbook of Robotics, 2nd ed. Cham: Springer International Publishing; 2016.
6.
Francis BA, Wonham WM. The internal model principle of control theory. Automatica 1976; 12(5): 457-465. https://doi.org/10.1016/0005-1....
7.
Huang J. Nonlinear Output Regulation: Theory and Applications. Philadelphia: Society for Industrial and Applied Mathematics; 2004. https://doi.org/10.1137/1.9780....
8.
Isidori A, Byrnes CI. Output regulation of nonlinear systems. IEEE Transactions on Automatic Control 1990; 35(2): 131-140. https://doi.org/10.1109/9.4516....
9.
Garcia de Marina H, Cao M, Jayawardhana B. Controlling rigid formations of mobile agents under inconsistent measurements. IEEE Transactions on Robotics 2015; 31(1): 31-39. https://doi.org/10.1109/TRO.20....
10.
Zhang C, Lin W, Ke D, Sun Y. Smoothing tie-line power fluctuations for industrial microgrids by demand side control: An output regulation approach. IEEE Transactions on Power Systems 2019; 34(5): 3716-3728. https://doi.org/10.1109/TPWRS.....
11.
Zhang J, Yang D, Zhang H, Su H. Adaptive secure practical fault-tolerant output regulation of multiagent systems with DoS attacks by asynchronous communications. IEEE Transactions on Network Science and Engineering 2023; 10(6): 4046-4055. https://doi.org/10.1109/TNSE.2....
12.
Li L, Rong D, Fu J, Guo Q. Multi-level event-triggered scheme of output regulation under long-duration DoS attacks in switched systems with dissipativity. IEEE Transactions on Network Science and Engineering 2024; 11(2): 1631-1641. https://doi.org/10.1109/TNSE.2....
13.
Sutton RS, Barto AG. Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA: MIT Press; 2018.
14.
Lewis FL, Vrabie DL, Syrmos VL. Optimal Control, 3rd ed. Hoboken, NJ: John Wiley & Sons; 2012.
15.
Lewis FL, Vrabie D. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE Circuits and Systems Magazine 2009; 9(3): 32-50. https://doi.org/10.1109/MCAS.2....
16.
Zhang H, Liu D, Luo Y, Wang D. Adaptive Dynamic Programming for Control: Algorithms and Stability. London: Springer; 2013.
17.
Jiang Y, Jiang ZP. Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics. Automatica 2012; 48(10): 2699-2704. https://doi.org/10.1016/j.auto....
18.
Vrabie D, Vamvoudakis KG, Lewis FL. Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles. London: Institution of Engineering and Technology; 2012.
19.
Chen C, Xie L, Xie K, Lewis FL, Liu Y, Xie S. Learning the continuous-time optimal decision law from discrete-time rewards. National Science Open 2024; 3(5): 20230054. https://doi.org/10.1360/nso/20....
20.
Kiumarsi B, Lewis FL. Actor–critic-based optimal tracking for partially unknown nonlinear discrete-time systems. IEEE Transactions on Neural Networks and Learning Systems 2015; 26(1): 140-151. https://doi.org/10.1109/TNNLS.....
21.
Kiumarsi B, Lewis FL, Modares H, Karimpour A, Naghibi-Sistani MB. Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics. Automatica 2014; 50(4): 1167-1175. https://doi.org/10.1016/j.auto....
22.
Lewis FL, Vamvoudakis KG. Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 2011; 41(1): 14-25. https://doi.org/10.1109/TSMCB.....
23.
Kiumarsi B, Lewis FL, Naghibi-Sistani MB, Karimpour A. Optimal tracking control of unknown discrete-time linear systems using input-output measured data. IEEE Transactions on Cybernetics 2015; 45(12): 2770-2779. https://doi.org/10.1109/TCYB.2....
24.
Chen C, Xie L, Xie K, Lewis FL, Xie S. Adaptive optimal output tracking of continuous-time systems via output-feedback-based reinforcement learning. Automatica 2022; 146: 110581. https://doi.org/10.1016/j.auto....
25.
Jiang Y, Fan J, Gao W, Chai T, Lewis FL. Cooperative adaptive optimal output regulation of nonlinear discrete-time multi-agent systems. Automatica 2020; 121: 109149. https://doi.org/10.1016/j.auto....
26.
Huang C, Chen C, Xie K, Li Z, Xie S. Adaptive output synchronization with designated convergence rate of multiagent systems based on off-policy reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics: Systems 2024; 54(8): 4667-4678. https://doi.org/10.1109/TSMC.2....
27.
Gao W, Jiang ZP. Adaptive dynamic programming and adaptive optimal output regulation of linear systems. IEEE Transactions on Automatic Control 2016; 61(12): 4164-4169. https://doi.org/10.1109/TAC.20....
28.
Jiang Y, Kiumarsi B, Fan J, Chai T, Li J, Lewis FL. Optimal output regulation of linear discrete-time systems with unknown dynamics using reinforcement learning. IEEE Transactions on Cybernetics 2020; 50(7): 3147-3156. https://doi.org/10.1109/TCYB.2....
29.
Bin M, Marconi L, Teel AR. Adaptive output regulation for linear systems via discrete-time identifiers. Automatica 2019; 105: 422-432. https://doi.org/10.1016/j.auto....
30.
Zhao F, Gao W, Liu T, Jiang ZP. Adaptive optimal output regulation of linear discrete-time systems based on event-triggered output-feedback. Automatica 2022; 137: 110103. https://doi.org/10.1016/j.auto....
31.
Wang Z, Wang Y, Kowalczuk Z. Adaptive optimal discrete-time output-feedback using an internal model principle and adaptive dynamic programming. IEEE/CAA Journal of Automatica Sinica 2024; 11(1): 131-140. https://doi.org/10.1109/JAS.20....
32.
Xia C, Dong Y, Wang C, Xu S. Data-driven output regulation control for constrained linear systems. Science China Information Sciences 2025; 68(3): 132209. https://doi.org/10.1007/s11432....
33.
Lin L, Huang J. Data-driven optimal output regulation for continuous-time linear systems via internal model principle. IEEE Transactions on Automatic Control 2025; 70(6): 4202-4208. https://doi.org/10.1109/TAC.20....
34.
Jiang Y, Chai T, Chen G. Output feedback-based adaptive optimal output regulation for continuous-time strict-feedback nonlinear systems. IEEE Transactions on Automatic Control 2025; 70(2): 767-782. https://doi.org/10.1109/TAC.20....
35.
Liu Z, Li C. Optimal adaptive output regulation of discrete-time nonlinear stochastic systems. SIAM Journal on Control and Optimization 2025; 63(4): 2369-2396. https://doi.org/10.1137/24M167....
36.
Liu T, Huang J. Adaptive cooperative output regulation of discrete-time linear multi-agent systems by a distributed feedback control law. IEEE Transactions on Automatic Control 2018; 63(12): 4383-4390. https://doi.org/10.1109/TAC.20....
37.
Li S, Wang F, Er MJ, Yang Z. Approximate output regulation of discrete-time stochastic multiagent systems subject to heterogeneous and unknown dynamics. IEEE Transactions on Systems, Man, and Cybernetics: Systems 2022; 52(10): 6373-6382. https://doi.org/10.1109/TSMC.2....
38.
Chen C, Xie L, Jiang Y, Xie K, Xie S. Robust output regulation and reinforcement learning-based output tracking design for unknown linear discrete-time systems. IEEE Transactions on Automatic Control 2023; 68(4): 2391-2398. https://doi.org/10.1109/TAC.20....
39.
Liang D, Dong Y, Wang C, Zhai G. Data-driven cooperative output regulation of linear discrete-time multiagent systems with unknown dynamics. IEEE Transactions on Systems, Man, and Cybernetics: Systems 2024; 54(8): 5025-5034. https://doi.org/10.1109/TSMC.2....
40.
Chen C, Xie LH. A data-driven prescribed convergence rate design for robust tracking of discrete-time systems. Journal of Guangdong University of Technology 2021; 38(6): 29-34. https://doi.org/10.12052/gdutx....
41.
Jiang Y, Fan JL, Chai TY. Data-driven optimal output regulation with assured convergence rate. Acta Automatica Sinica 2022; 48(4): 980-991. https://doi.org/10.16383/j.aas....
42.
Chen C, Lewis FL, Li B. Homotopic policy iteration-based learning design for unknown linear continuous-time systems. Automatica 2022; 138: 110153. https://doi.org/10.1016/j.auto....
| eISSN: | 2956-3860 |
| ISSN: | 1507-2711 |
We process personal data collected when visiting the website. The function of obtaining information about users and their behavior is carried out by voluntarily entered information in forms and saving cookies in end devices. Data, including cookies, are used to provide services, improve the user experience and to analyze the traffic in accordance with the Privacy policy. Data are also collected and processed by Google Analytics tool (more).
You can change cookies settings in your browser. Restricted use of cookies in the browser configuration may affect some functionalities of the website.
You can change cookies settings in your browser. Restricted use of cookies in the browser configuration may affect some functionalities of the website.
