1 Parallelization of Time Domain Dynamic Simulation: Existing Tools and New Techniques Panel Session “Faster than Real-time Dynamics Simulation” 2014 IEEE Power and Energy Society General Meeting, July 31, 2014 Naresh Acharya1, Juan Sanchez-Gasca2, Bruno Leonardi2, Zhenyu Huang3, Ruisheng Diao3 1GE Global Research, 2GE Energy Management, 3PNNL 2 Acknowledgement Acknowledgement: “This material is based upon work supported by the Department of Energy under Award Number DE-OE0000626” Disclaimer: “This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.” 3 Content • • • • • Context DAE Solution Techniques Parallelization in Existing Architecture Alternative DAE Solution Technique Conclusion 4 Context Objective: Improve the performance of dynamic simulation via parallel computing and advanced solvers 5 DAE Solution Techniques DAE Partitioned (Alternating) Simultaneous (Direct) Explicit Integration Implicit Integration Implicit Integration One Set of AEs Tow Sets of AEs One set of AEs Nonlinear and/or Linear Solvers Solution Shuangshuang Jin; Zhenyu Huang; Ruisheng Diao; Di Wu; Yousu Chen, "Parallel implementation of power system dynamic simulation," Power and Energy Society General Meeting (PES), 2013 IEEE , vol., no., pp.1,5, 21-25 July 2013 6 Parallelization Approaches: Pros & Cons Approach 1: Use parallelization under current PSLF architecture • Pros • Faster to implement • Less code changes • Cons • Speed gains are limited by the speed of slowest loop on current program architecture • Larger changes can be challenging and require significant modifications Approach 2: Alternative DAE modeling (implicit integration) • Pros • Expect greater speed gains than approach 1 • Can be used as the basis for other tools development • Cons • Slower to implement • More code changes and more code development • Need to reformulate solution approach in PSLF 7 PSLF Core Architecture Improvements • The most effective way to reduce the solution speed of PSLF dynamics is a combination of two techniques • Parallelization of ODEs • Fast linear solver (Network) • Solver speed will be directly dependent on matrix sparsity structure and problem size 8 Challenges Faced • Program is already well written and optimized • Complex program structure and legacy code (program has been written over 30 years) • Overhead costs of parallelization methods will adversely impact performance on small cases • Replacement of linear solver involves significant changes in the core 9 Selective an Appropriate Solver • Literature review (and preliminary results) have indicated that for current power system matrix sizes – Direct methods have superior performance over iterative methods – Serial solvers are faster than parallel ones • As problems grow larger, iterative methods are expected to outperform direct methods 10 Preliminary Results (ODE Parallelization) • Execution time of a 1s simulation on an real size system Run Serial Parallel (2 threads) Parallel (3 threads) 1 11.10s 9.88s 10.22s 2 11.91s 9.71s 10.15s 3 11.60s 9.78s 10.21s Average 11.53s 9.79s 10.19s Gain - Reduction of ~15.1% Reduction of ~12.7% Conclusion: A more substantial performance gain will require additional modifications in the PSLF solution scheme 11 Linear Solver Replacement • Successful code replacement (large modifications in the program) • Solution accuracy confirmed on small case 12 Achievements • Current parallel implementation leverages existing architecture of PSLF – Faster to incorporate – Utilizes extensive model database – Reduces the chances of errors • Identification of faster solvers that could improve factorization speed significantly (nearly 30x) 13 Lessons Learned • Speed improvements in the PSLF dynamic simulation • Code modularity facilitates future solver replacements • Understanding of state of the art solvers For more substantial speed gains, a change is solution architecture is required (integration methods/DAE) 14 Parallelization Approaches: Pros & Cons Approach 1: Use parallelization under current PSLF architecture • Pros • Faster to implement • Less code changes • Cons • Speed gains are limited by the speed of slowest loop on current program architecture • Larger changes can be challenging and require significant modifications Approach 2: Alternative DAE modeling (implicit integration) • Pros • Expect greater speed gains than approach 1 • Can be used as the basis for other tools development • Cons • Slower to implement • More code changes and more code development • Need to reformulate solution approach in PSLF 15 Advantages of Implicit Integration • Has better numerical stability • Enables larger time steps for simulation Trapezoidal method Modified Euler Method Max allowable time step: Max allowable time step: 0.134 s 0.01 s Tested on a 2-area system 16 Approach • • • • Implicit integration using Trapezoidal rule A time stacking method Faster linear solvers Adaptive time stepping with much larger time steps 17 Simultaneous Time Staking Method • Solving multiple time steps simultaneously – Combine discretized differential equation and algebraic equation – “Stack” multiple steps for simultaneous solution Sequential time-stepping process: Time-stacking method: 18 Jacobian Properties • An example of Jacobian matrix derived from time-stacking method for 16g68b system – Matrix properties: real, sparse, non-symmetric, non-diagonally dominant – With a large condition number: 8.6x10^6 Size: 1280x1280, nnz=14016 Zoom-in view 19 Linear Solvers for Time-Stacking • Direct solver vs iterative solvers (averaged 10,000 runs) – SparseLU (UMFPACK): 0.0165 sec – BiCGSTAB + ILU preconditioner: 0.0190 sec – GMRes + ILU preconditioner: 0.0266 sec • It is expected that iterative solvers outperform direct solvers for a much larger Jacobian matrix, using multiple processors 20 Adaptive Time Stepping • Time step is adjusted based on – Local error estimate – Performance of Newton corrector iteration – Switching events and faults • Comprehensive logics used to adaptively change the time stepping – 10%-30% speedup observed from various testing 21 Adaptive Time Stepping ω δ 21 Case 1: constant time step, 0.01s Case 2: adaptive time step, up to 0.1s 22 Conclusion • Some speed-up with parallelization in existing code (explicit integration technique) • Direct linear solvers have better performance than iterative solvers in existing architecture • Implicit integration technique with simultaneous time stacking method shows potential for better speed-up in parallel environment 23 THANK YOU Contact: [email protected]
© Copyright 2024 ExpyDoc
