Intersecting two large research areas--numerical analysis and applied probability/quering theory--this book is a self -contained introduction to the numerical solution of structured Markov chains, which have a wide applicability in queueing theory and stochastic modeling. Aimed at graduates and researchers in numerical analysis, applied mathematics, probability, engineering and computer science it provides a thorough overview of the current literature. The book, consisting of nine chapters, is presented in three parts. Part 1 covers a basic description of the fundamental concepts related to Markov chains, a systematic treatment of the structure matrix tools, including finite Toeplitz matrices, displacement operators, FFT, and the infinite block Toeplitz matrices, their relationship with matrix power series and the fundamental problems of solving matrix equations can computing canonical factorizations. Part 2 deals with the description and analysis of structure Markov chains and includes M/G/1, quasi-birth0death processes, non-skip-free queries and tree-like processes. Part 3 covers solution algorithms where new convergence and applicability results are proved. Each chapter ends with bibliographic notes for further reading, and the book ends with an appendix collecting the main general concepts and results used in the book, a list of the main annotations and algorithms used in the book, and an extensive index.