A qualitative study of high-dimensional dynamical systems

Over one hundred years ago, Poincar\'e pioneered the concept of the qualitative study of ordinary differential equations and dynamical systems. The idea was, in a way, revolutionary; instead of studying nature by studying a particular equation, Poincar\'e's vision was to study nature by studying the spaces of functions used to model nature. However, because of the level of abstraction, if one remains solely in an analytical realm this framework can often be difficult to connect to experimental science in a practical way. My research program utilizes the continuing advances in computing resources and is fundamentally a qualitative, computational study of dynamical systems. This approach lies in between the abstract, pure math study of dynamical systems and the scientific-minded numerical studies of particular natural systems. The work is close to mathematics in regards to the problems being addressed and the lack of focus on a particular equation that has a specific physical analogy. However, often the experimental scientific strategy of problem-solving is employed via statistical studies that often utilize physical analogies. Moreover, a key goal of this construction is to provide a framework within which practical connections can be made between abstract dynamical systems frameworks and concrete physical frameworks. This approach is fundamentally interdisciplinary in character because all concrete, deterministic, time-dependent models have geometric structures carved out in time associated with them that can, in theory, be related to the abstract mathematical results. Aside from potential applicability to branches science such as neurobiology, ecology, economics, cryptology, atmospheric science and physics, the approach combines several branches of mathematics, such as dynamical systems, learning theory, random matrix theory, time-series analysis, differential topology, measure and probability theory and information geometry.

The work highlighted in this summary focuses in particular on high-dimensional dynamics. The emphasis is motivated primarily by the outline of Poincare's goal of achieving a qualitative geometric understanding of mechanisms that yield persistent types of dynamics to gain an understanding of the required ingredients for ergodic-like dynamics to remain stable with perturbations. Moreover, achieving a geometric understanding of the transitions under parameter variation between dynamical types --- between fixed points, periodic and quasi-periodic orbits, high-entropy hyper-chaos, low-entropy chaos, and spatially-extended dynamics --- is often fundamental to understanding the stability mechanics of these dynamic types in and of themselves. These issues are approached in a siege style using whatever tools and methods work best for unwrapping and understanding the dynamics and are summarized in what follows.

Abstract: A space of time-delay dynamical systems known to be universal approximators (neural networks) is investigated qualitatively with respect to increasing dimension and number of parameters. The space of mappings is partitioned with a bifurcation parameter according to the qualitative dynamic type (fixed points, chaos, etc). Scaling laws are then investigated regarding the maximum largest Lyapunov exponent, the entropy, Kaplan-Yorke dimension, the maximum number of positive exponents dependent on the number of parameters in the mapping and the dimension of the mapping. All of the of the aforementioned quantities increase when the number of parameters and/or dimensions is increased.

Position and status: This paper discusses the stratification of the space of neural networks studied relative to a measure. Moreover, this work lays the foundation for how various dynamical diagnostics depend on the dimension of the state and parameter spaces. This paper is essential for much of the work done later as results from it are used for re-normalization of the dynamical diagnostics.

Submitted to:

Abstract: Results regarding probable bifurcations from fixed points are presented in the context of general dynamical systems (real, random matrices), time-delay dynamical systems (companion matrices), and a set of mappings known for their properties as universal approximators (neural networks). The eigenvalue spectra is considered both numerically and analytically using previous work of Edelman et. al. Based upon the numerical evidence, various conjectures are presented. The conclusion is that in many circumstances, most bifurcations from fixed points of large dynamical systems will be due to complex eigenvalues. Nevertheless, surprising situations are presented for which the aforementioned conclusion is not general, e.g. real random matrices with Gaussian elements with a large positive mean and finite variance.

Position and status: All the neural networks studied have fixed points for a particular parameter setting. This paper addresses the behavior of the local bifurcations from fixed points to all other types of dynamics, which, according to the stratification scheme presented in the scaling paper, is defined as region I. The particular arguments are heavily influenced by random matrix theory of various types.

Submitted to: J. Stat. Phys.

Abstract: This paper examines the most probable route to chaos in high-dimensional dynamical systems function space (time-delay neural networks) endowed with a probability measure in a computational setting . The most probable route to chaos (relative to the measure we impose on the function space) as the dimension is increased is observed to be a sequence of Neimark-Sacker bifurcations into chaos. The analysis is composed of the study of an example dynamical system followed by a probabilistic study of the ensemble of dynamical systems from which the example was drawn. A scenario depicting the decoupling of the stable manifolds of the torus leading up to the onset of chaos in high-dimensional dissipative dynamical systems is also presented.

Position and status: Following a study of bifurcations from fixed points, a next logical phase of analysis is a study of the transition to chaos following the bifurcation of a fixed point; this paper addresses this issue. This work is again inspired by results from random matrix theory and random matrix products. However, such analytical tools are currently difficult to apply in this context. Thus, this represents a purely computational study of an ensemble of mappings that addresses possible routes to chaos.

Submitted to: Physica D.

Abstract: An extensive statistical survey of universal approximators shows that as the dimension of a typical dissipative dynamical system is increased, the number of positive Lyapunov exponents increases monotonically and the number of parameter windows with periodic behavior decreases. A subset of parameter space remains where non-catastrophic topological change induced by small parameter variation becomes inevitable. A geometric mechanism depending on dimension and an associated conjecture depict why topological change is expected but not catastrophic, thus providing an explanation of how and why deterministic chaos is persistent in high dimensions.

Position and status: This is a survey paper analyzing a large part of the chaotic portion parameter space of the space of models relative to a measure on the state and parameter spaces. It is a short but dense paper that contains a conjecture that quantifies and qualifies the observed geometric structure and variation of the attractor as parameters are varied. In particular, this paper not only addresses geometric variation along a 1-dimensional interval, but also on an open ball in parameter space. Moreover, many of the detailed arguments and issues raised in other related work are summarized in this paper.

Published in: PRE

Abstract: This report investigates the dynamical stability conjectures of Palis and Smale, and Pugh and Shub from the standpoint of numerical observation and lays the foundation for a stability conjecture. As the dimension of a dissipative dynamical system is increased, it is observed that the number of positive Lyapunov exponents increases monotonically, the Lyapunov exponents tend toward continuous change with respect to parameter variation, the number of observable periodic windows decreases (at least below numerical precision), and a subset of parameter space exists such that topological change is very common with small parameter perturbation. However, this seemingly inevitable topological variation is never catastrophic (the dynamic type is preserved) if the dimension of the system is high enough.

Position and status: This paper is a comprehensive analysis of the geometric variation in the chaotic portion of parameter space along a 1-dimensional interval. The term bifurcation chains is introduced in this work and various arguments are made regarding how the geometry varies using analysis of individual Lyapunov exponents and thus represents an analysis from a micro-geometric perspective. Many connections between results from abstract dynamical systems and the computational results for the space of neural networks relative to a measure are discussed at length in this work.

Published in: Nonlinearity

Abstract:Poincare had a vision to study nature by qualitatively analyzing function spaces. This paper opens with a discussion of a computational framework useful for attacking this problem by studying ensembles of mappings relative to a measure. Then, a macro-geometric analysis of an ensemble of high-dimensional, discrete-time, time-delay dynamical systems is performed using a function, M, defined by counting the number of positive Lyapunov exponents as parameters are varied. Conclusions include a quantification of the number of positive Lyapunov exponents preserved under parameter variation, the quantification of a type of chaos defined by the existence of bifurcation chains, and the persistence relative to a measure of a curve fit of M.

Position and status: In this paper, an analysis of the variation of the geometric structure deemed bifurcation chains along a 1-dimensional interval in parameter space is studied using a single diagnostic --- the number of positive Lyapunov exponents. Because only the number and not the magnitudes of the exponents are used, only a macro-picture of the dynamics is retained. Nevertheless, with this, the equivalence relation introduced in ``Persistent Chaos in High Dimensions,'' the number of positive exponents, can be calculated for an ensemble of dynamical systems and surprisingly, a universal scaling can be found relative to the number of dimensions and parameters. Given this universal scaling, many of the results of previous conjectures (including those in the previous listed paper) are simplified and compressed; many of the assumptions previously required for the existence of the bifurcation chains are shown to be unnecessary. This represents the next logical analysis after what is done in ``Structural Stability and Hyperbolicity Violation in High-Dimensional Dynamical Systems.''

Submitted to: