## INTRODUCTION

Designing a cognitive structure that acts spontaneously in a real-world setting is likely one of the final objectives within the area of cognitive robotics (1). A cognitive agent is anticipated to have autonomy, i.e., the agent ought to behave independently of the designer’s management whereas sustaining its identification. Adaptability is one other requirement for cognitive performance. Thus, the agent should choose the suitable conduct repeatedly and robustly in response to the altering setting in actual time. To summarize, the agent’s cognitive conduct ought to be applied via the body-environment interplay whereas nonetheless enabling the agent to take care of its autonomy and flexibility.

Within the standard context of robotics and synthetic intelligence, designers typically take top-down approaches to supply an agent with a hierarchical construction akin to the behavioral class. This representation-based strategy has a vital limitation within the design of a cognitive agent: The static, one-to-one relationship between the conduct and construction makes it troublesome to adapt flexibly to the dynamically altering setting and growing physique. For instance, it has been thought-about that the movement management methods of residing issues, together with people, understand their high-order movement plans by combining reproducible motor patterns known as movement primitives (2). Impressed by this viewpoint, we have a tendency to appreciate an agent’s conduct management with a predetermined static hierarchical construction. Nonetheless, such a hierarchical construction doesn’t exist in residing organisms from the start; fairly, these buildings are cultivated via the physique’s improvement and dynamic interactions with the setting. Subsequently, you will need to introduce a dynamical perspective to know a hierarchical construction of conduct management generated in animals which have adaptability and versatile plasticity.

In robotics, approaches primarily based on dynamical methods principle have been utilized to research and management brokers being modeled as units of variables and parameters on a part area (3, 4). This dynamical methods strategy can cope with each the useful hierarchy and the elementary movement in a unified type by expressing the bodily constraints of the agent because the temporal improvement of state variables, particularly, dynamics. For instance, Jaeger (4) sketched a pioneering thought of an algorithm the place the conduct of an agent is expressed as dynamics, and each the behavioral regularity (known as transient attractors) and the higher-order relationships amongst them are extracted in a bottom-up method. Not like the stochastic strategy the place the randomness of the system is realized by a probabilistic mannequin (e.g., the Markov mannequin), the dynamical system strategy can persistently categorical the agent’s seemingly random behaviors through its chaoticity (5). Moreover, within the stochastic strategy, hierarchical buildings are inevitably launched because the mechanism of random quantity era is totally unbiased of the system’s dynamics. Thus, the dynamical methods strategy has the potential to mannequin an agent’s spontaneous hierarchical conduct within the type of dynamic interplay with out top-down construction given by the exterior designer. The necessary difficulty right here is that the idea of the dynamical system strategy itself affords no basic precept for implementing these behaviors on numerous nonlinear couplings that represent the body-environment interplay. Subsequently, you will need to examine and suggest the methodologies for designing spontaneous hierarchical conduct with a constant temporal evolution rule governing the system’s dynamics.

Following this dynamical methods perspective, chaotic itinerancy (CI) (68) is a strong possibility for modeling spontaneous conduct with the useful hierarchy realized via the dynamics. CI is a regularly noticed, nonlinear phenomenon in high-dimensional dynamical methods, and it’s characterised by chaotically itinerant transitions amongst regionally contracting domains, particularly, quasi-attractors (8). Typically, a chaotic system has an preliminary sensitivity, and a slight distinction in part area is exponentially expanded in a sure route with temporal improvement. Conversely, a number of transiently predictable dynamics will be repeatedly noticed in a chaotic system, yielding CI regardless of the worldwide chaoticity and preliminary sensitivity. Such a hierarchical dynamics regularly emerges from high-dimensional chaos even with out hierarchical mechanisms, implying that specific construction just isn’t essentially wanted for implementing hierarchical behaviors. The chaoticity performs an necessary function in forming the autonomy of an agent as it’s nearly not possible for the designer to utterly predict and management an agent’s conduct due to the agent’s preliminary sensitivities, which basically ensures the agent’s independence from the designer. Thus, CI would work as an efficient software for implementing the mental conduct of a cognitive agent by embedding the conduct within the type of a quasi-attractor and sustaining the autonomy of the agent with the chaoticity.

CI was first present in a mannequin of optical turbulence (6). Since its discovery, related phenomena have been numerically obtained in numerous setups (7, 9, 10). The properties of CI range amongst earlier research, and a few courses of CI current attention-grabbing options which might be troublesome to characterize utilizing the stochastic processes. Tsuda et al. (9), for instance, present that their asynchronous neural community mannequin produces itinerant conduct whose transition frequency is characterised by a long-tailed distribution. The coupled map lattice proposed by Kaneko (7) doubtlessly emits infinitely many states, whose transition rule ought to be represented by an infinite state machine. A number of physiological research have reported that CI-like dynamics have even occurred in mind exercise, suggesting that CI would possibly play a vital function in forming cognitive capabilities (11, 12). For instance, Freeman (13) revealed that an irregular transition amongst discovered states was noticed within the electroencephalogram sample of a rabbit olfactory system when a novel enter was given, indicating that the cognitive situations akin to “I don’t know” are internally realized as CI-like dynamics. Moreover, a latest remark of rat auditory cortex cell exercise revealed the existence of a random shift amongst completely different stereotypical actions akin to particular person exterior stimuli throughout anesthesia (14). On the premise of those stories, Kurikawa and Kaneko (15) urged the novel thought of “memory-as-bifurcation” to know the mechanism of the transitory phenomenon. They reproduced it in an associative reminiscence mannequin wherein a number of input-output capabilities have been embedded by Hebbian studying. An intermittent switching amongst metastable patterns was additionally noticed in a recurrent neural community (RNN) by putting in a number of suggestions loops skilled to output a particular transient dynamics akin to exterior transient inputs (16). CI-like dynamics come up not solely in nervous methods but additionally in interactions between agent’s our bodies and their surrounding environments (1719).

For instance, Kuniyoshi and Sangawa (17) developed a human fetal improvement mannequin by coupling chaotic central sample turbines and a musculoskeletal system. They reported that a number of frequent behaviors, equivalent to crawling and rolling over, emerged from the bodily constraint. Subsequently, CI is a nonlinear phenomenon of high-dimensional dynamical methods and is believed to play a considerable function in producing structural conduct.

Impressed by the contribution of CI to the cognitive capabilities and spontaneous movement era of brokers, CI has been used for movement management within the area of neurorobotics and cognitive robotics by designing the CI trajectory. For instance, Namikawa et al. (2022) designed stochastic movement switching amongst predetermined movement primitives in a humanoid robotic by utilizing a hierarchical, deterministic RNN controller. On this examine, it was confirmed that lower-order RNNs with smaller time constants stably produced the trajectories of movement primitives, whereas higher-order RNNs with bigger time constants realized a pseudo-stochastic transition by exploiting self-organized chaoticity. Steingrube et al. (23) designed a robotic that skillfully broke a impasse state wherein the movement had utterly stopped by utilizing chaos within the RNN controller. Therefore, it may be interpreted that CI-like dynamics have been embedded within the coupling of the physique and the encompassing setting.

Whereas CI is a crucial phenomenon in high-dimensional dynamical methods, roboticists additionally discover it a useful gizmo for designing an agent’s conduct construction whereas sustaining the agent’s autonomy. Nonetheless, it has typically been troublesome to embed desired quasi-attractors at will due to their nonlinearity and excessive dimensionality. For instance, within the technique of Namikawa et al. (2022), the inner connections of an RNN ware skilled with backpropagation via time (24); nevertheless, embedding a long-term input-output perform in an RNN by the gradient descent technique is usually unstable and requires numerous studying epochs (25). Moreover, their technique required each a hierarchical construction and the identical variety of separated modules because the movement primitives, proscribing the scalability and the vary of its utility. Yamashita and Tani (26) proposed a community mannequin studying useful hierarchy with none modules, wherein hierarchical conduct was applied by the specific hierarchical construction ruled by predetermined a number of time scales. As well as, strategies utilizing the related reminiscence mannequin (10, 2729) are additionally unsuitable for our goal since it’s troublesome to embed the quasi-attractors with sophisticated spatiotemporal patterns.

On this examine, we suggest an algorithm, freely designing each the trajectories of quasi-attractors and transition guidelines amongst them in a setup of high-dimensional chaotic dynamical methods. We goal to design the properties of CI characterised by a finite state machine and finite switching time, equivalent to these within the neurorobotics context (19, 21). We put together transition guidelines described by a Markov mannequin and goal to emulate them via CI utilizing a high-dimensional nonlinear dynamical system. Our technique makes use of batch studying composed of the next three-step process (Fig. 1):

Step 1. Put together a high-dimensional chaotic system the place goal quasi-attractors are embedded. We used a broadly used echo state community (ESN) (30), one kind of RNN, as a high-dimensional chaotic system. This ESN accommodates no hierarchical construction and modules (e.g., a number of time scales), and each community node shares the identical time scale parameter. On the similar time, modify the interactions (inside parameters) in order that the system reproducibly generates intrinsic complicated trajectories generated by an preliminary chaotic system (innate trajectories) akin to the kind of the discrete inputs (named image). In parallel, prepare the linear regression mannequin (named readout) to output the designated trajectories (output dynamics) by exploiting the embedded innate trajectory. This course of will be doubtlessly utilized to the opposite chaotic dynamical methods not restricted to RNN in silico (31) since neither modules nor hierarchical buildings are required. As well as, this embedding course of is achieved by modifying fewer parameters utilizing the tactic of reservoir computing (32, 33). Subsequently, our scheme is extra secure and fewer computationally costly than standard strategies utilizing backpropagation to coach the community parameters.

Step 2. Add a suggestions classifier to the skilled chaotic methods for autonomously producing particular symbolic dynamics. Within the coaching of the suggestions discriminator, the community’s inside parameters are mounted, as with the readout in step 1. Thus, by utilizing the embedded innate trajectory, the suggestions discriminator achieves a number of image transition guidelines with minimal further computational capability (i.e., nonlinearity and reminiscence).

Step 3. Regulate the suggestions unit added in step 2 to design designated stochastic image transition guidelines. The deterministic system is anticipated to mimic the stochastic course of by utilizing intrinsic chaoticity. The system repeatedly generates the quasi-attractors embedded in step 1 in synchronization with the pseudo-stochastic image transition, that means that the design of the specified CI dynamics is accomplished.

On this examine, we reveal that the trajectories of quasi-attractors and their transition guidelines will be designed utilizing the three steps described above. In step 1, we present that the specified output dynamics will be designed with excessive operability by utilizing the embedded inside dynamics reproducibly generated after the innate coaching. Subsequent, in step 2, we reveal that numerous sorts of periodic symbolic sequences switching at a sure interval will be applied just by adjusting the parameters of a suggestions loop connected to the system. Final, in step 3, we put together a number of stochastic image transition guidelines ruled by a finite state machine and present that the system can simulate these stochastic dynamics by making use of the system’s chaoticity. We additionally talk about the proposed technique’s validity and flexibility via a number of numerical experiments.

## MATERIALS AND METHODS

### System structure

In our technique, we aimed to embed M sorts of quasi-attractors and the transition guidelines amongst them in an RNN. We ready M discrete symbols sS (S ≔ {s1, s2, ⋯ sM}). Every image corresponds to every quasi-attractor. We used an ESN as a high-dimensional chaotic system. As proven in Fig. 1A, we ready an RNN composed of a nonchaotic enter ESN (Nin nodes) working as an enter transient generator in addition to a chaotic ESN (Nch nodes) yielding chaotic dynamics. The dynamics of enter ESN xin(t) ∈ ℝNin and chaotic ESN xch(t) ∈ ℝNch are given as the next differential equations

(1)

(2)the place τ ∈ ℝ is a time fixed, tanh is an element-wise hyperbolic tangent, gin and gch ∈ ℝ are scaling parameters, uin(s) ∈ ℝNin is discrete enter projected onto enter ESN when image s is given, Jin ∈ ℝNin × Nin and Jch ∈ ℝNch × Nch are connection matrices, and Jic ∈ ℝNch × Nin is a feed-forward connection matrix between enter ESN and chaotic ESN. Every aspect of Jin is sampled from a traditional distribution

$N(0,1Nin)$

. Jch is a random sparse matrix with density p = 0.1 whose components are additionally sampled from a traditional distribution

$N(0,1pNch)$

. We used τ = 10.0, gin = 0.9, and gch = 1.5 to make enter ESN nonchaotic and chaotic ESN chaotic (34). As well as, to stop chaotic ESN from changing into nonchaotic due to the bifurcation brought on by the sturdy bias time period, we tuned Jic earlier than hand to undertaking transient dynamics converging to 0 onto the chaotic ESN when the identical symbolic enter continues to be given (see the Supplementary Supplies for detailed details about the transient dynamics). In any case, the entire RNN dynamics x(t) ∈ ℝNin + Nch concatenating Eqs. 1 and 2 will be represented by the next single equation (⊙ represents an elementwise product)

(3)the place x, g, J, and u are outlined by the next equations

$x(t)≔[xin(t);xch(t)]$

(4)

$g≔[gin,⋯gin⏟Ningch,⋯gch⏟Nch]T$

(5)

$J≔[Jin0JicJch]$

(6)

$u(s)≔[uin(s);0]$

(7)

The output dynamics are calculated by the linear transformation of the inner dynamics x(t), that’s, the linear readout wout ∈ ℝNin + Nch is skilled to approximate the next goal dynamics fout(t)

$woutTx(t)≈fout(t)$

(8)

The symbolic dynamics s(t) itself, which is externally given in step 1, is lastly generated autonomously with a closed-loop system [Fig. 1B(2)]. Within the suggestions loop, the next classifier fmax : ℝNin + NchS is connected

(9)the place ws ∈ ℝ(Nin + Nch) × M represents the connection matrix whose components are skilled to autonomously emulate the designated symbolic dynamics s(t) [i.e., s(t + Δt) ≈ fmax(x(t)), where s(t + Δt) is the symbolic input for the next time step and Δt is a time width for discrete temporal evolution]. To summarize, we designed the specified quasi-attractors, output dynamics, and symbolic dynamics by tuning the parameters of the RNN connections J, the readout wout, and the classifier ws, respectively.

### First order–decreased and controlled-error studying and innate coaching

We used two reservoir computing strategies known as first order–decreased and controlled-error (FORCE) studying (35) and innate coaching (36). Each FORCE studying and innate coaching are strategies that harness the chaoticity of the system. Under, we briefly describe the algorithms of each FORCE studying and innate coaching.

FORCE studying is a technique that embeds designated dynamics in a system by harnessing the chaoticity of dynamical methods. Suppose the next ESN dynamics with a single suggestions loop

(10)

$z(t)=wTx(t)$

(11)the place u represents the linear suggestions vector. Sometimes, the scaling parameter g is ready to be better than 1 to make the entire system chaotic (34). In FORCE studying, to embed the goal dynamics f(t) within the system, w is skilled to optimize the next price perform CFORCE

$CFORCE≔〈∥z(t)−f(t)∥〉2$

(12)

Right here, the bracket denotes the averaged worth over a number of samples and trials. Specifically, within the FORCE studying, w is optimized on-line with a least-square error algorithm. It was reported from numerical experiments utilizing ESN that higher coaching efficiency was obtained when the preliminary RNN was in a chaotic regime (35).

Innate coaching can be a scheme for harnessing chaotic dynamics and is achieved by modifying the inner connection J utilizing FORCE studying. The novel facet of innate coaching is that the internal connection of ESN is skilled in a semisupervised method, that’s, the connection matrix J of the ESN is modified to attenuate the next price perform Cinnate to breed the chaotic dynamics yielded by the preliminary chaotic RNN (xgoal(t), innate trajectory)

$Cinnate≔〈∥x(t)−xgoal(t)∥〉2$

(13)

Intriguingly, the innate trajectory is reproducibly generated for a sure interval with the enter whereas sustaining the chaoticity after the coaching. In different phrases, innate coaching is a technique that permits a chaotic system to reproducibly yield the innate trajectory with sophisticated spatiotemporal patterns. As well as, innate coaching applies the FORCE studying technique to the modifications of the inner connection, that’s, the presynaptic connection of a node within the community is taken into account because the linear weight from the opposite nodes and skilled by FORCE studying. On this examine, we suggest a technique of designing CI by utilizing each FORCE studying and innate coaching strategies.

### Recipe for designing CI

Our proposed technique is a batch-learning scheme consisting of the next three-step course of (Fig. 1C).

Step 1. Designing quasi-attractor. In step 1, the connection matrix Jch of the chaotic ESN is adjusted by innate coaching to design the trajectories of quasi-attractors. First, the goal trajectories

$xgoals(t)$

are recorded for M symbols underneath an preliminary connection matrix Jinit and a few preliminary states

$xgoals(0)$

, the place

$xgoals(t)$

denotes chaotic dynamics when the image is switched to s at t = 0 ms (for simplification, the switching time is mounted to t = 0 ms in step 1; notice that the image will be switched at any time). In step 1, Jch is skilled to optimize the next price perform C1−in

$C1−in≔Σs∈SΣt=0Linnate∥xs(t)−xgoals(t)∥2$

(14)

Right here, xs(t) represents the dynamics when the image is switched to s at t = 0 ms, and Linnate represents the time interval of the goal trajectory. We randomly select half the community nodes (Nch/2 nodes) and modify their presynaptic connections to scale back the redundancy of the coaching parameter. The chosen components in connection matrix Jch are skilled for 200 epochs for every s. We lastly use Jch recording the minimal C1 − in (see the Supplementary Supplies for the detailed algorithm utilized in step 1). After the innate coaching in step 1, the system is anticipated to breed the recorded innate trajectories

$xgoals$

for Linnate.

Though there are not any particular standards for figuring out the preliminary states of the a number of innate trajectories, the big distances amongst

$xgoals(0)$

are most well-liked because the temporal sample of quasi-attractors is more likely to differ, enhancing the separability. Furthermore, the necessary trick of the innate coaching lies in its semisupervised scheme, that’s, the coaching stability will increase by guiding the offset states throughout coaching to the neighborhood of

$xgoals(0)$

. Though a scheme for coaching the inner connections of the RNN has already been proposed in FORCE studying (35), the tuning of all connections to generate the identical perform is generally unstable (37). Subsequently, we randomly chosen the offset state of the innate trajectory

$xgoals(0)$

on the part area.

Equally, wout is skilled to supply designated output dynamics fs(t) akin to image s. The next price perform C1−out is optimized

$C1−out≔Σs∈SΣ0Lout∥fs(t)−woutTxs(t)∥2$

(15)

Excessive-dimensional nonlinear dynamical methods typically have excessive separability for enter info, that’s, it turns into simpler for a linear mannequin to resolve nonlinear input-output perform duties by projecting enter info into the system (38). Specifically, the innate trajectories of chaotic methods are recognized to have such excessive expressive functionality that numerous orbits will be designed just by adjustment of the connected linear mannequin (36). On this examine, the tuned readout can be anticipated to stably reproduce the ready trajectory by exploiting the excessive dimensionality and nonlinearity of the innate trajectories. Right here, notice that Linnate doesn’t all the time match Lout, that’s, Lout will be better than Linnate. The coaching is achieved by an offline algorithm Ridge regression primarily based on the recorded inside dynamics xs(t).

Step 2. Embedding autonomous transitions of image. In step 2, we tune a suggestions loop fmax to realize the autonomous image transition. We particularly put together goal periodic transition guidelines switching each T (ms). Suppose a goal periodic symbolic time sequence sper(t). First, the community dynamics x(t) of the open-loop setup [Fig. 1B(1)] is recorded with a symbolic dynamics sper(t) for Trec ≔ 500,000 ms. On the premise of the recorded dataset, fmax is tuned to output the following symbolic enter sper(t + Δt) from x(t). The parameters ws of fmax is skilled to optimize the next price perform C2

$C2≔−Σs∈SΣ0Trec1{sper(t+Δt)=s}logewsTx(t)Σok∈SewokTx(t)$

(16)

Because the optimization algorithm, we use the limited-memory Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm (39).

This optimization activity is taken into account to be a kind of timer activity, a generally used benchmark activity to guage the temporal computational capability, the place the readout is skilled to output a pulse-like wave with a sure delay after enter is given. By projecting the enter sign right into a high-dimensional nonlinear dynamical system, the timer activity will be achieved just by adjusting the linear readout. As well as, the complicated trajectory embedded by the innate coaching considerably will increase the efficiency of the timer activity in contrast with a nonchaotic random ESN (36). In our mannequin, the tuned classifier fmax is anticipated to emulate the delayed symbolic switching by exploiting the embedded innate trajectory.

Step 3. Embedding stochastic transitions of image. In step 3, we implement a stochastic transition rule ruled by a finite state machine by modifying a suggestions loop fmax. As mentioned in Introduction, the chaoticity of the system is anticipated for use to emulate the stochastic course of within the deterministic setup. We ready the goal stochastic time sequence ssto(t) generated from a Markov mannequin with sure switching intervals. The method of the training is similar as that in step 2, that’s, the pair of (x(t), ssto(t)) recorded within the open-loop setup for 500,000 ms is used to coach the fmax to emulate ssto(t). Right here, we use the next price perform C3 within the coaching

$C3≔−Σs∈SΣ0Trec1{ssto(t+Δt)=s}logewsTx(t)Σok∈SewokTx(t)$

(17)

As with the optimization of the associated fee perform C2, C3 is optimized with the limited-memory BFGS algorithm.

Notice that the formulation of C3 is identical as that of C2, that’s, the properties of goal dynamics to be embedded should not expressed in the associated fee perform formulation. Reasonably, each optimization processes in steps 2 and three are data-driven, that means that the properties of ready goal symbolic sequences decide whether or not the embedded dynamics are required to be chaotic or nonchaotic.

## RESULTS

On this part, we present the demonstration and analytic outcomes of the numerical experiments for every step.

### Step 1. Designing quasi-attractor

As mentioned within the earlier part, the inner connection of the chaotic ESN Jch is skilled to reproducibly output the corresponding innate trajectories to the symbolic switching. Figure 2A demonstrates the change of the community dynamics of a 1500-node RNN (Nin = 500, Nch = 1000) whose connection matrix is modified with innate coaching underneath the situation (M, Linnate) = (1,1000). The trajectory rapidly spreads earlier than t = Linnate within the pretrained system, whereas the goal trajectory

$xsgoal$

(dotted line) is reproducibly yielded for 1000 ms (lined by the yellow rectangle) within the posttrained system. Furthermore, intriguingly, the dispersion of the trajectories continues to be suppressed even after t = Linnate.

Subsequent, Fig. 2B shows each the community dynamics and the output dynamics. The 1500-node RNN (Nin = 500, Nch = 1000) skilled underneath the situation (M, Linnate) = (3,1000) was used. At first, the symbolic enter was absent, after which symbols have been switched with random intervals from the center. As well as, the two-dimensional readout was skilled to output the Lissajous curve for image A, the “at” signal for image B, and the xz coordinates of the Lorenz attractor for image C for Lout = 1500 ms (the goal trajectory for the “at” signal was constituted of a centerline of font knowledge). It was noticed that the specified spatiotemporal patterns have been stably and reproducibly generated for a sure interval in each trajectory with completely different preliminary states after the image transition (see film S1). Notice that the identical linear mannequin wout was used within the demonstration, implying that the trajectory of every quasi-attractor has wealthy sufficient info to independently output the designated time-series patterns even with the only linear regressor. Our scheme for designing transient dynamics could be extremely helpful within the area of robotics as a result of the method in step 1 is definitely achieved by adjusting the partial components of a high-dimensional chaotic system. For instance, the system working in a real-world setting ought to instantly and adaptively change its movement based on the change of environmental enter like a system developed by Ijspeert et al. (40), which will be simply achieved by our computationally low cost technique. On this method, our technique would work successfully within the context of robotics, the place quick responsiveness and flexibility are required.

We additionally examined each the scalability and the validity of innate coaching intimately via a number of numerical experiments (Fig. 3). First, we examined the connection between the variety of enter symbols M and the accuracy of innate coaching. To guage the efficiency of innate coaching, we used the normalized imply sq. error (NMSE) between the output and the innate trajectory

$xgoals$

represented by the next method

$NMSE≔1MΣs∈S⟨Σ0Linnate∥xs(t)−xgoals(t)∥2Σ0Linnate∥xgoals(t)∥2⟩$

(18)the place the bracket represents the common over 10 trials for every image. We calculated the NMSE for 10 trials. Figure 3A exhibits the innate coaching performances with the completely different coaching situations, suggesting that NMSEs usually tend to improve with an extended goal trajectory and a bigger variety of symbols. This outcome implies that innate coaching has its limitation within the design of the quasi-attractors. We additionally examined the impact of community measurement on the potential to embed the quasi-attractors. We investigated the connection between the variety of nodes within the chaotic ESN Nc and the accuracy of innate coaching underneath the situation M = 1 (Fig. 2B), suggesting that the NMSEs have been much less more likely to improve with a bigger community. To summarize, our evaluation signifies that longer trajectories will be embedded in a bigger community by innate coaching.

Subsequent, we evaluated the impact of innate coaching on the capability of the system’s info processing. We ready a timer activity and measured how lengthy the inputted info was saved within the RNN. Within the timer activity, the pulse-like wave with a peak tpeak (ms) after the image transition was ready because the goal, and the efficiency was outlined because the accuracy of the pulse-like wave reconstruction by a skilled readout. Right here, we outlined the coefficient of dedication worth R2 between the output and the pulse-like wave because the timer activity perform R2(tpeak). On the similar time, we additionally calculated the integral worth of the timer activity perform

$∫0∞R2(t)dt$

and outline it because the timer activity capability (see the Supplementary Supplies for detailed details about the setup of the timer activity). Figure 4C exhibits the timer activity perform with completely different innate coaching situations, indicating that RNNs skilled with the longer-length goal trajectory Linnate carry out higher. It was additionally noticed that the timer activity capability saturated round Linnate = 5000 ms within the 1000-node RNN, and the border of the saturation decreased in a smaller system (Fig. 3D). These outcomes indicate that the temporal info capability of the system is improved by innate coaching with the longer goal size Linnate however saturates at a sure worth, which is set by the system measurement.

Moreover, we assessed the impact of innate coaching on the system’s chaoticity by measuring the Lyapunov exponents of the system. Because the transition amongst quasi-attractors is pushed by the system’s chaoticity, it’s essential to hold the system chaotic. On this experiment, we measured the native Lyapunov exponent (LLE) to guage the diploma of trajectory variation after the symbolic switching. We additionally measured the utmost Lyapunov exponent (MLE) with none inputs (u(t) = 0) to estimate the worldwide chaoticity of the system (see the Supplementary Supplies for the detailed calculation algorithm of each the LLE and MLE). Figure 3E shows the LLE values of the methods with the completely different goal trajectory size Linnate, suggesting that the trajectories inconsistently broaden after the image transition. Specifically, it was noticed from the LLE evaluation that contracting areas existed (areas with detrimental LLEs akin to the lengths of the quasi-attractors) brought on by the transient dynamics projected by the enter ESN, and the diploma of the growth turned gradual within the skilled interval t ∈ [0, Linnate). These results imply that innate training yields a locally contractive phase space structure, that is, a quasi-attractor. Moreover, positive MLE values were constantly obtained from the MLE analysis depicted in Fig. 3F, supporting the conjecture that the system chaoticity was maintained especially well with the larger RNNs even after the innate training. (Note that a sharp increase in MLE was observed with shorter Linnate, which is caused by the increase in the spectral radius of the connection matrix J of the system. See the Supplementary Materials for detailed information of the analysis.)

### Step 2. Periodic symbol transition

In step 2, the system autonomously generates a symbolic sequence externally given in step 1. The additional feedback loop realizes the autonomous periodic switching of the symbols. We demonstrate that various types of periodic symbolic sequences switching at a fixed interval can be easily designed simply by tuning the parameter of the feedback loop fmax. Figure 4A demonstrates the embedding of the periodic symbolic sequence A-B-C (2000-ms interval and 6000-ms period) with a trained RNN ((M, Linnate) = (3,1000)). Figure 4B also exhibits the embedding of the periodic symbolic sequence A-B-C-D-E-F-G-H-I-J (500-ms interval and 5000-ms period), with the same RNN used as the demonstration in Fig. 4A (note that fmax was changed from one in Fig. 4A). In both demonstrations, the system succeeded not only in generating the desired symbol transition rules but also in stably outputting the designated output dynamics with high accuracy.

We also show that the system can solve tasks requiring higher-order memory in the same scheme. We prepared the two periodic symbolic sequences A-B-C-B and A-B-C-B-A and separately trained fmax. These two symbolic sequences are more difficult to embed because the system must change the output according to the previous output. In the symbol transition A-B-C-B, for example, the system must output the next symbol depending on the previous symbol when switching from B, though the total number of symbols is the same as in the task A-B-C. We used the same RNN and setup used in the Fig. 4A and only changed the parameters in fmax to realize the symbol transitions. Figure 4C displays the network dynamics and symbol transition of the two tasks, showing that the system successfully achieves both the periodic sequence A-B-C-B with an 8000-ms period and A-B-A-B-C with a 10,000-ms period. These results suggest that the trained RNN had the higher-order memory capacity, that is, the generated trajectories have sufficient separability to distinguish the contextual situation depending on the previous symbol sequence (see movie S2). In robotics, periodic motion control has often been implemented by an additional oscillator (e.g., a central pattern generator) to yield limit cycles (23, 4042). Our method in step 2 would be useful in designing limit cycles with longer periods and more complicated patterns. The analysis in fig. S2 shows that our method outperforms FORCE learning in the embedding of a long-term periodic attractor including multiple transitions in order (see the Supplementary Materials and fig. S2).

We also analyzed the effect of perturbation to investigate the stability of the embedded symbol transition. Figure 4D shows the output dynamics of both the original and perturbed trajectories, clarifying that the trajectory returned to the original one after the addition of the perturbation. We also calculated the MLE values of the system and obtained the value −1.89 × 10−4, which was very close to zero. These analyses indicate that the trained feedback loop fmax made the system nonchaotic, that is, the generated internal dynamics was a limit cycle.

### Step 3. Stochastic symbol transition (CI)

In step 1, we constructed the trajectories of the quasi-attractors and the corresponding output dynamics. In step 2, we showed that periodic transitions among quasi-attractors can be freely designed by simply tuning the feedback loop fmax. In step 3, we realize a stochastic transition, that is, CI. As discussed above, the system is expected to use its chaoticity to emulate a stochastic transition in deterministic dynamical systems.

First, we demonstrate that stochastic transition can be freely designed by adjusting fmax (see Fig. 5A and movie S3). In this demonstration, we used the same RNN as in Fig. 4A. We prepared a symbol transition rule uniformly switching among symbols A, B, and C at 3000-ms intervals. Figure 5B shows the symbolic dynamics, network dynamics, and output dynamics, suggesting that the symbol transitions started to spread at around t = 10,000 ms and lastly settle down to completely different transition patterns. Nevertheless, the system continued to stably generate Lissajous curves. These demonstrations imply that the system constantly reproduced quasi-attractors embedded by innate training, while the quasi-stochastic transition was achieved by the global chaoticity. [Note that, although we demonstrated our approach by embedding typical stochastic processes (i.e., Markov processes) to illustrate the usability of our scheme, our method can also design a history-dependent stochastic rule that cannot be represented by a Markov model. See our demonstration represented in the Supplementary Materials and fig. S3.]

To investigate the pliability of our technique, we measured the stochastic transition matrix and the common symbolic intervals (Fig. 5B). We ready two stochastic image transition guidelines because the targets: the transition rule ruled by the uniform finite state machine (sample 1) and the transition rule ruled by the finite state machine with a restricted transition (sample 2). Notice that we used the identical skilled RNN as within the demonstration in step 2 and embedded the transition guidelines just by adjusting fmax. Figure 5B exhibits the outcomes of the obtained trajectories, implying that the system efficiently embedded patterns much like the goal guidelines, though there have been some errors and variations within the transition likelihood and the switching time. The constructive MLEs have been obtained in each instances (+2.01 × 10−3 in sample 1 and +1.71 × 10−3 in sample 2), suggesting that the system was weakly chaotic as an entire. As well as, we analyzed the historical past dependence of the transition intimately, displaying that transition chances didn’t differ a lot based on the previous image in each instances although the popular routes have been noticed in output dynamics (see the Supplementary Supplies and fig. S4). On this sense, it may be stated that the system efficiently expressed the random transitions in a macroscopic scale (i.e., a scale in image transitions) utilizing the chaoticity.

Final, we analyzed each the buildings of the obtained chaotic attractors and the symbolic dynamics intimately. Figure 6A exhibits the impact of small perturbations on the symbolic dynamics, implying that the patterns of symbolic dynamics various after a sure interval. To investigate the structural change of the terminal symbolic state, we measured the symbolic dynamics accompanied by the temporal improvement of the set of preliminary states on a airplane constructed by the 2 chosen dimensions (Fig. 6B), clarifying {that a} complicated terminal symbolic construction emerges after a sure interval (Fig. 6B). Specifically, within the embedding of the sample 1 rule, the entropy of the terminal symbolic sample converges to a worth near the utmost entropy

$log239≈14.26$

(notice that the entropy was measured on the premise of the likelihood distribution constructed by the frequency of three × 3 grid patterns). These outcomes point out that the image transition markedly modified even with a small perturbation and was unpredictable after a sure interval, that’s, the prediction of symbolic dynamics required the entire remark of the preliminary state worth and calculation of the temporal improvement with infinite precision.

## DISCUSSION

On this examine, we proposed a technique of designing CI primarily based on reservoir computing strategies. We additionally confirmed that the assorted sorts of output dynamics and image transition guidelines could possibly be designed with excessive operability just by adjusting the partial parameters of a chaotic system with our three-step recipe. On this part, we first talk about the scalability of our technique and the mechanism of how CIs are efficiently embedded by reviewing a number of numerical analyses that confirm the validity of our strategies. Subsequent, we talk about the effectiveness and significance of our technique from a number of viewpoints.

### Scalability and validity

First, the outcomes of the innate coaching performances displayed in Fig. 3 (A and B) point out that the variety of RNN nodes constrains the overall size of the quasi-attractors that may be embedded within the system by the innate coaching. Nonetheless, the LLE analyses in Fig. 3E present that the system has the expanded area of the detrimental LLE even when the NMSE between the innate trajectory and the embedded trajectory turns into massive (e.g., Linnate = 5000 ms). These outcomes indicate that, even when the innate trajectories should not efficiently embedded within the system, the system stably yields high-dimensional trajectories with sophisticated spatiotemporal patterns for every image transition over Linnate, which is brought on by the weakening of the system chaoticity. The identical RNN skilled underneath the situation (M, Linnate) = (3,1000) was repeatedly utilized in our sequence of demonstrations, the specified output dynamics (e.g., the Lissajous curves) being continuously generated for Lout = 1500-ms intervals after the symbolic shift (Figs. 2, 4, and 5). We additionally demonstrated that the system can autonomously generate an emblem transition rule with an interval better than Linnate (Figs. 4 and 5), suggesting that the system exploited high-dimensional reproducible trajectories longer than Linnate.

Furthermore, it’s assumed that the size of the quasi-attractors constrains the goal stochastic transition guidelines that may be embedded. The system didn’t imitate the stochastic transition, and the transition turned periodic when the goal transition had a shorter switching interval, whereas the coaching of fmax turned unstable when it had an extended switching interval. These outcomes counsel that the next two mechanisms ought to be required within the design of CI in our technique: (i) The variations among the many trajectories are sufficiently enlarged via the temporal improvement to appreciate the stochastic image transition and (ii) an analogous spatiotemporal sample ought to be reproducibly yielded till the switching second to exactly discriminate the switching timing. These two mechanisms are contradictory, in fact, and the specified CI can probably be embedded when each situations are reasonably happy.

Subsequent, we talk about the validity of CI designed with our technique via comparability with earlier CI research. We demonstrated that the embedded transition yields transition likelihood much like the specified one by utilizing the chaoticity of the community. A number of CI works, nevertheless, identified that transition of CI is historical past dependent, and due to this fact, CI exhibits some desire within the transition decided by the historical past. Kaneko and Tsuda (43), for instance, identified that the transition of CI has a particular order that’s distinguished from a easy random hopping. Itoh and Kimoto (44) additionally reported that the transition presents a most well-liked route ruled by the worldwide part area dynamics. Our experiment indicated that embedded CI additionally exhibits most well-liked trajectories in output dynamics based on the earlier image earlier than the switching (see the Supplementary Supplies and fig. S4A). Subsequently, the symbolic dynamics applied with our strategies ought to be historical past dependent.

Furthermore, it ought to be famous that our proposed technique can efficiently emulate the randomness of the image transition regardless of the existence of the popular trajectory. Our evaluation revealed that the transition chances didn’t differ a lot based on the distinction within the earlier image in each demonstrations (i.e., stochastic guidelines 1 and a pair of), implying that prediction of the following transition remains to be not straightforward if finished solely by referring to the macroscopic symbolic historical past (see the Supplementary Supplies and fig. S4B). On this method, macroscopic image transitions nonetheless seem like random, regardless of the trajectory desire. Thus, it may be stated that our proposed technique emulates the transition likelihood and its randomness on a high-dimensional nonlinear dynamical system by utilizing the chaoticity of the system.

We present that our technique can understand CI characterised by the random transition of a finite variety of quasi-attractors as proven in (19, 21). Some courses of CI, nevertheless, are troublesome to design even with our technique. There exists a CI whose transition frequency has a long-tailed distribution (9). As well as, CI can have infinitely many quasi-attractors whose dynamics ought to be described with an infinite state machine (7). These CI properties are laborious to design with our technique since an infinite variety of auxiliary symbols and an infinite size of corresponding trajectories ought to be ready, which ought to be solved sooner or later.

### Effectiveness and significance

First, the excessive operability of our proposed mannequin could be useful to know the underlying mechanism of mind’s info processing from a sure perspective. Earlier research have identified that an unlimited variety of nonlinear items and their interplay basically represent an animal’s nervous system and yields extremely sophisticated actions characterised by nonlinear phenomena equivalent to chaos and CI. It has been reported that chaotic conduct seems in a variety of mind actions from the cell degree (e.g., motion potential) to the worldwide measurement degree (e.g., electroencephalogram) (46). As well as, it has been identified that the collective neural actions not solely encode the exterior info but additionally rework it based on the historical past of actions (47, 48), suggesting that the animal mind realizes its info processing via the high-dimensional actions. We discovered that high-dimensional chaos has sufficient wealthy expressive functionality to design CI, implying that the high-dimensional chaotic mind actions doubtlessly have the potential to appreciate numerous useful hierarchies. On this sense, our mannequin would supply a clue to know the mechanism of how high-dimensional chaos that contributes to the data processing in animal brains.

The designing technique for the CI exhibited on this examine affords basically completely different advantages in contrast with the earlier strategies, which merely exploit chaotic dynamics. The dynamics of CI presents an attention-grabbing property and counsel that native coherence and international chaoticity coexist. This property of CI would successfully work particularly in designing cognitive fashions the place each autonomy and spontaneity are required. For instance, a designer can implement the movement primitives via the quasi-attractor whereas sustaining the autonomy of the robotic via the worldwide unpredictability. Moreover, our algorithm can design the likelihood distribution, that’s, the worldwide tendency of conduct, as proven in Fig. 5. This coexistence is, nevertheless, troublesome to precise when utilizing solely the traditional chaotic attractors. As well as, a number of research identified that animals’ cognitive capabilities, equivalent to reminiscence recall and affiliation, could be realized via the transition phenomenon amongst stereotypical actions (13, 14), suggesting that CI would possibly play an necessary function in animal cognition. On this sense, our technique could be used for implementing cognitive fashions in a high-dimensional dynamical system.

Though a number of research have used chaotic dynamical methods to embed desired trajectories (35, 36), these standard strategies are incapable of mixing a number of predetermined transient dynamics like our technique. It may also be potential to merge a number of transients into one large attractor and embed it concurrently by a studying scheme equivalent to FORCE studying. Nonetheless, the extra experiment exhibits that FORCE studying is extra unstable than our strategies with regards to embedding long-term periodic trajectories consisting of a number of transients (see the Supplementary Supplies and fig. S2). On this sense, excessive operability in our technique is unavailable within the standard strategies.

Not like earlier strategies that assemble desired trajectories by tuning the whole dynamics by backpropagation algorithm, our technique is achieved by adjusting the decreased variety of parameters and utilizing the intrinsic high-dimensional chaos, which alleviates the organic implausibility and computational complexity of backpropagation algorithms. For instance, latest physiological research on the motor cortex (49, 50) counsel that a big number of behaviors will be instantaneously generated by the partial plasticity of the nervous system, supporting the organic plausibility of our studying scheme. As well as, our studying scheme is computationally cheaper than backpropagation since changes of whole neural circuits should not essential. These properties could be particularly useful within the context of bioinspired robotics, the place quick responsiveness and real-time processing are required.

One other benefit of our technique is that it doesn’t require the specific construction of dynamical methods. For instance, within the technique proposed by Namikawa and Tani (2022), the controller wants a set hierarchical construction and modularity. Subsequently, the skilled controller was specialised in implementing a particular conduct, making it troublesome to divert it for some other goal. As well as, it could be potential to design CI-like dynamics in an structure the place the symbolic sequence and the corresponding trajectory are individually generated, which requires an exterior mechanism to carry the image and wait till the era of lower-order trajectory finishes. Subsequently, the separation of the symbolic sequence mannequin and trajectory encoder implicitly makes use of the hierarchical construction and can’t be realized by a high-dimensional chaos alone. In distinction, we proposed a technique of designing CI with a setup consisting of a single chaotic ESN, auxiliary symbols, and an interface between them (enter ESN) with excessive scalability. Furthermore, the modifications of inside connections within the chaotic ESN will be realized by including a number of linear suggestions loops and coaching them with the FORCE studying because the presynaptic connection within the chaotic ESN will be thought to be a linear connection. Thus, our technique permits us to design the assorted trajectories and their transition guidelines in a constant high-dimensional chaotic system, thereby significantly increasing the scope of utility of high-dimensional chaotic dynamical methods. Neuromorphic units primarily based on bodily reservoir computing frameworks could be a wonderful candidate for implementing our scheme (31). Sprintronics units, for instance, have lately been proven to exhibit chaotic dynamics (51, 52) and are actively exploited as bodily reservoirs (5355). We count on that this framework would supply one of many promising utility situations for real-world implementations of our scheme.

Our technique can use a priori data via the introduction of auxiliary symbols. A number of neurorobotics frameworks have been proposed up to now wherein symbolic dynamics are self-organized on the community by end-to-end studying (26, 56). Though these strategies are handy since they don’t require specific a priori data, they can not actively use the prior symbolic construction, and thus, symbolic construction solely seems after the coaching. In distinction, we confirmed that the training efficiency is significantly improved by auxiliary symbols (see the Supplementary Supplies and fig. S3). In that sense, our technique has a bonus over the traditional end-to-end scheme.

Our technique can be scalable to autonomous image era required in additional superior performance. For instance, in our technique, M sorts of auxiliary symbols are given as a priori data. Nonetheless, in a extremely autonomous system, equivalent to people, symbols are dynamically generated and destroyed due to developmental processes. As demonstrated by Kuniyoshi and Sangawa (17), these self-organizing symbolic dynamics will be realized by offering an extra computerized labeling mechanism within the system. As well as, unsupervised algorithms for extracting discrete symbols from the dynamics just like the one launched in (5759) will be integrated into our system. In different phrases, it’s potential to spontaneously generate symbols by embedding an unsupervised studying algorithm within the system; it is a topic for future work.

Final, the dynamic phenomena obtained by our technique are important from the perspective of high-dimensional dynamical methods. As proven in Fig. 6, we demonstrated that small variations within the preliminary community state have been expanded by the chaoticity of the system, which finally led to drastic change in each the worldwide image transition sample s(t) and the native dynamics x(t). Such tight interplay between microlayer and macrolayer is a phenomenon distinctive to deterministic dynamical methods; that’s, it can’t happen, in precept, in a system the place the higher-order mechanism is totally separated from the lower-order one (e.g., unbiased random variables). As well as, the worldwide traits of dynamical methods are sometimes analyzed by the mean-field principle. Nonetheless, the evaluation by the mean-field approximation can’t seize the contribution of microscopic dynamics to the macroscopic change. Thus, our CI design technique has a significant function in shedding gentle on the interplay between micro- and macrodynamics in deterministic chaotic dynamical methods.

Acknowledgments: This work was primarily based on outcomes obtained from a undertaking commissioned by the New Power and Industrial Expertise Improvement Group (NEDO). Funding: Ok.I. was supported by JSPS KAKENHI (grant quantity JP20J12815). Ok.N. was supported by JSPS KAKENHI (grant quantity JP18H05472) and by MEXT Quantum Leap Flagship Program (MEXT Q-LEAP) (grant quantity JPMXS0120319794). This work was supported by NEDO [serial numbers 15101156-0 (dated 24 June 2016) and 18101806-0 (dated 5 September 2018)] and Chair for Frontier AI Training, Faculty of Data Science and Expertise and Subsequent Era AI Analysis Heart [serial number not applicable (dated 1 June 2016)]. Writer contributions: Ok.I. and Ok.N. conceived the concept and designed the experiments. Ok.I. carried out the experiments and created the demonstration. Ok.I. and Ok.N. wrote the manuscript. All authors mentioned and commented on the manuscript. Ok.N. and Y.Ok. directed the undertaking. Competing pursuits: The authors declare that they haven’t any competing pursuits. Information and supplies availability: All knowledge wanted to guage the conclusions within the paper are current within the paper and/or the Supplementary Supplies. Extra knowledge associated to this paper could also be requested from the authors.