Viewpoint

, Volume: 11( 1) DOI: 10.37532/2319-9822.2022.11(1).195

• View PDF
• Download PDF

Exploring Earth-Moon Halo Orbit Transfers Multi-Objective Reinforcement Learning (MORL)

*Correspondence:

Filip WalterManaging Editor, Journal of Space Exploration,United Kingdom ; E-mail: walter.F@gmail.com

Abstract

Introduction

The goal of this collection is to handle cislunar space's dynamical difficulties. The Earth-Moon system may be thought of as a dynamic laboratory, characterized by unusual characteristics. On the one hand, we can see the large mass ratio between the two. The bulk of the Moon, its orbit, its closeness to the Sun, and the chaotic nature of its surface. The dynamics of tiny entities and their behavior (artificial and non-artificial) on the contrary, in the cislunar location, critical tests are going on. To begin with, it is well acknowledged that a concerted attempt to investigate the lunar surface with both unscrewed and crewed missions, as well as to study the moon's interior to maintain a Lunar Gateway.

Low-thrust trajectories are solutions to a multi-objective, high-dimensional optimization problem. Furthermore, hardware specifications, mission objectives, mission limits, deployment circumstances, power limitations, and other operational requirements all have an impact on the availability and attributes of possible solutions. Understanding the design space of low-thrust trajectories for SmallSats as the spacecraft and mission parameters grow allows for trade-offs between numerous goals, such as lowering divergence from a reference trajectory and limiting propellant mass needs over a tolerable flight length. The accompanying Pareto front, which represents the set of no dominated solutions, is frequently used to study the solutions to a multi-objective issue. Developing point solutions that fall into the multi-objective solution space is one step toward discovering the global Pareto front.

These point solutions provide broad insights into the trajectory design space, and they may be used to construct locally optimal or even globally optimal solutions. However, given the complicated gravitational environment of cislunar space, creating one plausible, low-thrust route may be time-consuming and computationally costly.

Machine learning is a good contender for providing solutions in multi-modal, high-dimensional solution spaces. Deep Reinforcement Learning (DRL) has been more popular among mission designers for recovering difficult solutions in high-dimensional design situations. Multi-objective deep reinforcement learning algorithms provide a fast way to find various solutions with different geometries and goal prioritizations while using fewer computing resources and time.

Using MRPPO, numerous policies are simultaneously taught, each with different weights scaling the conflicting objectives. Each policy is taught to develop a unique control scheme based on the reward function assigned to it: a weighted mixture of objectives that direct the spacecraft to the target mission orbit minimize deviations from a reference trajectory and punish propellant mass utilization.

In this paper, MRPPO is used to train policies to guide a low-thrust-enabled SmallSat through three different transfer design scenarios in the Earth-Moon CR3BP: (1) from an L1 Lyapunov orbit to an L2 Lyapunov orbit of equal energy, (2) from an L1 northern halo orbit to an L2 southern halo orbit of equal energy, and (3) from an L1 northern halo orbit to a higher-energy L2 southern halo orbit.

Due to the availability of insights from dynamical systems theory, the first scenario acts as a verification test; particularly, a known natural relationship between the two periodic orbits. The second and third cases present more complicated evaluations in which there are no obvious natural answers. The training policies are assessed on a shared set of perturbed beginning circumstances randomly generated along the initial periodic orbit for each trajectory design scenario to gain insights into the properties of each control scheme. Regions of the multiobjective space are investigated using the maximum, minimum, and mean values of various properties. Insights into the needed propellant quantity, flight time, and trajectory geometry are quickly obtained by autonomously generating a subset of the multi-objective solution space; such knowledge is crucial during spacecraft development.

The dynamics of artificial objects are studied in the framework of the CR3BP. All scientists, including Cipriano, study the behavior of artificial objects in the CR3BP scenario. The European Space Agency picked the LUMIO (Lunar Meteoroid Impact Observer) project for future implementation as part of the SysNova Competition "Lunar CubeSats for Exploration," and Cipriano and colleagues present it. The concept is particularly noteworthy since it not only addresses the trajectory design of a Libration Point Orbit (LPO) mission in the Earth-Moon system, but it also plans to detect meteoroids' impact flashes on the lunar farside, in addition to ground-based observations.

The Earth-Moon systems are the focus of LPO operations. An Extended State Observer, a halo orbit disturbance, and the potential for injection mistakes may all be used to demonstrate how to compute the SRP. The main purpose is to improve station control and administration. In the CR3BP, an essential subject for the institution, Lizy Destrez, and her colleagues have trouble getting together. Finally, this displays how to locate homoclinic and heteroclinic linkages, in addition to the capabilities of a Lunar Gateway Heiliger. SRP acceleration is constant in the CR3BP, which is situated between planar and Lyapunov orbits. These dynamic corridors emerged as a result of the haphazard application of solar sails to natural transportation networks.

Equilibrium points, periodic orbits, and hyperbolic invariant manifolds are among the natural dynamical features found in the Earth-Moon CR3BP that are frequently integrated into the trajectory planning process. The CR3BP describes a multibody system with five equilibrium states.

L1-L5 are the points that exist throughout the system. Several periodic orbit families occur near the CR3BP. Points of equilibrium The Earth-Moon L1 and L2 equilibrium points, which have favorable conditions, are of great importance. Qualities for long-term spaceship installation. For example, the Lunar Gateway is presently planned to be completed by 2025. The planet is in an L2 southern halo orbit. In the Earth-Moon system, MRPPO, a MODRL algorithm, is employed to train different policies in three libration point orbit transfer scenarios. A subset of the multi-objective solution space for a low-thrust-enabled SmallSat linking two periodic orbits via a low-thrust transfer with a specified shape is revealed using this method. As a means of revealing the solution space, eight policies are taught on an L1 Lyapunov orbit to an L2 Lyapunov orbit transfer. Then, using this method, two more complex scenarios are examined, concentrating on exiting an L1 northern halo orbit and reaching an L2 southern halo orbit at different energies.