The expected solution for overcoming this bottleneck includes near-or in-memory computing, [2] which imposes memory functions on processing units or vice versa. A representative example is found in electronics: the memristor, which is the so-called missing fourth circuit element. [3,4] The nonvolatile analog memory functions incorporated with Ohm's law and Kirchhoff 's law in an electronic memristor enable in-memory signal processing for artificial neural networks. [5] On the other hand, the realization of an in-memory processing unit in integrated photonics is not yet successful, especially for all-optical devices, although some insightful memory-related applications have been demonstrated such as stopping light, [6] electro-optical quantum memristors, [7] flip-flop memory, [8] and all-optical integrators. [9] To realize a photonic in-memory processor, its unit element must satisfy the following design criteria: multiple error-robust optical memory states and efficient transitions between them. First, the unit element needs to support optical states that are robust to possible noise or defects in its operation to stably memorize the state of light. Second, similar to a one-transistor-one-memristor (1T1R) configuration in electronics, [5] the unit element should include a toolkit for bidirectional transitions between multiple memory states. Notably, these criteria should be satisfied with a platform that is integratable with all-optical signal processors, for example, allowing state-preserving readout of the memory states. However, these criteria-the robustness of optical states to possible defects and the energy-efficient tunability-are usually contradictory, as proven in topological [10] and disordered photonics [11] and sensing applications. [12] The design of a photonic memristor-analogous unit for in-memory processors is thus not a straightforward task.In this paper, we propose an all-optical building block for photonic in-memory processors by exploiting dynamical parity-time (PT)-symmetric systems. We classify PT-symmetric phases and analyze the Lyapunov stability [13] for a triatomic PT-symmetric system including saturable nonlinearities. The analysis shows the coexistence of topologically protected stable states, that is, oscillation quenching states. We also demonstrate that the building block allows incoherent switching between these oscillation quenching states through all-optical modulations. With the simultaneous achievement of topology-enabled error robustness and all-optical transition in a platform integratable with all-optical signal processors, our study will pave the way for realizing robust photonic in-memory processors.The in-memory processor has played an essential role in overcoming the von Neumann bottleneck, which arises from the partition of memory and a processing unit. Although photonic technologies have recently attracted attention for ultrafast and power-efficient in-memory computing, the realization of an alloptical in-memory processor remains a challenge. This difficulty originates...