My research focuses on measurement and evaluation in reinforcement learning — developing principled tools for understanding how RL algorithms learn, and applying RL to novel real-world engineering design problems. Full profile on Google Scholar →
Challenges widely-used Policy Information Capacity (PIC) and Policy-Optimal Information Capacity (POIC) metrics for measuring task complexity in robotic RL, demonstrating empirically that they contradict established understanding of robotic task difficulty. Full experimental code is open source.
Represents the learning process of an RL algorithm as a sequence of policies in the manifold of state-action occupancy measures, and uses optimal transport to measure the length and character of the resulting trajectory. Introduces the Effort of Sequential Learning (ESL) metric, which captures exploration behaviour in a way no prior metric did.
Introduces the Controllability Ellipsoid method for evaluating the performance of mobile manipulators across manufacturing task configurations.
Edited volume surveying AI techniques for rational decision-making across engineering and autonomous systems domains.
Khalo MG; Nkhumise RM; Seboyeng LP; Lefoka MP. — South African Patent
Khalo MG; Nkhumise RM. — South African Patent
Alharbi N; Normand E; Nkhumise RM; Stephenson C. — US Provisional Patent
Installable Python package implementing PIC/POIC task-complexity metrics for RL. Includes custom PyBullet robotic environments of escalating difficulty, SAC/DDPG baselines, statistical testing, and HPC scripts.
github.com/nkhumise-rea/task_complexity →RL analysis framework using optimal transport to study exploration behaviour across discrete (Gridworld) and continuous (MountainCar) environments. Fully reproducible with Conda environment spec.
github.com/nkhumise-rea/analysis_of_omt →PyTorch implementations of A2C, DQN, SARSA, DDPG, and SAC across CartPole, GridWorld, and Pendulum. Written for accessibility — lowering the barrier to entry for new RL practitioners.
github.com/nkhumise-rea/Reinforcement-Learning →ROS2 + Gazebo + RViz pick-and-place simulation of the Universal Robots UR10 arm. Written in C++/CMake. Bridges academic RL work with real industrial robotics hardware.
github.com/nkhumise-rea/UR10-Gazebo →Complete open-source mechatronics project: Autodesk Inventor CAD assemblies, circuit schematics, and Arduino C++ firmware controlling stepper, servo, and DC motors for automated maize-meal feeding, stirring, lid actuation, and heat control. Full hardware-software stack, openly licensed.
github.com/nkhumise-rea/lobolas.github.io →