NEAT Super Mario Bros

I remember seeing a video years ago about neuroevolution in a Super Mario emulator. I liked that you could actually watch the neural network improve as it modified its parameters. I decided that a visual game like Mario iwas perfect for noticing changes when you tune parameters.

I also knew that I didn’t want to use a pre-existing emulator. I didn’t like the constraints that existing emulators might impose on my project. I wanted full control over the observations and the fitness function that is given to the NEAT agent. So I decided to make a Super Mario Bros. clone using the pygame library and then implement the NEAT agent using the neat-python library.

NEAT works by giving an agent (Mario) observations and a fitness function. Observations control what Mario can see. The fitness gives the agent its purpose. The higher the fitness score Mario gets, the better he performs.

I decided to use a library called neat-python. Integrating the library into the game was pretty simple once the game had been made.

def get_observation():
    Mario.centerx / screen_width,
    Mario.centery / screen_height,
    player_velocity_x / max_velocity,
    1.0 if player_on_ground else 0.0,
    1.0 if player_jumping else 0.0,

def evaluate_genome():
    metrics = env.get_metrics(env, genome, config)
    if metrics["x"] > best_x:
        best_x = metrics["x"]
    return best_x

I also added a visual neural network visualization that neat-python made easy to implement. The library preserves all the nodes and connections in a clean list.

The agent learned the first stage of Super Mario Bros way faster than I anticipated.

Looking at the footage I realized that it didn’t really understand what it was doing. It only learned to press buttons in the correct order for the first stage, meaning it exploited the reward function.

I realized that the first stage I had created was too easy for the agent. It could be exploited easily by simply jumping and running right.

I created a second stage with different enemies such as a piranha plant and jumping koopas.

However the plants were a new addition, meaning that new observations were needed so the agent could actually notice them.

def get_observation():
    ground_dist_1, ground_dist_2, ground_dist_3 = get_terrain_ground_probes()
    enemy_present, enemy_dx, enemy_dy = probe_nearest_enemy_ahead()
    pipe_dx, piranha_up = probe_nearest_piranha_pipe_ahead()
    return [
        player_velocity_x / max(1.0, run_max_speed),
        update_velocity / 20.0,
        1.0 if player_on_ground else 0.0,
        ground_dist_1,
        ground_dist_2,
        ground_dist_3,
        probe_wall_distance(),
        probe_headroom(),
        enemy_present,
        enemy_dx,
        enemy_dy,
        pipe_dx,
        piranha_up,
    ]

for _ in range(MAX_STEPS):
        observation = env.get_observation()
        action = input_from_neat(net, observation)
        done = env.step(action)
        metrics = env.get_metrics()
        steps += 1

        fitness -= FITNESS_TIME_PENALTY
        if metrics["vel_x"] > 0:
            fitness += FITNESS_VELOCITY_REWARD * metrics["vel_x"]

        if metrics["x"] > best_x:
            best_x = metrics["x"]
        if metrics["mushroom_amount"] > mushroom_amount:
            mushroom_amount = metrics["mushroom_amount"]
        if metrics["coins"] > coins:
            coins = metrics["coins"]

        if done:
            completed = bool(metrics.get("level_complete", False))
            died = bool(metrics.get("died", False))
            break

    fitness += best_x * FITNESS_X_SCALE
    fitness += mushroom_amount * FITNESS_MUSHROOM_BONUS
    fitness += coins * FITNESS_COIN_BONUS
    
    if completed:
    fitness += FITNESS_GOAL_BONUS
    elif died:
    fitness -= FITNESS_DEATH_PENALTY

However, this wasn’t enough yet to beat the harder level that I had made for the agent.

First, I tried to give the agent observations that were not dependent on a specific level in hopes of generalization. However, after spending hours trying to get it to pass the harder level, I had to switch my approach.

I ended up giving the agent a small grid view of what is around it. This resulted in a better understanding of its environment. However, this would come with a downside, which is worse generalization of the rules and more memorization of a specific stage.

The fitness function remained the same

I never got the agent to generalize the broader rules of Super Mario Bros. However, I still got what I wanted from the project. It taught me how sensitive neuroevolution is to observation design and reward (fitness) shaping. There were many knobs to turn and tweak, but some weighed more than the others.

At the end, I made a third stage for Mario to clear. However, by this time I had been spending hours upon hours just tuning the parameters. I tried to give the newest frontier model: GPT-5.6-sol-max (was the newest and best model when I was working on this project) the repo and asked it to complete the project. It consumed over 10€ and didn’t get Mario to generalize the rules (or even complete it!).

However, in fairness to GPT-5.6-sol, the stage contained a problem which is particularly difficult for NEAT to solve. There was a puzzle, Mario needed to be able to jump onto a platform to jump over a larger wall.