{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "%matplotlib inline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n# Example for evolutionary regression\n\nExample demonstrating the use of Cartesian genetic programming for\ntwo regression tasks.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# The docopt str is added explicitly to ensure compatibility with\n# sphinx-gallery.\ndocopt_str = \"\"\"\n Usage:\n example_evo_regression.py [--max-generations=]\n\n Options:\n -h --help\n --max-generations= Maximum number of generations [default: 1000]\n\"\"\"\n\nimport functools\nimport warnings\n\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport scipy.constants\nfrom docopt import docopt\n\nimport cgp\n\nargs = docopt(docopt_str)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We first define target functions. For illustration purposes, we\ndefine two functions which present different levels of difficulty\nfor the search.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def f_target_easy(x):\n return x[:, 0] ** 2 + 2 * x[:, 0] * x[:, 1] + x[:, 1] ** 2\n\n\ndef f_target_hard(x):\n return 1.0 + 1.0 / (x[:, 0] + x[:, 1])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we define the objective function for the evolution. It uses the\nmean-squared error between the expression represented by a given\nindividual and the target function evaluated on a set of random\npoints.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def objective(individual, target_function, seed):\n \"\"\"Objective function of the regression task.\n\n Parameters\n ----------\n individual : Individual\n Individual of the Cartesian Genetic Programming Framework.\n target_function : Callable\n Target function.\n\n Returns\n -------\n Individual\n Modified individual with updated fitness value.\n \"\"\"\n if not individual.fitness_is_None():\n return individual\n\n n_function_evaluations = 1000\n\n np.random.seed(seed)\n\n f = individual.to_func()\n y = np.empty(n_function_evaluations)\n x = np.random.uniform(-4, 4, size=(n_function_evaluations, 2))\n for i, x_i in enumerate(x):\n with warnings.catch_warnings(): # ignore warnings due to zero division\n warnings.filterwarnings(\n \"ignore\", message=\"divide by zero encountered in double_scalars\"\n )\n warnings.filterwarnings(\n \"ignore\", message=\"invalid value encountered in double_scalars\"\n )\n try:\n y[i] = f(x_i[0], x_i[1])\n except ZeroDivisionError:\n individual.fitness = -np.inf\n return individual\n\n loss = np.mean((target_function(x) - y) ** 2)\n individual.fitness = -loss\n\n return individual" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we define the main loop of the evolution. To easily execute it\nfor different target functions, we wrap it into a function here. It\ncomprises:\n\n- defining the parameters for the population, the genome of individuals,\n and the evolutionary algorithm.\n- creating a Population instance and instantiating the evolutionary algorithm.\n- defining a recording callback closure for bookkeeping of the progression of the evolution.\n\nFinally, we call the `evolve` method to perform the evolutionary search.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "def evolution(f_target):\n \"\"\"Execute CGP on a regression task for a given target function.\n\n Parameters\n ----------\n f_target : Callable\n Target function\n\n Returns\n -------\n dict\n Dictionary containing the history of the evolution\n Individual\n Individual with the highest fitness in the last generation\n \"\"\"\n population_params = {\"n_parents\": 10, \"seed\": 1234}\n\n genome_params = {\n \"n_inputs\": 2,\n \"n_outputs\": 1,\n \"n_columns\": 12,\n \"n_rows\": 2,\n \"levels_back\": 5,\n \"primitives\": (cgp.Add, cgp.Sub, cgp.Mul, cgp.Div, cgp.ConstantFloat),\n }\n\n ea_params = {\"n_offsprings\": 10, \"tournament_size\": 2, \"mutation_rate\": 0.03, \"n_processes\": 2}\n\n evolve_params = {\"max_generations\": int(args[\"--max-generations\"]), \"termination_fitness\": 0.0}\n\n # create population that will be evolved\n pop = cgp.Population(**population_params, genome_params=genome_params)\n\n # create instance of evolutionary algorithm\n ea = cgp.ea.MuPlusLambda(**ea_params)\n\n # define callback for recording of fitness over generations\n history = {}\n history[\"fitness_parents\"] = []\n\n def recording_callback(pop):\n history[\"fitness_parents\"].append(pop.fitness_parents())\n\n # the objective passed to evolve should only accept one argument,\n # the individual\n obj = functools.partial(objective, target_function=f_target, seed=population_params[\"seed\"])\n\n # Perform the evolution\n cgp.evolve(obj, pop, ea, **evolve_params, print_progress=True, callback=recording_callback)\n return history, pop.champion" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We execute the evolution for the two different target functions\n('easy' and 'hard'). After finishing the evolution, we plot the\nresult and log the final evolved expression.\n\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "if __name__ == \"__main__\":\n width = 9.0\n fig, axes = plt.subplots(2, 2, figsize=(width, width / scipy.constants.golden))\n\n for i, (label, target_function) in enumerate(\n zip([\"easy\", \"hard\"], [f_target_easy, f_target_hard])\n ):\n history, champion = evolution(target_function)\n\n ax_fitness, ax_function = axes[i]\n ax_fitness.set_xlabel(\"Generation\")\n ax_fitness.set_ylabel(\"Fitness\")\n\n history_fitness = np.array(history[\"fitness_parents\"])\n ax_fitness.plot(np.max(history_fitness, axis=1), label=\"Champion\")\n ax_fitness.plot(np.mean(history_fitness, axis=1), label=\"Population mean\")\n\n ax_fitness.set_yscale(\"symlog\")\n ax_fitness.set_ylim(-1.0e4, 0.0)\n ax_fitness.legend()\n\n f_graph = champion.to_func()\n x_0_range = np.linspace(-5.0, 5.0, 20)\n x_1_range = np.ones_like(x_0_range) * 2.0\n # fix x_1 such than 1d plot makes sense\n y = [f_graph(x_0, x_1_range[0]) for x_0 in x_0_range]\n y_target = target_function(np.hstack([x_0_range.reshape(-1, 1), x_1_range.reshape(-1, 1)]))\n\n ax_function.plot(x_0_range, y_target, lw=2, alpha=0.5, label=\"Target\")\n ax_function.plot(x_0_range, y, \"x\", label=\"Champion\")\n ax_function.legend()\n ax_function.set_ylabel(r\"$f(x)$\")\n ax_function.set_xlabel(r\"$x$\")\n\n fig.savefig(\"example_evo_regression.pdf\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.6" } }, "nbformat": 4, "nbformat_minor": 0 }