This tutorial introduces PyGAD, an open-source Python room for implementing the familial algorithm and training instrumentality learning algorithms. PyGAD supports 19 parameters for customizing the familial algorithm for various applications.
Within this tutorial we’ll talk 5 different applications of the familial algorithm and build them utilizing PyGAD.
The outline of the tutorial is arsenic follows:
- PyGAD Installation
- Getting Started pinch PyGAD
- Fitting a Linear Model
- Reproducing Images
- 8 Queen Puzzle
- Training Neural Networks
- Training Convolutional Neural Networks
Prerequisites
- Python: Basic knowing of Python programming.
- Deep Learning: Familiarity pinch neural networks, peculiarly CNNs and entity detection.
PyGAD Installation
PyGAD is disposable done PyPI (Python Package Index) and frankincense it tin beryllium installed simply utilizing pip. For Windows, simply usage the pursuing command:
pip instal pygadFor Mac/Linux, usage pip3 alternatively of pip successful the terminal command:
pip3 instal pygadThen make judge the room is installed by importing it from the Python shell:
python import pygadThe latest PyGAD type is presently 2.3.2, which was released connected June 1st 2020. Using the __version__ typical variable, the existent type tin beryllium returned.
import pygad print(pygad.__version__)Now that PyGAD is installed, let’s screen a little preamble to PyGAD.
Getting Started pinch PyGAD
The main extremity of PyGAD is to supply a elemental implementation of the familial algorithm. It offers a scope of parameters that let the personification to customize the familial algorithm for a wide scope of applications. Five specified applications are discussed successful this tutorial.
The afloat archiving of PyGAD is disposable astatine Read the Docs. Here we’ll screen a much digestible breakdown of the library.
In PyGAD 2.3.2 location are 5 modules:
- pygad: The main module comes already imported.
- pygad.nn: For implementing neural networks.
- pygad.gann: For training neural networks utilizing the familial algorithm.
- pygad.cnn: For implementing convolutional neural networks.
- pygad.gacnn: For training convolutional neural networks utilizing the familial algorithm.
Each module has its ain repository connected GitHub, linked below.
- pygad
- pygad.nn
- pygad.gann
- pygad.cnn
- pygad.gacnn
The main module of the room is named pygad. This module has a azygous people named GA. Just create an lawsuit of the pygad.GA people to usage the familial algorithm.
The steps to usage the pygad module are:
- Create the fittingness function.
- Prepare the basal parameters for the pygad.GA class.
- Create an lawsuit of the pygad.GA class.
- Run the familial algorithm.
In PyGAD 2.3.2, the constructor of the pygad.GA people has 19 parameters, of which 16 are optional. The 3 required parameters are:
- num_generations: Number of generations.
- num_parents_mating: Number of solutions to beryllium selected arsenic parents.
- fitness_func: The fittingness usability that calculates the fittingness worth for the solutions.
The fitness_func parameter is what allows the familial algorithm to beryllium customized for different problems. This parameter accepts a user-defined usability that calculates the fittingness worth for a azygous solution. This takes 2 further parameters: the solution, and its scale wrong the population.
Let’s spot an illustration to make this clearer. Assume location is simply a organization pinch 3 solutions, arsenic fixed below.
[221, 342, 213] [675, 32, 242] [452, 23, -212]The assigned usability to the fitness_func parameter must return a azygous number representing the fittingness of each solution. Here is an illustration that returns the sum of the solution.
def fitness_function(solution, solution_idx): return sum(solution)The fittingness values for the 3 solutions are then:
- 776
- 949
- 263
The parents are selected based connected specified fittingness values. The higher the fittingness value, the amended the solution.
For the complete database of parameters successful the pygad.GA people constructor, cheque retired this page.
After creating an lawsuit of the pygad.GA class, the adjacent measurement is to telephone the run() method which goes done the generations that germinate the solutions.
import pygad ga_instance = pygad.GA(...) ga_instance.run()These are the basal steps for utilizing PyGAD. Of people location are further steps that tin beryllium taken arsenic well, but this is the minimum needed.
The adjacent sections talk utilizing PyGAD for respective different usage cases.
Fitting a Linear Model
Assume location is an equation pinch 6 inputs, 1 output, and 6 parameters, arsenic follows:
y = f(w1:w6) = w1x1 + w2x2 + w3x3 + w4x4 + w5x5 + 6wx6Let’s presume that the inputs are (4,-2,3.5,5,-11,-4.7) and the output is 44. What are the values for the 6 parameters to fulfill the equation? The familial algorithm tin beryllium utilized to find the answer.
The first point to do is to hole the fittingness usability arsenic fixed below. It calculates the sum of products betwixt each input and its corresponding parameter. The absolute quality betwixt the desired output and the sum of products is calculated. Because the fittingness usability must beryllium a maximization function, the returned fittingness is adjacent to 1.0/difference. The solutions pinch the highest fittingness values are selected arsenic parents.
function_inputs = [4,-2,3.5,5,-11,-4.7] desired_output = 44 def fitness_func(solution, solution_idx): output = numpy.sum(solution*function_inputs) fittingness = 1.0 / numpy.abs(output - desired_output) return fitnessNow that we’ve prepared the fittingness function, here’s a database pinch different important parameters.
sol_per_pop = 50 num_genes = len(function_inputs) init_range_low = -2 init_range_high = 5 mutation_percent_genes = 1You should besides specify your desired mandatory parameters arsenic you spot fit. After the basal parameters are prepared, the pygad.GA people is instantiated. For accusation astir each of the parameters, mention to this page.
ga_instance = pygad.GA(num_generations=num_generations, num_parents_mating=num_parents_mating, fitness_func=fitness_func, sol_per_pop=sol_per_pop, num_genes=num_genes, init_range_low=init_range_low, init_range_high=init_range_high, mutation_percent_genes=mutation_percent_genes)The adjacent measurement is to telephone the run() method which starts the generations.
ga_instance.run()After the run() method completes, the plot_result() method tin beryllium utilized to show the fittingness values complete the generations.
ga_instance.plot_result()Using the best_solution() method we tin besides retrieve what the champion solution was, its fitness, and its scale wrong the population.
solution, solution_fitness, solution_idx = ga_instance.best_solution() print("Parameters of the champion solution : {solution}".format(solution=solution)) print("Fitness worth of the champion solution = {solution_fitness}".format(solution_fitness=solution_fitness)) print("Index of the champion solution : {solution_idx}".format(solution_idx=solution_idx))Reproducing Images
In this exertion we’ll commencement from a random image (random pixel values), past germinate the worth of each pixel utilizing the familial algorithm.
The tricky portion of this exertion is that an image is 2D aliases 3D, and the familial algorithm expects the solutions to beryllium 1D vectors. To tackle this rumor we’ll usage the img2chromosome() usability defined beneath to person an image to a 1D vector.
def img2chromosome(img_arr): return numpy.reshape(a=img_arr, newshape=(functools.reduce(operator.mul, img_arr.shape)))The chromosome2img() usability (below) tin past beryllium utilized to reconstruct the 2D aliases 3D image backmost from the vector.
def chromosome2img(vector, shape): if len(vector) != functools.reduce(operator.mul, shape): raise ValueError("A vector of magnitude {vector_length} into an array of style {shape}.".format(vector_length=len(vector), shape=shape)) return numpy.reshape(a=vector, newshape=shape)Besides the regular steps for utilizing PyGAD, we’ll request 1 further measurement to publication the image.
import imageio import numpy target_im = imageio.imread('fruit.jpg') target_im = numpy.asarray(target_im/255, dtype=numpy.float)This sample image tin beryllium downloaded here.
Next, the fittingness usability is prepared. This will cipher the quality betwixt the pixels successful the solution and the target images. To make it a maximization function, the quality is subtracted from the sum of each pixels successful the target image.
target_chromosome = gari.img2chromosome(target_im) def fitness_fun(solution, solution_idx): fittingness = numpy.sum(numpy.abs(target_chromosome-solution)) fittingness = numpy.sum(target_chromosome) - fitness return fitnessThe adjacent measurement is to create an lawsuit of the pygad.GA class, arsenic shown below. It is captious to the occurrence of the exertion to usage due parameters. If the scope of pixel values successful the target image is 0 to 255, past the init_range_low and init_range_high must beryllium group to 0 and 255, respectively. The logic is to initialize the organization pinch images of the aforesaid information type arsenic the target image. If the image pixel values scope from 0 to 1, past the 2 parameters must beryllium group to 0 and 1, respectively.
import pygad ga_instance = pygad.GA(num_generations=20000, num_parents_mating=10, fitness_func=fitness_fun, sol_per_pop=20, num_genes=target_im.size, init_range_low=0.0, init_range_high=1.0, mutation_percent_genes=0.01, mutation_type="random", mutation_by_replacement=True, random_mutation_min_val=0.0, random_mutation_max_val=1.0)When the mutation_type statement is group to random, past the default behaviour is to adhd a random worth to each cistron selected for mutation. This random worth is selected from the scope specified by the random_mutation_min_val and random_mutation_max_val parameters.
Assume the scope of pixel values is 0 to 1. If a pixel has the worth 0.9 and a random worth of 0.3 is generated, past the caller pixel worth is 1.2. Because the pixel values must autumn wrong the 0 to 1 range, the caller pixel worth is truthful invalid. To activity astir this issue, it is very important to group the mutation_by_replacement parameter to True. This causes the random worth to switch the existent pixel alternatively than being added to the pixel.
After the parameters are prepared, past the familial algorithm tin run.
ga_instance.run()The plot_result() method tin beryllium utilized to show really the fittingness worth evolves by generation.
ga_instance.plot_result()After the generations complete, immoderate accusation tin beryllium returned astir the champion solution.
solution, solution_fitness, solution_idx = ga_instance.best_solution() print("Fitness worth of the champion solution = {solution_fitness}".format(solution_fitness=solution_fitness)) print("Index of the champion solution : {solution_idx}".format(solution_idx=solution_idx))The champion solution tin beryllium converted into an image to beryllium displayed.
import matplotlib.pyplot result = gari.chromosome2img(solution, target_im.shape) matplotlib.pyplot.imshow(result) matplotlib.pyplot.show()Here is the result.
8 Queen Puzzle
The 8 Queen Puzzle involves 8 chess queens distributed crossed an 8×8 matrix, pinch 1 queen per row. The extremity is to spot these queens specified that nary queen tin onslaught different 1 vertically, horizontally, aliases diagonally. The familial algorithm tin beryllium utilized to find a solution that satisfies specified conditions.
This task is disposable connected GitHub. It has a GUI built utilizing Kivy that shows an 8×8 matrix, arsenic shown successful the adjacent figure.
The GUI has 3 buttons astatine the bottommost of the screen. The usability of these buttons are arsenic follows:
- The Initial Population fastener creates the first organization of the GA.
- The Show Best Solution fastener shows the champion solution from the past procreation the GA stopped at.
- The Start GA fastener starts the GA iterations/generations.
To usage this task commencement by pressing the Initial Population button, followed by the Start GA button. Below is the method called by the Initial Population fastener which, arsenic you mightiness person guessed, generates the first population.
def initialize_population(self, *args): self.num_solutions = 10 self.reset_board_text() self.population_1D_vector = numpy.zeros(shape=(self.num_solutions, 8)) for solution_idx in range(self.num_solutions): initial_queens_y_indices = numpy.random.rand(8)*8 initial_queens_y_indices = initial_queens_y_indices.astype(numpy.uint8) self.population_1D_vector[solution_idx, :] = initial_queens_y_indices self.vector_to_matrix() self.pop_created = 1 self.num_attacks_Label.text = "Initial Population Created."Each solution successful the organization is simply a vector pinch 8 elements referring to the file indices of the 8 queens. To show the queens’ locations connected the screen, the 1D vector is converted into a 2D matrix utilizing the vector_to_matrix() method. The adjacent fig shows the queens connected the screen.
Now that the GUI is built, we’ll build and tally the familial algorithm utilizing PyGAD.
The fittingness usability utilized successful this task is fixed below. It simply calculates the number of attacks that tin beryllium made by each of the 8 queens and returns this arsenic the fittingness value.
def fitness(solution_vector, solution_idx): if solution_vector.ndim == 2: solution = solution_vector else: solution = numpy.zeros(shape=(8, 8)) row_idx = 0 for col_idx in solution_vector: solution[row_idx, int(col_idx)] = 1 row_idx = row_idx + 1 total_num_attacks_column = attacks_column(solution) total_num_attacks_diagonal = attacks_diagonal(solution) total_num_attacks = total_num_attacks_column + total_num_attacks_diagonal if total_num_attacks == 0: total_num_attacks = 1.1 else: total_num_attacks = 1.0/total_num_attacks return total_num_attacksBy pressing the Start GA button, an lawsuit of the pygad.GA people is created and the run() method is called.
ga_instance = pygad.GA(num_generations=500, num_parents_mating=5, fitness_func=fitness, num_genes=8, initial_population=self.population_1D_vector, mutation_percent_genes=0.01, mutation_type="random", mutation_num_genes=3, mutation_by_replacement=True, random_mutation_min_val=0.0, random_mutation_max_val=8.0, callback_generation=callback) ga_instance.run()Here is simply a imaginable solution successful which the 8 queens are placed connected the committee wherever nary queen attacks another.
The complete codification for this task tin beryllium recovered on GitHub.
Training Neural Networks
Among different types of instrumentality learning algorithms, the familial algorithm tin beryllium utilized to train neural networks. PyGAD supports training neural networks and, successful particular, convolutional neural networks, by utilizing the pygad.gann.GANN and pygad.gacnn.GACNN modules. This conception discusses really to usage the pygad.gann.GANN module for training neural networks for a classification problem.
Before building the familial algorithm, the training information is prepared. This illustration builds a web that simulates the XOR logic gate.
data_inputs = numpy.array([[1, 1], [1, 0], [0, 1], [0, 0]]) data_outputs = numpy.array([0, 1, 1, 0])The adjacent measurement is to create an lawsuit of the pygad.gann.GANN class. This people builds a organization of neural networks that each person the aforesaid architecture.
num_inputs = data_inputs.shape[1] num_classes = 2 num_solutions = 6 GANN_instance = pygad.gann.GANN(num_solutions=num_solutions, num_neurons_input=num_inputs, num_neurons_hidden_layers=[2], num_neurons_output=num_classes, hidden_activations=["relu"], output_activation="softmax")After creating the lawsuit of the pygad.gann.GANN class, the adjacent measurement is to create the fittingness function. This returns the classification accuracy for the passed solution.
import pygad.nn import pygad.gann def fitness_func(solution, sol_idx): global GANN_instance, data_inputs, data_outputs predictions = pygad.nn.predict(last_layer=GANN_instance.population_networks[sol_idx], data_inputs=data_inputs) correct_predictions = numpy.where(predictions == data_outputs)[0].size solution_fitness = (correct_predictions/data_outputs.size)*100 return solution_fitnessBesides the fittingness function, the different basal parameters are prepared which we discussed previously.
population_vectors = pygad.gann.population_as_vectors(population_networks=GANN_instance.population_networks) initial_population = population_vectors.copy() num_parents_mating = 4 num_generations = 500 mutation_percent_genes = 5 parent_selection_type = "sss" crossover_type = "single_point" mutation_type = "random" keep_parents = 1 init_range_low = -2 init_range_high = 5After each parameters are prepared, an lawsuit of the pygad.GA people is created.
ga_instance = pygad.GA(num_generations=num_generations, num_parents_mating=num_parents_mating, initial_population=initial_population, fitness_func=fitness_func, mutation_percent_genes=mutation_percent_genes, init_range_low=init_range_low, init_range_high=init_range_high, parent_selection_type=parent_selection_type, crossover_type=crossover_type, mutation_type=mutation_type, keep_parents=keep_parents, callback_generation=callback_generation)The callback_generation parameter refers to a usability that is called aft each generation. In this application, this usability is utilized to update the weights of each the neural networks aft each generation.
def callback_generation(ga_instance): global GANN_instance population_matrices = pygad.gann.population_as_matrices(population_networks=GANN_instance.population_networks, population_vectors=ga_instance.population) GANN_instance.update_population_trained_weights(population_trained_weights=population_matrices)The adjacent measurement is to telephone the run() method.
ga_instance.run()After the run() method completes, the adjacent fig shows really the fittingness worth evolved. The fig shows that a classification accuracy of 100% is reached.
Training Convolutional Neural Networks
Similar to training multilayer perceptrons, PyGAD supports training convolutional neural networks utilizing the familial algorithm.
The first measurement is to hole the training data. The information tin beryllium downloaded from these links:
- dataset_inputs.npy: Data inputs.
- dataset_outputs.npy: Class labels.
The adjacent measurement is to build the CNN architecture utilizing the pygad.cnn module.
import pygad.cnn input_layer = pygad.cnn.Input2D(input_shape=(80, 80, 3)) conv_layer = pygad.cnn.Conv2D(num_filters=2, kernel_size=3, previous_layer=input_layer, activation_function="relu") average_pooling_layer = pygad.cnn.AveragePooling2D(pool_size=5, previous_layer=conv_layer, stride=3) flatten_layer = pygad.cnn.Flatten(previous_layer=average_pooling_layer) dense_layer = pygad.cnn.Dense(num_neurons=4, previous_layer=flatten_layer, activation_function="softmax")After the layers successful the web are stacked, a exemplary is created.
model = pygad.cnn.Model(last_layer=dense_layer, epochs=5, learning_rate=0.01)Using the summary() method, a summary of the exemplary architecture is returned.
----------Network Architecture---------- <class 'cnn.Conv2D'> <class 'cnn.AveragePooling2D'> <class 'cnn.Flatten'> <class 'cnn.Dense'> ----------------------------------------After the exemplary is prepared, the pygad.gacnn.GACNN people is instantiated to create the first population. All the networks person the aforesaid architecture.
import pygad.gacnn GACNN_instance = pygad.gacnn.GACNN(model=model, num_solutions=4)The adjacent measurement is to hole the fittingness function. This calculates the classification accuracy for the passed solution.
def fitness_func(solution, sol_idx): global GACNN_instance, data_inputs, data_outputs predictions = GACNN_instance.population_networks[sol_idx].predict(data_inputs=data_inputs) correct_predictions = numpy.where(predictions == data_outputs)[0].size solution_fitness = (correct_predictions/data_outputs.size)*100 return solution_fitnessThe different parameters are besides prepared.
population_vectors = pygad.gacnn.population_as_vectors(population_networks=GACNN_instance.population_networks) initial_population = population_vectors.copy() num_parents_mating = 2 num_generations = 10 mutation_percent_genes = 0.1 parent_selection_type = "sss" crossover_type = "single_point" mutation_type = "random" keep_parents = -1After each parameters are prepared, an lawsuit of the pygad.GA people is created.
ga_instance = pygad.GA(num_generations=num_generations, num_parents_mating=num_parents_mating, initial_population=initial_population, fitness_func=fitness_func, mutation_percent_genes=mutation_percent_genes, parent_selection_type=parent_selection_type, crossover_type=crossover_type, mutation_type=mutation_type, keep_parents=keep_parents, callback_generation=callback_generation)The callback_generation parameter is utilized to update the web weights aft each generation.
def callback_generation(ga_instance): global GACNN_instance, last_fitness population_matrices = pygad.gacnn.population_as_matrices(population_networks=GACNN_instance.population_networks, population_vectors=ga_instance.population) GACNN_instance.update_population_trained_weights(population_trained_weights=population_matrices)The past measurement is to telephone the run() method.
ga_instance.run()Conclusion
This tutorial introduced PyGAD, an open-source Python room for implementing the familial algorithm. The room supports a number of parameters to customize the familial algorithm for a number of applications.
In this tutorial we utilized PyGAD to build 5 different applications including fitting a linear model, solving the 8 queens puzzle, reproducing images, and training neural networks (both accepted and convolutional). I dream you recovered this tutorial useful, and please consciousness free scope retired successful the comments aliases check retired the docs if you person immoderate questions!