I am interested in updating existing layer parameters in Keras (not removing a layer and inserting a new one instead, rather just modifying existing parameters).

I will give an example of a function I'm writing:

def add_filters(self, model):conv_indices=[i for i, layer in enumerate(model.layers) if 'convolution' in layer.get_config()['name']]random_conv_index=random.randint(0, len(conv_indices)-1)factor=2conv_layer=model.layers[random_conv_index]conv_layer.filters=conv_layer.filters * factorprint('new conv layer filters after transform is:', conv_layer.filters)print('just to make sure, its:', model.layers[random_conv_index].filters)return model

so what's basically happening here is me taking a random convolutional layer from my network (all my conv layers have 'convolution' in their name) and trying to double the filters. As far as I know this shouldn't cause any 'compilation issues' with input/output size compatibility in any case.

The thing is, my model doesn't change at all. The 2 print-outs I added in the end print the correct number (double the previous amount of filters). But when I compile the model and print model.summary(), I still see the previous filter amount.

BTW, I'm not constricted to Keras. If anyone has an idea how to pull this off with PyTorch for example I'll also buy it :D

    Well, if you would like to create the architecture of a new model based on an existing model, though with some modifications, you can use to_json and model_from_json() functions. Here is an example:

    model=Sequential()model.add(Conv2D(10, (3,3), input_shape=(100,100,3)))model.add(Conv2D(40, (3,3)))model.summary()

    Model summary:

    Layer (type) Output Shape Param #=================================================================conv2d_12 (Conv2D) (None, 98, 98, 10) 280 _________________________________________________________________conv2d_13 (Conv2D) (None, 96, 96, 40) 3640=================================================================Total params: 3,920Trainable params: 3,920Non-trainable params: 0_________________________________________________________________

    Now we modify the number of filters of the first layer and create a new model based on the modified architecture:

    from keras.models import model_from_jsonmodel.layers[0].filters *=2new_model=model_from_json(model.to_json())new_model.summary()

    New model summary:

    Layer (type) Output Shape Param #=================================================================conv2d_12 (Conv2D) (None, 98, 98, 20) 560 _________________________________________________________________conv2d_13 (Conv2D) (None, 96, 96, 40) 7240=================================================================Total params: 7,800Trainable params: 7,800Non-trainable params: 0_________________________________________________________________

    You can also modify the output of model.to_json() directly without modifying the model instance.


    You can easily use get_weights() method to get the current weights of the convolution layer. It would return a list of two numpy arrays. The first one corresponds to filter weights and the second one corresponds to bias parameters. Then you can use set_weights() method to set the new weights:

    conv_layer=model.layers[random_conv_index]weights=conv_layer.get_weights()weights[0] *=factor # multiply filter weights by `factor`conv_layer.set_weights(weights)

    As a side note, the filters attribute of a convolution layer which you have used in your code corresponds to the number of filters in this layer and not their weights.

    • Thanks for the answer but you didn't get my point. I'm trying to raise the number of filters in the layer - not change the weights. For example - convert a [28,28,64] conv layer to a [28,28,128] conv layer. In my example the first axis the height, second is width and third is depth (didn't include the number of samples as the first axis).– erap129Oct 15 at 10:51
    • @erap129 Oh! Sorry about that. As for changing the number of filter after defining the model I guess you can't do that since it would affect all the following layers, their input shapes and their parameters (i.e. weights). What is the specific scenario that you need to do such a thing? i.e. Why couldn't you create a new model or specify the correct number of filter from the beginning?– todayOct 15 at 11:10
    • I'm trying to create an algorithm to morph conv-nets. I want to go from one architecture to another without rebuilding the whole architecture. Seeing as convolutions go "full depth" anyway - I don't see why a shape mismatch would be a problem. Of course previous weights will be rendered useless but I'll just retrain the whole network. Another option is to keep a "meta" data structure that will contain network definitions, and morph the networks based on that- but I was looking for a cleaner option.– erap129Oct 15 at 12:26
    • @erap129 The problem is that when you change the number of filters say from the 32 to 64 the output shape of that layer changes, i.e from (?,?,32) to (?,?,64). As a result the following layer's input shape has changed and therefore it affects its weights, e.g. if it is a conv layer each filter in this layer previously had a shape of (3,3,32) (assuming kernel size is 3) but now they should have a shape of (3,3,64) and this may propagates further down in the model.– todayOct 15 at 12:44
    • yeah I know, I want to change the shape :) Anyway, I made a hacky solution in which I save the model "description" as a list of strings. Each time I want to add more filters to a layer I simply take the stored description, find the layer I want to add filters to and update its description. Then I make a whole new model based on the updated description. It's messy and I hope I can find a better way, my guess would be that TF/pytorch are up to the job but I'm less experienced in those frameworks.– erap129Oct 15 at 16:02

    Your Answer

     

    By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

    Not the answer you're looking for? Browse other questions tagged or ask your own question.